Assistance control device, assistance control method, and program

Abstract

To provide an assistance control device and an assistance control method that provide appropriate assistance to traffic participants.SOLUTION: An assistance control device includes a first acquisition unit that acquires identification information and position information on a user terminal associated with a user, a second acquisition unit that acquires recognition information and position information on a moving object acquired by analyzing an image of the moving object acquired by an imaging device, a memory control unit that performs control to store the recognition information on the moving object acquired by the second acquisition unit in correspondence with the identification information on the user terminal acquired by the first acquisition unit when the position information on the user terminal acquired by the first acquisition unit and the position information on the moving object acquired by the second acquisition unit are consistent, and an assistance control unit that performs control regarding assistance to traffic participants based on the identification information on the user terminal and the recognition information on the moving object that are stored in correspondence under the control of the memory control unit.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an assistance control device, an assistance control method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable people among transportation participants have been gaining momentum. To achieve this, we are focusing on research and development to further improve the safety and convenience of transportation through research and development of preventive safety technology. Patent documents 1 and 2 describe technology related to assigning attributes to vehicles and drivers. [Prior art document] [Patent documents] Patent Document 1: International Publication No. 2019-188429 Patent Document 2: JP 2007-241729 A Summary of the Invention [Problem to be solved by the invention]

[0003] However, in preventive safety technology, the issue is how to support traffic participants. The present application aims to provide appropriate support to traffic participants in order to solve the above issue, and ultimately to contribute to the development of a sustainable transportation system. [Means for solving the problem]

[0004] In a first aspect of the present invention, an assistance control device is provided. The assistance control device includes a first acquisition unit that acquires identification information and location information of a user terminal associated with a user. The assistance control device includes a second acquisition unit that acquires recognition information and location information of the moving object acquired by analyzing an image of the moving object acquired by an imaging device. The assistance control device includes a storage control unit that performs control to store the recognition information of the moving object acquired by the second acquisition unit in association with the identification information of the user terminal acquired by the first acquisition unit when the location information of the user terminal acquired by the first acquisition unit and the location information of the moving object acquired by the second acquisition unit match. The assistance control device includes an assistance control unit that performs control regarding assistance for a traffic participant based on the identification information of the user terminal and the recognition information of the moving object that are associated and stored under the control of the storage control unit.

[0005] In the above assistance control device, even if the second acquisition unit is unable to acquire recognition information of the moving object, when identification information of the user terminal is acquired by the first acquisition unit, the assistance control unit may provide assistance to a traffic participant based on the identification information of the user terminal and the recognition information of the moving object, which are associated and stored under the control of the memory control unit.

[0006] In any of the above assistance control devices, the assistance control unit may perform control to notify traffic participants other than the user of the recognition information.

[0007] In any of the above assistance control devices, the recognition information may include information indicating a color of the moving object.

[0008] In any of the above assistance control devices, the recognition information may include information indicating a type of the moving object.

[0009] In any of the above assistance control devices, the recognition information may include information indicating a size of the moving object.

[0010] In any of the above-mentioned assistance control devices, the memory control unit may perform control to delete the identification information of the user terminal and the recognition information of the moving object, which are stored in correspondence with each other, every time a predetermined time elapses.

[0011] In any of the above-mentioned assistance control devices, the memory control unit may perform control to update the identification information of the user terminal and the recognition information of the moving object, which are stored in correspondence with each other, when the location information of the user terminal newly acquired by the first acquisition unit and the location information of the moving object newly acquired by the second acquisition unit are consistent.

[0012] In any of the above assistance control devices, the moving object may include at least one of a pedestrian and a vehicle.

[0013] In any of the above assistance control devices, the second acquisition unit may further acquire movement characteristic information of the moving object and position information of the moving object acquired by analyzing an image of the moving object acquired by an imaging device. The storage control unit may perform control to store the recognition information and movement characteristic information of the moving object acquired by the second acquisition unit in association with the identification information of the user terminal acquired by the first acquisition unit when the position information of the user terminal acquired by the first acquisition unit and the position information of the moving object acquired by the second acquisition unit match. The assistance control unit may perform control regarding assistance to a traffic participant based on the identification information of the user terminal and the recognition information and movement characteristic information of the moving object that are associated and stored under the control of the storage control unit.

[0014] In any of the above assistance control devices, the moving object may include a pedestrian. The recognition information may include information indicating a color of the pedestrian. The movement characteristic information may include information indicating a behavioral tendency of the pedestrian regarding passage of the pedestrian on a road.

[0015] In any of the above assistance control devices, the moving object may include a vehicle. The recognition information may include information indicating a color of the vehicle. The movement characteristic information may include information regarding dangerous driving of the vehicle.

[0016] In a second aspect of the present invention, there is provided an assistance control method. The assistance control method includes a step of acquiring identification information and position information of a user terminal associated with a user. The assistance control method includes a step of acquiring recognition information and position information of the moving object acquired by analyzing an image of the moving object acquired by an imaging device. The assistance control method includes a step of performing control for storing the recognition information of the moving object in association with the identification information of the user terminal when the position information of the user terminal and the position information of the moving object match. The assistance control method includes a step of performing control regarding assistance to a traffic participant based on the identification information of the user terminal and the recognition information of the moving object that are stored in association with each other by the step of performing control for storing.

[0017] In a third aspect of the present invention, there is provided a program for causing a computer to function as any one of the above-mentioned assistance control devices.

[0018] The above summary of the invention does not list all of the features of the present invention. Also, subcombinations of these features may also be inventions. [Brief description of the drawings]

[0019] [Figure 1] 1 illustrates a schematic diagram of a scenario in which the support system 10 is used. [Diagram 2] 2 shows the functional configuration of the support device 60. [Diagram 3] 13 shows an example of a data structure of vehicle attribute information including recognition information and movement characteristic information stored under the control of the storage control unit 240. [Figure 4] 13 shows an example of a data structure of vehicle attribute information including recognition information and movement characteristic information stored under the control of the storage control unit 240. [Diagram 5] 13A and 13B show schematic diagrams of a situation in which assistance is provided to a traffic participant vehicle by the assistance control unit 260. [Figure 6] 13A and 13B are schematic diagrams illustrating other situations in which the assistance control unit 260 provides assistance to a traffic participant. [Figure 7] An example of an execution sequence of processes related to the assistance control method performed by the imaging device 70, the user terminal 82, the in-vehicle processing device 40a provided in the vehicle 20a, and the assistance device 60 is shown. [Figure 8] An example of an execution sequence of processes related to the assistance control method performed in the user terminal 82b, the assistance device 60, and the user terminal 82c is shown. [Figure 9] An example of an execution sequence of processes related to the assistance control method performed in the user terminal 82a, the assistance device 60, and the in-vehicle processing device 40c provided in the vehicle 20c is shown. [Figure 10] An example of a computer 2000 is shown. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0021] 1 is a schematic diagram showing a usage scenario of the assistance system 10. The assistance system 10 includes an assistance device 60, an imaging device 70, and a traffic light 50.

[0022] In Fig. 1, a user 80a, a vehicle 20a, and a vehicle 20b are traffic participants. The vehicle 20a is equipped with an on-board processing device 40a. The user terminal 82b is a user terminal carried by a passenger of the vehicle 20b. The user terminal 82a is a terminal carried by the user 80a. In this embodiment, the user 80a is a pedestrian. The user 80a is an example of a moving object.

[0023] In this embodiment, the user terminal 82a and the user terminal 82b may be collectively referred to as "user terminal 82." The user 80a and the user 80b may be collectively referred to as "user 80."

[0024] The vehicle 20a is a vehicle that travels on a road 90. The vehicle 20a is an example of a moving object. The on-board processing device 40a includes various sensors, such as a position sensor including a Global Navigation Satellite System (GNSS) receiver, a speed sensor such as a vehicle speed sensor, and an imaging device that captures images of the surroundings of the vehicle 20a. The on-board processing device 40a has a function of processing information acquired by the various sensors provided in the on-board processing device 40a, and a function of communicating with the assistance device 60. The on-board processing device 40a provides an advanced driver assistance system (ADAS) function provided in the vehicle 20a.

[0025] The user terminal 82a and the user terminal 82b are, for example, mobile terminals such as smartphones. The user terminal 82a and the user terminal 82b periodically transmit current position information of each of the user terminals detected by a position sensor including a GNSS receiver, together with identification information of the user terminal 82a and the user terminal 82b, to the support device 60.

[0026] The imaging device 70 is an imaging device fixedly installed on a traffic infrastructure. The imaging device 70 acquires the positions of the vehicle 20a, the vehicle 20b, and the user 80a that are present within the imaging range of the imaging device 70 by analyzing the captured image, and transmits the acquired positions of the vehicle 20a, the vehicle 20b, and the user 80a to the assistance device 60.

[0027] The support device 60 receives information transmitted from the in-vehicle processing device 40a, the user terminal 82a, and the imaging device 70 via mobile communication. The support device 60 may receive information transmitted from the in-vehicle processing device 40a, the user terminal 82a, and the imaging device 70 via mobile communication and a communication line such as the Internet or a dedicated line.

[0028] The imaging device 70 analyzes the image captured by the imaging device 70 to identify the positions of the vehicle 20b and the user 80a. The imaging device 70 further analyzes the image captured by the imaging device 70 to acquire the color, size, and type of the vehicle 20b. The color, size, and type are examples of recognition information of the vehicle 20b. The recognition information is, for example, visually recognizable information of the vehicle 20b. The imaging device 70 further analyzes the image captured by the imaging device 70 to acquire the color of clothing, age, and height of the user 80a. The color of clothing, age, and height are examples of recognition information of the user 80a. The recognition information is, for example, visually recognizable information of the user 80a.

[0029] The imaging device 70 further analyzes the image captured by the imaging device 70 to obtain driving characteristic information of the vehicle 20b. The driving characteristic information includes information indicating whether or not the vehicle 20b is prone to dangerous driving. For example, the imaging device 70 identifies the indication state of the traffic signal of the traffic light 50, and when it is detected from the image that the vehicle 20b is running while ignoring the traffic signal, it determines that the vehicle 20b is prone to dangerous driving. When the vehicle 20b turns right or left without activating the turn indicator of the vehicle 20b, it determines that the vehicle 20b is prone to dangerous driving. In this way, the imaging device 70 determines that the vehicle 20b is prone to dangerous driving when the vehicle 20b is running in a manner that does not comply with traffic rules. In addition, the imaging device 70 determines that the vehicle 20b is prone to dangerous driving when the vehicle 20b turns right or left while ignoring oncoming vehicles, rear vehicles, and / or nearby pedestrians. The driving characteristic information is an example of the movement characteristic information of the vehicle 20b.

[0030] The imaging device 70 further analyzes the image captured by the imaging device 70 to obtain walking characteristic information of the user 80a. The running characteristic information is information indicating the behavioral tendency of the user 80a when passing through a road. For example, the imaging device 70 identifies the indication state of the traffic signal of the traffic light 50, and when it is detected from the image that the user 80a is crossing the road 90 without following the traffic signal from the history of the position information of the user 80a, it determines that the user 80a tends to cross the road 90 without following the indication state of the traffic light 50. The imaging device 70 recognizes road markings and / or traffic signs on the road 90, and when it is detected from the image that the user 80a is passing through without following the road markings and / or traffic signs from the history of the position information of the user 80a, it determines that the user 80a tends to pass through without following the road markings and / or traffic signs. For example, the imaging device 70 determines whether or not a pedestrian tends to cross a road at a place where a crosswalk is not provided. Furthermore, the imaging device 70 judges whether the user 80a tends to cross the road 90 at a place where no crosswalk is provided near a crosswalk. For example, the imaging device 70 judges whether the user 80a tends to cross the road 90 diagonally near a crosswalk. In addition, the imaging device 70 judges whether the user 80a tends to start crossing the road within a time shorter than a predetermined time when the indication state of the traffic signal 50 switches from a proceed indication to a stop indication. In this way, the imaging device 70 judges that the user 80a tends to perform dangerous behavior when walking in a manner that does not comply with traffic rules. The walking characteristic information is an example of the movement characteristic information of the user 80a.

[0031] The position information of the vehicle 20b and the user 80a, the recognition information of the vehicle 20b and the user 80a, the driving characteristic information of the vehicle 20b, and the walking characteristic information of the user 80a acquired by the imaging device 70 are transmitted from the imaging device 70 to the support device 60. The in-vehicle processing device 40a may acquire the position information of the vehicle 20b and the user 80a, the recognition information of the vehicle 20b and the user 80a, the driving characteristic information of the vehicle 20b, and the walking characteristic information of the user 80a from the images captured by the imaging device included in the in-vehicle processing device 40a, and transmit the acquired information to the support device 60.

[0032] The support device 60 compares the history of position information acquired from the imaging device 70 and / or the in-vehicle processing device 40a with the history of position information acquired from the user terminal 82a, performs matching based on the similarity of position and displacement, and extracts the recognition information and walking characteristic information of the user terminal 82a from the recognition information and walking characteristic information acquired from the imaging device 70 and / or the in-vehicle processing device 40a. The support device 60 stores the extracted recognition information and walking characteristic information of the user terminal 82a in association with the identification information received from the user terminal 82a.

[0033] The support device 60 compares the history of position information acquired from the imaging device 70 and / or the in-vehicle processing device 40a with the history of position information acquired from the user terminal 82b, performs matching based on the similarity of positions and displacements, and extracts the recognition information and driving characteristic information of the vehicle 20b from the recognition information and walking characteristic information acquired from the imaging device 70 and / or the in-vehicle processing device 40a. The support device 60 stores the extracted recognition information and driving characteristic information of the vehicle 20b in association with the identification information received from the user terminal 82b.

[0034] In this way, the assistance device 60 can associate the identification information of the user terminal 82b with the recognition information and driving characteristic information of the vehicle 20b. As a result, when the vehicle 20b is traveling in a place where the imaging device 70 is not installed or where there are no other vehicles with ADAS functions in the vicinity, the assistance device 60 can provide assistance to traffic participants based on the recognition information and driving characteristic information upon receiving the identification information of the user terminal 82b. For example, when the vehicle 20b is a red vehicle and tends to drive dangerously, the assistance device 60 can alert users of the user terminals around the vehicle 20b by outputting warning information such as "Be careful as a red vehicle is approaching" to the user terminals around the vehicle 20b by voice or the like.

[0035] Similarly, the support device 60 can associate the identification information of the user terminal 82a with the recognition information and walking characteristic information of the user 80a. As a result, when the user 80a is walking in a place where the imaging device 70 is not installed or where there are no other vehicles having an ADAS function, the support device 60 can provide support to the traffic participants based on the recognition information and the driving characteristic information when receiving the identification information of the user terminal 82b. For example, when the user 80a is wearing red clothes and tends to cross the road ignoring road markings and traffic signs, the support device 60 can alert the passengers of the vehicles around the user 80a by outputting warning information to the effect that "Please be careful as a pedestrian wearing red clothes is nearby."

[0036] 2 shows a functional configuration of the assistance device 60. The assistance device 60 includes a assistance control device 200, a communication device 290, and a storage device 280.

[0037] The communication device 290 is responsible for communication between the in-vehicle processing device 40a, user terminals such as the user terminal 82a and the user terminal 82b, and the imaging device 70, and the assistance device 60, based on the control of the assistance control device 200. The assistance control device 200 is realized by including a circuit such as an arithmetic processing device including a processor. The assistance control device 200 may be realized by a microcomputer including a CPU, a ROM, a RAM, an I / O, a bus, and the like. The storage device 280 is realized by including a non-volatile storage medium. The assistance control device 200 performs processing using information stored in the storage device 280. The storage device 280 may store information related to traffic, such as map information used for control by the assistance control device 200.

[0038] The assistance control device 200 includes an acquisition unit 202, a storage control unit 240, and an assistance control unit 260. The acquisition unit 202 includes a first acquisition unit 210 and a second acquisition unit 220. Note that the assistance device 60 may not include some of the functions of the functional configuration shown in FIG.

[0039] The first acquisition unit 210 acquires identification information and location information of a user terminal 82 associated with a user 80. The first acquisition unit 210 acquires the identification information and location information of the user terminal 82 transmitted from the user terminal 82 and received by the communication device 290.

[0040] The second acquisition unit 220 acquires the recognition information and the position information of the moving object acquired by analyzing the image of the moving object acquired by the imaging device. Here, the imaging device is the imaging device 70 or the imaging device provided in the in-vehicle processing device 40a. The second acquisition unit 220 acquires the recognition information and the position information of the moving object acquired by the imaging device 70 or the imaging device provided in the in-vehicle processing device 40a.

[0041] When the position information of the user terminal 82 acquired by the first acquisition unit 210 and the position information of the moving object acquired by the second acquisition unit 220 match, the storage control unit 240 performs control to store the recognition information of the moving object acquired by the second acquisition unit 220 in association with the identification information of the user terminal 82 acquired by the first acquisition unit 210. For example, when the degree of agreement between the position information of the user terminal 82 and the position information of the moving object is higher than a predetermined value, the storage control unit 240 performs control to store the recognition information of the moving object in association with the identification information of the user terminal 82. For example, the storage control unit 240 stores the recognition information of the moving object in the storage device 280 in association with the identification information of the user terminal 82.

[0042] The assistance control unit 260 performs control regarding assistance for the traffic participant based on the identification information of the user terminal 82 and the recognition information of the moving object, which are associated and stored under the control of the storage control unit 240. For example, even if the second acquisition unit 220 cannot acquire the recognition information of the moving object, when the identification information of the user terminal 82 is acquired by the first acquisition unit 210, the assistance control unit 260 performs assistance for the traffic participant based on the identification information of the user terminal 82 and the recognition information of the moving object, which are associated and stored under the control of the storage control unit 240. As an example, the assistance control unit 260 performs control to notify traffic participants other than the user 80 of the recognition information.

[0043] The recognition information may include information indicating a color of the moving object, information indicating a type of the moving object, and information indicating a size of the moving object.

[0044] The storage control unit 240 may perform control to delete the identification information of the user terminal 82 and the recognition information of the moving object, which are stored in association with each other, every time a predetermined time elapses.

[0045] When the location information of the user terminal 82 newly acquired by the first acquisition unit 210 and the location information of the moving object newly acquired by the second acquisition unit 220 are consistent, the memory control unit 240 may perform control to update the identification information of the user terminal 82 and the recognition information of the moving object, which are stored in correspondence with each other.

[0046] The second acquisition unit 220 acquires the movement characteristic information and the position information of the moving object acquired by analyzing the image of the moving object acquired by the imaging device. Here, the imaging device is the imaging device 70 or the imaging device provided in the in-vehicle processing device 40a. The second acquisition unit 220 acquires the movement characteristic information and the position information of the moving object acquired by the imaging device 70 or the imaging device provided in the in-vehicle processing device 40a.

[0047] When the position information of the user terminal 82 acquired by the first acquisition unit 210 and the position information of the moving object acquired by the second acquisition unit 220 match, the storage control unit 240 performs control to store the movement characteristic information of the moving object acquired by the second acquisition unit 220 in association with the identification information of the user terminal 82 acquired by the first acquisition unit 210. For example, when the degree of agreement between the position information of the user terminal 82 and the position information of the moving object is higher than a predetermined value, the storage control unit 240 performs control to store the movement characteristic information of the moving object in association with the identification information of the user terminal 82. For example, the storage control unit 240 causes the storage device 280 to store the movement characteristic information of the moving object in association with the identification information of the user terminal 82.

[0048] The assistance control unit 260 performs control regarding assistance for the traffic participant based on the identification information of the user terminal 82 and the movement characteristic information of the moving object, which are associated and stored under the control of the storage control unit 240. For example, even if the second acquisition unit 220 cannot acquire the movement characteristic information of the moving object, when the identification information of the user terminal 82 is acquired by the first acquisition unit 210, the assistance control unit 260 performs assistance for the traffic participant based on the identification information of the user terminal 82 and the movement characteristic information of the moving object, which are associated and stored under the control of the storage control unit 240. As an example, the assistance control unit 260 performs control to notify traffic participants other than the user 80 of the movement characteristic information of the moving object.

[0049] The storage control unit 240 may perform control to update the identification information of the user terminal 82 and the movement characteristic information of the moving object, which are stored in association with each other, when the position information of the user terminal 82 newly acquired by the first acquisition unit 210 and the position information of the moving object newly acquired by the second acquisition unit 220 match. The storage control unit 240 may perform control to delete the identification information of the user terminal 82 and the movement characteristic information of the moving object, which are stored in association with each other, every time a predetermined time elapses.

[0050] The moving object includes at least one of a pedestrian and a vehicle. In this embodiment, the moving object is a user 80a and / or a vehicle 20b.

[0051] When the moving object is a pedestrian, the movement characteristic information may include information indicating a behavioral tendency of the pedestrian regarding passage of the road by the pedestrian. The movement characteristic information may include information indicating whether or not the pedestrian tends to cross the road at a place where a pedestrian crossing is not provided. The movement characteristic information may include information indicating whether or not the pedestrian tends to cross the road at a place where a pedestrian crossing is not provided in the vicinity of a pedestrian crossing. The movement characteristic information may include information indicating whether or not the pedestrian tends to cross the road without following the indication state of the traffic signal. The movement characteristic information may include information indicating whether or not the pedestrian tends to start crossing the road within a time shorter than a predetermined time when the indication of the traffic signal switches from a proceed indication to a stop indication.

[0052] If the moving object is a vehicle, the movement characteristic information may include information regarding dangerous driving of the vehicle.

[0053] The second acquisition unit 220 may acquire the movement characteristic information of the moving object and the position information of the moving object acquired by analyzing the image of the moving object acquired by the imaging device. When the position information of the user terminal 82 acquired by the first acquisition unit 210 and the position information of the moving object acquired by the second acquisition unit 220 match, the storage control unit 240 may perform control to store the recognition information and the movement characteristic information of the moving object acquired by the second acquisition unit 220 in association with the identification information of the user terminal 82 acquired by the first acquisition unit 210. The assistance control unit 260 may perform control regarding assistance to the traffic participant based on the identification information of the user terminal 82 and the recognition information and the movement characteristic information of the moving object that are associated and stored under the control of the storage control unit 240.

[0054] When the moving object is a pedestrian, the recognition information may include information indicating the color of the pedestrian, and the movement characteristic information may include information indicating the behavioral tendency of the pedestrian regarding passage on the road.

[0055] If the moving object is a vehicle, the recognition information may include information indicating the color of the vehicle, and the movement characteristic information may include information regarding dangerous driving of the vehicle.

[0056] 3 shows an example of a data structure of vehicle attribute information including recognition information and movement characteristic information stored under the control of the storage control unit 240. The vehicle attribute information includes a terminal ID, time, color, type, size, and driving characteristics.

[0057] The terminal ID is identification information of the user terminal 82 stored in association with the vehicle. The time, color, type, and size are examples of vehicle recognition information. The driving characteristics are an example of vehicle movement characteristic information.

[0058] The time may be, for example, the time when information including the terminal ID, time, color, type, size, and driving characteristics is stored in storage device 280.

[0059] The color may be information indicating a typical color recognized from the vehicle. The color may be information indicating the color of the body of the vehicle. The type may be information indicating the vehicle type, or may be information indicating the distinction between a truck, a bus, and a passenger car, or may be information indicating the distinction between a regular car, a large car, etc. The size may be information indicating the length of the vehicle. The size may be information indicating the height, width, etc. of the vehicle.

[0060] In this embodiment, the memory control unit 240 matches the location information of the user terminal 82b with the location information of the vehicle 20b obtained from image analysis, thereby associating the user terminal 82b with the vehicle 20b, and stores the identification information of the user terminal 82b in association with the color, type, size, and driving characteristics of the vehicle 20b as attribute information in the memory device 280.

[0061] The driving characteristics include information indicating whether or not dangerous driving tends to occur in the vehicle 20b. The driving characteristics may include a plurality of data items as data items related to dangerous driving. For example, the driving characteristics may include a plurality of data items related to compliance with rules and safety, such as not obeying traffic signals, not paying attention to pedestrians crossing the road, not turning on a turn signal when turning right or left, not paying attention to oncoming vehicles when turning right or left, not paying attention to pedestrians when turning right or left, not paying attention to following vehicles when turning right or left, and applying sudden brakes when turning right or left.

[0062] The recognition information of the vehicle 20b may be transmitted from each of the imaging device 70 and the in-vehicle processing device 40a. In addition, the recognition information of the vehicle 20b may be transmitted from another imaging device installed on a traffic infrastructure in another location. Furthermore, the recognition information of the vehicle 20b may also be transmitted from an imaging device of another vehicle in another location. Therefore, there may be differences in the recognition results of the color, type, and size by each imaging device.

[0063] Therefore, the storage control unit 240 may update the recognition information stored in association with the specific terminal ID every time new recognition information of a vehicle associated with the specific terminal ID is acquired. The storage control unit 240 may update the recognition information associated with the specific terminal ID to the latest recognition information. When there is a difference in the colors recognized by the multiple imaging devices, the storage control unit 240 may update the color associated with the specific terminal ID to the color recognized by the most imaging devices. Similarly, when there is a difference in the types recognized by the multiple imaging devices, the storage control unit 240 may update the type associated with the specific terminal ID to the type recognized by the most imaging devices.

[0064] If there is a difference in the sizes recognized by the multiple imaging devices, the storage control unit 240 may update the size associated with the specific terminal ID to the average value of the sizes recognized by the multiple imaging devices. If there is a difference in the sizes recognized by the multiple imaging devices, the storage control unit 240 may update the size associated with the specific terminal ID to the maximum value of the sizes recognized by the multiple imaging devices.

[0065] When the user terminal 82b is a terminal that can be carried by the user, the user terminal 82b is not necessarily always mounted on the vehicle 20b. Therefore, the storage control unit 240 may delete the attribute information according to the elapsed time from the time included in the attribute information. For example, the storage control unit 240 deletes the attribute information when a predetermined time has elapsed from the time included in the attribute information. As an example, the storage control unit 240 deletes the attribute information when 24 hours or more have elapsed from the time included in the attribute information.

[0066] 4 shows an example of a data structure of user attribute information including recognition information and movement characteristic information stored under the control of the storage control unit 240. The user attribute information includes a terminal ID, time, color, age, height, and walking characteristics.

[0067] The terminal ID is identification information of the user terminal 82 stored in association with the user. The time, color, age, and height are examples of vehicle recognition information. The walking characteristics are an example of user movement characteristic information.

[0068] The time may be the time when the attribute information is stored in storage device 280 .

[0069] The color may be information indicating a typical color recognized by the user. The color may be information indicating the color of the user's clothing, such as the clothes worn by the user. The age may be information indicating the age group of the user. The age may be information indicating an estimated age group of the user recognized from the image. The age may be information indicating the age group of the user pre-registered in the user terminal identified by the terminal ID. The height may be information indicating the height of the user.

[0070] In this embodiment, the storage control unit 240 matches the position information of the user terminal 82a with the position information of the user 80a obtained from image analysis, thereby associating the user terminal 82a with the user 80a, and stores the identification information of the user terminal 82a in association with the color, age, height, and walking characteristics in the storage device 280 as attribute information.

[0071] The walking characteristics are information indicating the behavioral tendency of the user 80a when passing through a road. The walking characteristics may include a plurality of data items as data items related to the behavioral tendency. For example, the walking characteristics may include a plurality of data items related to compliance with rules and safety, such as whether the user 80a tends to cross the road without following the instruction state of the traffic signal, whether the user 80a tends to pass through without following road markings and / or traffic signs, whether a pedestrian tends to cross the road at a place where there is no crosswalk, whether the user 80a tends to cross the road at a place where there is no crosswalk near the crosswalk, whether the user 80a tends to cross the road diagonally near the crosswalk, and whether the user 80a tends to suddenly start crossing the road when the instruction state of the traffic signal switches from a proceed instruction to a stop instruction.

[0072] The recognition information of the user 80a may be transmitted from each of the imaging device 70 and the in-vehicle processing device 40a. In addition, the recognition information of the user 80a may be transmitted from another imaging device installed on a traffic infrastructure in another location. Furthermore, the recognition information of the user 80a may also be transmitted from an imaging device of another vehicle in another location. Therefore, there may be differences in the recognition results of color, age, and height by each imaging device.

[0073] Therefore, the storage control unit 240 may update the recognition information stored in association with the specific terminal ID every time new recognition information of a user associated with the specific terminal ID is acquired. The storage control unit 240 may update the recognition information associated with the specific terminal ID to the latest recognition information. When there is a difference between colors recognized by a plurality of imaging devices, the storage control unit 240 may update the color associated with the specific terminal ID to the color recognized by the most imaging devices.

[0074] When there is a difference in the ages recognized by the multiple imaging devices, the storage control unit 240 may update the age associated with the specific terminal ID to the average value of the ages recognized by the multiple imaging devices. When there is a difference in the ages recognized by the multiple imaging devices, the storage control unit 240 may update the age associated with the specific terminal ID to the maximum value of the ages recognized by the multiple imaging devices. Similarly, when there is a difference in the heights recognized by the multiple imaging devices, the storage control unit 240 may update the height associated with the specific terminal ID to the average value of the heights recognized by the multiple imaging devices. When there is a difference in the heights recognized by the multiple imaging devices, the storage control unit 240 may update the height associated with the specific terminal ID to the maximum value of the heights recognized by the multiple imaging devices.

[0075] When the user terminal 82a is a terminal that can be carried by a user, the user terminal 82a is not necessarily always carried by a specific user 80a. Therefore, the storage control unit 240 may delete the attribute information according to the elapsed time from the time included in the attribute information. For example, the storage control unit 240 deletes the attribute information when a predetermined time has elapsed from the time included in the attribute information. As an example, the storage control unit 240 deletes the attribute information when 24 hours or more have elapsed from the time included in the attribute information.

[0076] 5 is a schematic diagram showing a situation in which assistance is provided to a traffic participant vehicle by the assistance control unit 260. Here, it is assumed that attribute information of the vehicle 20b has already been stored in the storage device 280. In other words, it is assumed that the recognition information and driving characteristic information of the vehicle 20b acquired by the imaging device 70 and / or the in-vehicle processing device 40a have been stored in the storage device 280 in association with the identification information of the user terminal 82b.

[0077] The user terminal 82b transmits the position information and the identification information to the assistance device 60. The assistance control unit 260 acquires the recognition information and the driving characteristic information stored in the storage device 280 in association with the identification information transmitted from the user terminal 82b, based on the identification information transmitted from the user terminal 82b. As a result, the assistance control unit 260 determines that a vehicle having the recognition information and the driving characteristic information is present at the position transmitted from the user terminal 82b.

[0078] The user terminal 82c transmits the position information and the identification information to the assistance device 60. The assistance control unit 260 provides traffic assistance to the user terminal 82c based on the history of the position information transmitted from the user terminal 82b, the history of the position information transmitted from the user terminal 82c, and the recognition information and driving characteristic information associated with the identification information of the user terminal 82b.

[0079] As an example, assume that the user terminal 82c is about to cross the road 90 as shown in FIG. 5. When the assistance control unit 260 determines that the user 80c is about to cross the road 90 based on the history of the position information of the user terminal 82b and the user terminal 82c and the map information, and determines that the user terminal 82b and the user terminal 82c are approaching within a predetermined distance, if the driving characteristic information associated with the identification information of the user terminal 82b includes information indicating a tendency to neglect pedestrians crossing the road, the assistance control unit 260 transmits assistance information to the user terminal 82c instructing the user terminal 82c to output warning information. The assistance information for the user terminal 82c may be information instructing the user terminal 82c to output warning information to the effect that "beware of vehicles" by voice. The assistance information for the user terminal 82c may be information instructing the user terminal 82c to output warning information to the effect that "beware of red vehicles" by voice based on the recognition information. The assistance information to the user terminal 82c may be information instructing the user terminal 82c to further output warning information such as "Please be careful when crossing the road" based on driving characteristic information associated with the identification information of the user terminal 82b.

[0080] In addition, the assistance control unit 260 may determine whether the user terminal 82b and the user terminal 82c approach each other within the predetermined time by estimating the respective positions of the user terminal 82b and the user terminal 82c when a predetermined time has elapsed. When the assistance control unit 260 determines that the user terminal 82b and the user terminal 82c approach each other within the predetermined time, the assistance control unit 260 may transmit assistance information to the user terminal 82c instructing the user terminal 82c to output warning information by voice, such as "Be careful, a vehicle is approaching." The assistance information may be information instructing the user terminal 82c to output warning information by voice, such as "Be careful, a red vehicle is approaching," based on the recognition information. When determining whether the user terminal 82b and the user terminal 82c approach each other within the predetermined time, the assistance control unit 260 may set the "predetermined time" based on the driving characteristic information associated with the identification information of the user terminal 82b. For example, when the driving characteristic information associated with the identification information of the user terminal 82b indicates that there is a tendency for dangerous driving to occur, the assistance control unit 260 may set the "predetermined time" to be long.

[0081] 6 is a schematic diagram showing another situation in which the assistance control unit 260 provides assistance to a traffic participant. Here, it is assumed that the attribute information of the user 80a has already been stored in the storage device 280. In other words, it is assumed that the recognition information and driving characteristic information of the user 80a acquired by the imaging device 70 and / or the in-vehicle processing device 40a have been stored in the storage device 280 in association with the identification information of the user terminal 82a.

[0082] 6, the traffic participants are a user 80a and a vehicle 20c. The vehicle 20c includes an on-board processing device 40c. The on-board processing device 40c has a function of communicating with the assistance device 60, similar to the on-board processing device 40a of the vehicle 20a.

[0083] The user terminal 82a transmits the position information and the identification information to the support device 60. The support control unit 260 acquires the recognition information and the walking characteristic information stored in the storage device 280 in association with the identification information transmitted from the user terminal 82a, based on the identification information transmitted from the user terminal 82a. As a result, the support control unit 260 determines that the user having the recognition information and the walking characteristic information is present at the position transmitted from the user terminal 82a.

[0084] The in-vehicle processing device 40c transmits the position information to the assistance device 60. The assistance control unit 260 provides traffic assistance to the user 80a and the vehicle 20c based on the history of the position information transmitted from the in-vehicle processing device 40c, the history of the position information transmitted from the user terminal 82a, and the recognition information and walking characteristic information associated with the identification information of the user terminal 82a.

[0085] As an example, assume that the user terminal 82a is in the vicinity of a pedestrian crossing as shown in Fig. 6. When the assistance control unit 260 determines from the position information and map information of the user terminal 82a and the in-vehicle processing device 40c that the user terminal 82a is located in the vicinity of the pedestrian crossing and that the user terminal 82a and the in-vehicle processing device 40c are approaching within a predetermined distance, and when the driving characteristic information associated with the identification information of the user terminal 82a includes information indicating that the user terminal 82a has a tendency to cross a road at a place where no pedestrian crossing is provided in the vicinity of the pedestrian crossing, the assistance control unit 260 transmits assistance information to the user terminal 82a and the in-vehicle processing device 40c instructing them to output warning information.

[0086] The support information to the user terminal 82a may be information instructing the user terminal 82a to output, by voice, warning information to the effect that "Please be careful as a vehicle is approaching." The support information to the user terminal 82a may be information instructing the user terminal 82a to output, by voice, warning information to the effect that "There is a pedestrian crossing, so please use the pedestrian crossing," based on walking characteristic information associated with the identification information of the user terminal 82a.

[0087] The support information to the on-vehicle processing device 40c may be information instructing the on-vehicle processing device 40c to output warning information to the effect that "Please be careful of pedestrians" by voice or display. The support information to the on-vehicle processing device 40c may be information instructing the on-vehicle processing device 40c to output warning information to the effect that "Please be careful of pedestrians wearing red clothes" based on recognition information associated with the identification information of the user terminal 82a. The support information to the on-vehicle processing device 40c may be information instructing the on-vehicle processing device 40c to further output warning information to the effect that "Please also be careful of pedestrians crossing diagonally" based on walking characteristic information associated with the identification information of the user terminal 82a.

[0088] As described in relation to Figures 5 and 6, for example, appropriate assistance can be provided to traffic participants based on the recognition information and driving characteristic information of the vehicle 20b associated with the user terminal 82b, and the recognition information and walking characteristics of the user 80a associated with the user terminal 82a.

[0089] FIG. 7 shows an example of an execution sequence of processes related to the assistance control method performed by the imaging device 70, the user terminal 82, the in-vehicle processing device 40a provided in the vehicle 20a, and the assistance device 60.

[0090] In S720, the user terminal 82a transmits location information indicating the current location of the user terminal 82a based on the signal received from the GNSS satellite together with identification information of the user terminal 82a to the support device 60. The location information and the identification information are transmitted from the user terminal 82a to the support device 60 periodically.

[0091] In S722, the user terminal 82b transmits location information indicating the current location of the user terminal 82b based on the signal received from the GNSS satellite together with identification information of the user terminal 82b to the support device 60. The location information and the identification information are transmitted from the user terminal 82b to the support device 60 periodically.

[0092] In S700, the in-vehicle processing device 40a transmits to the assistance device 60 position information, recognition information and walking characteristic information of the user 80a, and / or position information, recognition information and driving characteristic information of the vehicle 20b, obtained by analyzing an image captured by an imaging device provided in the in-vehicle processing device 40a.

[0093] In S730, the imaging device 70 transmits to the assistance device 60 position information, recognition information, and walking characteristic information of the user 80a, and / or position information, recognition information, and driving characteristic information of the vehicle 20b, obtained by analyzing the image captured by the imaging device 70.

[0094] In S712, the assistance control unit 260 of the assistance device 60 performs matching based on the similarity of positions and displacements based on the history of position information received from the user terminal 82, the imaging device 70, and the in-vehicle processing device 40a in S720, S722, S700, and S730. As a result, the assistance control unit 260 determines that the position information of the user terminal 82a matches with the position information of the user 80a transmitted from the imaging device 70 and / or the in-vehicle processing device 40a, and further determines that the position information of the user terminal 82b matches with the position information of the vehicle 20b transmitted from the imaging device 70 and / or the in-vehicle processing device 40a.

[0095] In S714, the storage control unit 240 stores in the storage device 280 the identification information of the user terminal 82b and the attribute information based on the recognition information and driving characteristic information of the vehicle 20b transmitted from the imaging device 70 and / or the in-vehicle processing device 40a. In S716, the storage control unit 240 stores in the storage device 280 the identification information of the user terminal 82a and the attribute information based on the recognition information and driving characteristic information of the user 80a transmitted from the imaging device 70 and / or the in-vehicle processing device 40a.

[0096] Fig. 8 shows an example of an execution sequence of processes related to the assistance control method performed in the user terminal 82b, the assistance device 60, and the user terminal 82c. Fig. 8 particularly shows an example of an execution sequence when traffic assistance is provided to the user terminal 82c.

[0097] In S822, the user terminal 82b transmits location information indicating the current location of the user terminal 82b based on the signal received from the GNSS satellite together with identification information of the user terminal 82b to the support device 60. The location information and the identification information are transmitted from the user terminal 82b to the support device 60 periodically.

[0098] In S800, the user terminal 82c transmits position information indicating the current position of the user terminal 82c based on a signal received from a GNSS satellite to the support device 60. The transmission of the position information from the user terminal 82c to the support device 60 is performed periodically.

[0099] In S814, the assistance control unit 260 of the assistance device 60 determines whether to provide assistance based on the information transmitted from the user terminal 82a and the user terminal 82c. For example, as described in relation to FIG. 5 and the like, when the assistance control unit 260 determines that the user terminal 82c is about to cross the road 90 based on the history of the position information of the user terminal 82c and the map information, and determines that the user terminals 82b and 82c are approaching within a predetermined distance based on the history of the position information of the user terminals 82b and 82c, the assistance control unit 260 determines to provide assistance to the user terminal 82c. As described in relation to FIG. 5 and the like, the assistance control unit 260 may determine whether to provide assistance based on the driving characteristic information stored in the storage device 280 in association with the identification information of the user terminal 82b.

[0100] When it is determined that assistance is to be provided, in S816, the assistance control unit 260 transmits assistance information to the user terminal 82c via the communication device 290. As described in relation to FIG. 5 and the like, the assistance information may be information instructing the user terminal 82c to output warning information. As described in relation to FIG. 5 and the like, the assistance information may be information based on recognition information and / or driving characteristic information stored in the storage device 280 in association with the identification information of the user terminal 82b.

[0101] When the user terminal 82c receives the assistance information from the assistance device 60, in S806, the user terminal 82c outputs warning information based on the assistance information by an HMI (Human Machine Interface) function of the user terminal 82c. The user terminal 82c may output the warning information by voice.

[0102] Fig. 9 shows an example of an execution sequence of processing related to the assistance control method performed in the user terminal 82a, the assistance device 60, and the in-vehicle processing device 40c of the vehicle 20c. Fig. 9 particularly shows an example of an execution sequence when traffic assistance is provided to the in-vehicle processing device 40c and the user terminal 82a.

[0103] In S922, the user terminal 82a transmits location information indicating the current location of the user terminal 82a based on the signal received from the GNSS satellite together with identification information of the user terminal 82a to the support device 60. The location information and the identification information are transmitted from the user terminal 82a to the support device 60 periodically.

[0104] In S900, the in-vehicle processing device 40c transmits position information indicating the current position of the vehicle 20c based on signals received from GNSS satellites to the assistance device 60. The transmission of the position information from the in-vehicle processing device 40c to the assistance device 60 is performed periodically.

[0105] In S914, the assistance control unit 260 of the assistance device 60 judges whether to provide assistance to the on-board processing device 40c and / or the user terminal 82a based on the information transmitted from the user terminal 82a and the on-board processing device 40c. For example, as described in relation to FIG. 6 and the like, when it is judged from the position information and map information of the user terminal 82a that the user terminal 82a is located near a pedestrian crossing, and from the history of the position information of the on-board processing device 40c and the user terminal 82a that the user terminal 82a and the on-board processing device 40c are approaching within a predetermined distance, it judges to provide assistance to the user 80a of the user terminal 82a and the vehicle 20c. As described in relation to FIG. 6 and the like, the assistance control unit 260 may judge whether to provide assistance based on the walking characteristic information stored in the storage device 280 in association with the identification information of the user terminal 82a.

[0106] When it is determined that assistance is to be performed, the assistance control unit 260 transmits assistance information to the in-vehicle processing device 40c through the communication device 290 in S916, and transmits the assistance information to the user terminal 82a through the communication device 290 in S918. As described in relation to FIG. 6 and the like, the assistance information may be information instructing the in-vehicle processing device 40c and the auxiliary user terminal 82c to output warning information. As described in relation to FIG. 6 and the like, the assistance information may be information based on the recognition information and / or the driving characteristic information stored in the storage device 280 in association with the identification information of the user terminal 82a. When the in-vehicle processing device 40c receives the assistance information from the assistance device 60, in S906, the in-vehicle processing device 40c outputs warning information based on the assistance information by the HMI function of the in-vehicle processing device 40c. When the user terminal 82a receives the assistance information from the assistance device 60, in S926, the user terminal 82a outputs warning information based on the assistance information by the HMI function of the user terminal 82a.

[0107] According to the assistance system 10 described above, the recognition information and / or walking characteristic information of a pedestrian acquired by an imaging device on a vehicle and / or a traffic infrastructure having an imaging function and a communication function can be stored in association with the identification information of a user terminal carried by the pedestrian. As a result, even in a place where a vehicle and / or an imaging device on a traffic infrastructure having an imaging function is not present around the user terminal, the recognition information and / or walking characteristic information of a user who holds the user terminal can be identified based on the identification information of the user terminal carried by the pedestrian, and appropriate and effective assistance can be provided to a traffic participant based on the identified recognition information and / or walking characteristic information. Furthermore, the recognition information and / or driving characteristic information of a vehicle acquired by an imaging device on a vehicle and / or a traffic infrastructure having an imaging function and a communication function can be stored in association with the identification information of a user terminal moving together with the vehicle. As a result, even in locations where there are no vehicles with imaging capabilities and / or imaging devices on traffic infrastructure around the vehicle, it is possible to identify the recognition information and / or driving characteristic information of the vehicle based on the identification information of the user terminal located inside the vehicle, and provide appropriate and effective traffic assistance to traffic participants based on the identified recognition information and / or driving characteristic information.

[0108] 10 shows an example of a computer 2000 in which multiple embodiments of the present invention may be embodied in whole or in part. A program installed in the computer 2000 may cause the computer 2000 to function as an apparatus such as the support device 60 according to an embodiment or each part of the apparatus, to execute operations associated with the apparatus or each part of the apparatus, and / or to execute a process or steps of the process according to an embodiment. Such a program may be executed by the CPU 2012 to cause the computer 2000 to execute specific operations associated with some or all of the blocks of the process steps and block diagrams described herein.

[0109] The computer 2000 according to this embodiment includes a CPU 2012 and a RAM 2014, which are connected to each other by a host controller 2010. The computer 2000 also includes a ROM 2026, a flash memory 2024, a communication interface 2022, and an input / output chip 2040. The ROM 2026, the flash memory 2024, the communication interface 2022, and the input / output chip 2040 are connected to the host controller 2010 via the input / output controller 2020.

[0110] The CPU 2012 operates according to the programs stored in the ROM 2026 and RAM 2014, thereby controlling each unit.

[0111] The communication interface 2022 communicates with other electronic devices via a network. The flash memory 2024 stores programs and data used by the CPU 2012 in the computer 2000. The ROM 2026 stores a boot program and the like executed by the computer 2000 upon activation, and / or a program that depends on the hardware of the computer 2000. The input / output chip 2040 may also connect various input / output units, such as a keyboard, mouse, and monitor, to the input / output controller 2020 via input / output ports, such as a serial port, a parallel port, a keyboard port, a mouse port, a monitor port, a USB port, an HDMI port, and the like.

[0112] The programs are provided via a computer-readable storage medium such as a CD-ROM, a DVD-ROM, or a memory card, or a network. The RAM 2014, the ROM 2026, or the flash memory 2024 are examples of computer-readable storage media. The programs are installed in the flash memory 2024, the RAM 2014, or the ROM 2026, and executed by the CPU 2012. Information processing described in these programs is read by the computer 2000, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing an operation or processing of information according to the use of the computer 2000.

[0113] For example, when communication is performed between the computer 2000 and an external device, the CPU 2012 may execute a communication program loaded in the RAM 2014 and instruct the communication interface 2022 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 2012, the communication interface 2022 reads transmission data stored in a transmission buffer processing area provided in a recording medium such as the RAM 2014 and the flash memory 2024, transmits the read transmission data to a network, and writes reception data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0114] Furthermore, the CPU 2012 may cause all or a necessary portion of a file or database stored on a recording medium such as the flash memory 2024 to be read into the RAM 2014, and perform various types of processing on the data on the RAM 2014. The CPU 2012 then writes back the processed data to the recording medium.

[0115] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and subjected to information processing. The CPU 2012 may execute various types of processing, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., described herein and specified by the instruction sequence of the program, on the data read from the RAM 2014, and write back the results to the RAM 2014. The CPU 2012 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 2012 may search for an entry that matches a condition in which an attribute value of the first attribute is specified from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0116] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 2000 or in the vicinity of the computer 2000. A recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as the computer-readable storage medium. The programs stored in the computer-readable storage medium may be provided to the computer 2000 via a network.

[0117] A program installed in the computer 2000 and causing the computer 2000 to function as the support device 60 may act on the CPU 2012, etc., to cause the computer 2000 to function as each part of the support device 60 (e.g., the support control device 200, etc.). When the information processing described in these programs is read into the computer 2000, it functions as each part of the support device 60, which is a specific means in which the software and the various hardware resources described above work together. These specific means then realize calculation or processing of information according to the intended use of the computer 2000 in this embodiment, thereby constructing a specific support device 60 according to the intended use.

[0118] Various embodiments have been described with reference to block diagrams or the like. In the block diagrams, each block may represent (1) a stage of a process where an operation is performed or (2) a portion of an apparatus responsible for performing an operation. Particular stages and portions may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuitry and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuitry including logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, memory elements such as flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

[0119] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon constitutes at least a portion of a product that includes instructions that can be executed to provide a means for performing the operations specified in a process or block diagram. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray (RTM) disks, memory sticks, integrated circuit cards, etc.

[0120] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0121] The computer readable instructions may be provided to a processor or programmable circuitry of a general purpose computer, special purpose computer, or other programmable data processing apparatus, either locally or over a wide area network (WAN) such as a local area network (LAN), the Internet, etc., to cause the computer readable instructions to execute means for performing operations specified in the process steps or block diagrams described. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0122] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0123] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically indicated as "before," "prior to," etc., and may be realized in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, it does not mean that it is essential to perform the process in this order. [Explanation of symbols]

[0124] 10 Support System 20 Vehicles 40 On-board processing device 50 traffic lights 60 Support equipment 70 Imaging Device 80 users 82 User terminal 90 road 200 Assistance control device 202 Acquisition Department 210 First acquisition part 220 Second Acquisition Department 240 Memory control unit 260 Support control section 280 Storage device 290 Communication Equipment 2000 Computer 2010 Host Controller 2012 CPU 2014 RAM 2020 Input / Output Controller 2022 Communication Interface 2024 Flash Memory 2026 ROM 2040 Input / Output Chip

Claims

1. A first acquisition unit that acquires identification information and location information of a user terminal associated with a user; a second acquisition unit that acquires recognition information and position information of a moving object by analyzing an image of the moving object acquired by an imaging device; a storage control unit that performs control to store the recognition information of the moving object acquired by the second acquisition unit in association with the identification information of the user terminal acquired by the first acquisition unit when the position information of the user terminal acquired by the first acquisition unit and the position information of the moving object acquired by the second acquisition unit match; an assistance control unit that controls assistance for traffic participants based on the identification information of the user terminal and the recognition information of the moving object that are stored in association with each other under the control of the storage control unit; An assistance control device comprising:

2. Even if the second acquisition unit cannot acquire the recognition information of the moving object, when the first acquisition unit acquires the identification information of the user terminal, the assistance control unit provides assistance to the traffic participant based on the identification information of the user terminal and the recognition information of the moving object that are associated and stored under the control of the storage control unit. The assistance control device according to claim 1 .

3. The assistance control unit performs control for notifying traffic participants other than the user of the recognition information. The assistance control device according to claim 1 or 2.

4. The recognition information includes information indicating a color of the moving object. The assistance control device according to claim 1 or 2.

5. The recognition information includes information indicating a type of the moving object. The assistance control device according to claim 1 or 2.

6. The recognition information includes information indicating a size of the moving object. The assistance control device according to claim 1 or 2.

7. The storage control unit performs control to delete the identification information of the user terminal and the recognition information of the moving object, which are stored in association with each other, every time a predetermined time elapses. The assistance control device according to claim 1 or 2.

8. The storage control unit performs control to update the identification information of the user terminal and the recognition information of the moving object, which are stored in association with each other, when the position information of the user terminal newly acquired by the first acquisition unit and the position information of the moving object newly acquired by the second acquisition unit match. The assistance control device according to claim 1 or 2.

9. The moving object includes at least one of a pedestrian and a vehicle. The assistance control device according to claim 1 or 2.

10. The second acquisition unit further acquires movement characteristic information and position information of the moving object acquired by analyzing an image of the moving object acquired by an imaging device, the storage control unit, when the position information of the user terminal acquired by the first acquisition unit and the position information of the moving object acquired by the second acquisition unit are consistent, performs control to store the recognition information and the movement characteristic information of the moving object acquired by the second acquisition unit in association with the identification information of the user terminal acquired by the first acquisition unit; The assistance control unit performs control regarding assistance for traffic participants based on the identification information of the user terminal and the recognition information and movement characteristic information of the moving object that are associated and stored under the control of the storage control unit. The assistance control device according to claim 1 or 2.

11. the moving object includes a pedestrian; the recognition information includes information indicating a color of the pedestrian, The movement characteristic information includes information indicating a behavioral tendency of the pedestrian regarding passage along a road. The assistance control device according to claim 10.

12. the moving object includes a vehicle; the recognition information includes information indicating a color of the vehicle, The movement characteristic information includes information regarding dangerous driving of the vehicle. The assistance control device according to claim 10.

13. obtaining identification and location information of a user terminal associated with the user; acquiring recognition information and position information of a moving object by analyzing an image of the moving object acquired by an imaging device; performing control for storing recognition information of the moving object in association with identification information of the user terminal when the position information of the user terminal and the position information of the moving object match; a step of controlling assistance for a traffic participant based on the identification information of the user terminal and the recognition information of the moving object that are stored in association with each other in the step of controlling storage; The assistance control method includes:

14. A program for causing a computer to function as the assistance control device according to claim 1 or 2.

Images