Auxiliary control device, auxiliary control method and computer readable storage medium

Abstract

The invention provides an assist control apparatus, an assist control method, and a computer readable storage medium. The assist control device is provided with: a first acquisition unit that acquires identification information and location information of a user terminal associated with a user; a second acquisition unit that acquires movement characteristic information and position information of the moving object acquired by analyzing an image of the moving object acquired by the imaging device; a storage control unit that, if the position information acquired by the first acquisition unit matches the position information acquired by the second acquisition unit, performs control for storing the movement characteristic information of the moving object acquired by the second acquisition unit in association with the identification information of the user terminal acquired by the first acquisition unit; and an assistance control unit that performs control relating to assistance with respect to the traffic participant on the basis of the identification information of the user terminal and the movement characteristic information of the moving object, which are stored in association with each other by the control by the storage control unit.

Description

Technical Field

[0001] The invention relates to an auxiliary control device, an auxiliary control method and a computer-readable storage medium. Background Art

[0002] In recent years, efforts to provide access to sustainable transportation systems for disadvantaged people among traffic participants have become increasingly active. To achieve this goal, research and development are being devoted to further improving the safety and convenience of traffic through research and development related to preventive safety technologies. Patent documents 1 to 3 describe technologies related to attribute information for pedestrians.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2019-28916

[0004] Patent Document 2: International Publication No. 2019 / 188429

[0005] Patent Document 3: Japanese Patent Application Publication No. 2019-32174 Summary of the invention

[0006] However, in preventive safety technology, comprehensive assurance has become a problem. This application aims to provide an alarm in order to solve the above-mentioned problem. Moreover, it also contributes to the development of a sustainable transportation system.

[0007] In a first aspect of the present invention, an auxiliary control device is provided. The auxiliary control device includes a first acquisition unit that acquires identification information of a user terminal associated with a user and location information of the user terminal. The auxiliary control device includes a second acquisition unit that acquires movement characteristic information of a mobile object acquired by analyzing an image of the mobile object acquired by a photographing device and location information of the mobile object. The auxiliary control device includes a storage control unit that performs control for storing the movement characteristic information of the mobile object acquired by the second acquisition unit in correspondence with the identification information of the user terminal acquired by the first acquisition unit when the location information of the user terminal acquired by the first acquisition unit matches the location information of the mobile object acquired by the second acquisition unit. The auxiliary control device includes an auxiliary control unit that performs control related to assistance for traffic participants based on the identification information of the user terminal and the movement characteristic information of the mobile object stored in correspondence through control of the storage control unit.

[0008] In the above-mentioned auxiliary control device, even when the second acquisition unit is unable to acquire the movement characteristic information of the mobile object, when the identification information of the user terminal is acquired by the first acquisition unit, the auxiliary control unit can also provide assistance to traffic participants based on the identification information of the user terminal and the movement characteristic information of the mobile object stored correspondingly under the control of the storage control unit.

[0009] In any of the above-mentioned assistance control devices, the assistance control unit may perform control for notifying traffic participants other than the user of the movement characteristic information of the moving object.

[0010] In any of the above auxiliary control devices, the moving object may include a pedestrian.

[0011] In any of the above-mentioned auxiliary control devices, the movement characteristic information may include information indicating a behavior tendency related to the passage of the pedestrian on the road.

[0012] In any of the above-mentioned auxiliary control devices, the movement characteristic information may include information indicating whether the pedestrian has a tendency to cross the road in a place where a crosswalk is not provided.

[0013] In any of the above-mentioned auxiliary control devices, the movement characteristic information may include information indicating whether the pedestrian has a tendency to cross the road in a place where no crosswalk is provided near a crosswalk.

[0014] In any of the above-mentioned auxiliary control devices, the movement characteristic information may include information indicating whether the pedestrian has a tendency to cross the road without following the indication state of the traffic light.

[0015] In any of the above-mentioned auxiliary control devices, the movement characteristic information may include information indicating whether the pedestrian has a tendency to start crossing the road in a time shorter than a predetermined time when the indication of the traffic light is switched from a proceeding indication to a stop indication.

[0016] In any of the above auxiliary control devices, the moving object may include a vehicle.

[0017] In any of the above auxiliary control devices, the movement characteristic information may include information related to dangerous driving of the vehicle.

[0018] In any of the above-mentioned auxiliary control devices, the storage control unit may perform control for deleting the identification information of the user terminal and the movement characteristic information of the mobile object stored in correspondence with each other every time a predetermined time has passed.

[0019] In any of the above-mentioned auxiliary control devices, when the location information of the user terminal newly acquired by the first acquisition unit matches the location information of the mobile object newly acquired by the second acquisition unit, the storage control unit can perform control for updating and establishing the correspondingly stored identification information of the user terminal and the movement characteristic information of the mobile object.

[0020] In any of the above-mentioned auxiliary control devices, the second acquisition unit may also acquire identification information of the mobile object and position information of the mobile object acquired by analyzing an image of the mobile object acquired by a photographing device. When the position information of the user terminal acquired by the first acquisition unit matches the position information of the mobile object acquired by the second acquisition unit, the storage control unit may perform control for establishing a corresponding storage of the movement characteristic information and identification information of the mobile object acquired by the second acquisition unit and the identification information of the user terminal acquired by the first acquisition unit. The auxiliary control unit may perform control related to assistance for traffic participants based on the identification information of the user terminal, the movement characteristic information and identification information of the mobile object established and stored in correspondence through the control of the storage control unit.

[0021] In any of the above auxiliary control devices, the moving object may include a pedestrian. The identification information may include information indicating the color of the pedestrian. The movement characteristic information may include information indicating a behavior tendency of the pedestrian in traveling on a road.

[0022] In any of the above auxiliary control devices, the moving object may include a vehicle. The identification information may include information indicating the color of the vehicle. The movement characteristic information may include information related to dangerous driving of the vehicle.

[0023] In a second aspect of the present invention, an auxiliary control method is provided. The auxiliary control method includes the step of acquiring identification information of a user terminal associated with a user and location information of the user terminal. The auxiliary control method includes the step of acquiring movement characteristic information of a mobile object acquired by analyzing an image of the mobile object acquired by a photographing device and location information of the mobile object. The auxiliary control method includes the step of performing control for establishing a corresponding storage of the movement characteristic information of the mobile object and the identification information of the user terminal when the location information of the user terminal matches the location information of the mobile object. The auxiliary control method includes the step of performing control related to assistance for traffic participants based on the identification information of the user terminal and the movement characteristic information of the mobile object established and stored in correspondence by the step of performing control for storage.

[0024] In a third aspect of the present invention, there is provided a program that causes a computer to function as any one of the above-mentioned auxiliary control devices.

[0025] In addition, the above summary of the invention does not list all the features of the present invention. In addition, a sub-combination of these feature groups may also be an invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A usage scenario of the assistance system 10 is schematically shown.

[0027] Figure 2 The functional structure of the auxiliary device 60 is shown.

[0028] Figure 3 An example of the data structure of the attribute information of the vehicle including the identification information and the movement characteristic information stored under the control of the storage control unit 240 is shown.

[0029] Figure 4 An example of the data structure of the user's attribute information including the identification information and the movement characteristic information stored under the control of the storage control unit 240 is shown.

[0030] Figure 5 The following schematically shows a situation in which the assistance control unit 260 assists a road user.

[0031] Figure 6 Other situations in which the assistance control unit 260 assists road users are schematically shown.

[0032] Figure 7 An example of the execution flow of the process related to the support control method performed by the imaging device 70 , the user terminal 82 , the vehicle-mounted processing device 40 a included in the vehicle 20 a , and the support device 60 is shown.

[0033] Figure 8 An example of the execution flow of the processing related to the assistance control method performed in the user terminal 82b, the assistance device 60, and the user terminal 82c is shown.

[0034] Fig. 9 An example of the execution flow of the process related to the support control method performed in the user terminal 82a, the support device 60, and the vehicle-mounted processing device 40c included in the vehicle 20c is shown.

[0035] Fig.10 An example of a computer 2000 is shown. DETAILED DESCRIPTION

[0036] Hereinafter, the present invention will be described by way of the embodiments of the invention, but the following embodiments do not limit the claimed invention. In addition, not all combinations of features described in the embodiments are essential to the solution means of the invention.

[0037] Figure 1 The following schematically shows a usage scenario of the support system 10. The support system 10 includes a support device 60, a camera 70, and a traffic light 50.

[0038] exist Figure 1 In the embodiment, user 80a, vehicle 20a and vehicle 20b are traffic participants. Vehicle 20a is provided with an onboard processing device 40a. User terminal 82b is a user terminal held by a passenger of vehicle 20b. User terminal 82a is a terminal held by user 80a. In the present embodiment, user 80a is a pedestrian. User 80a is an example of a moving object.

[0039] In the present embodiment, the user terminal 82a and the user terminal 82b may be collectively referred to as “user terminal 82”. The user 80a and the user 80b may be collectively referred to as “user 80”.

[0040] The vehicle 20a is a vehicle traveling on the road 90. The vehicle 20a is an example of a mobile object. The vehicle-mounted processing device 40a is composed of various sensors including a position sensor including a global navigation satellite system (GNSS) receiver, a speed sensor such as a vehicle speed sensor, and a camera that captures the surroundings of the vehicle 20a. The vehicle-mounted processing device 40a has a function of processing information acquired by various sensors of the vehicle-mounted processing device 40a and a function of communicating with the auxiliary device 60. The vehicle-mounted processing device 40a provides an advanced driving assistance system (ADAS) function of the vehicle 20a.

[0041] User terminals 82a and 82b are portable terminals such as smartphones, etc. User terminals 82a and 82b periodically transmit current location information of each user terminal detected by a location sensor including a GNSS receiver together with identification information of user terminals 82a and 82b to the assistance device 60.

[0042] The camera 70 is a camera fixedly installed on a traffic infrastructure. The camera 70 obtains the positions of the vehicle 20a, the vehicle 20b and the user 80a within the shooting range of the camera 70 by analyzing the captured images, and transmits the acquired positions of the vehicle 20a, the vehicle 20b and the user 80a to the auxiliary device 60.

[0043] The support device 60 receives information sent from the vehicle-mounted processing device 40a, the user terminal 82a, and the camera 70 via mobile communication. The support device 60 can receive information sent from the vehicle-mounted processing device 40a, the user terminal 82a, and the camera 70 via a communication line such as mobile communication, the Internet, or a dedicated line.

[0044] The camera 70 analyzes the image captured by the camera 70 and determines the positions of the vehicle 20b and the user 80a. The camera 70 also analyzes the image captured by the camera 70 to obtain the color, size, and category of the vehicle 20b. The color, size, and category are examples of identification information of the vehicle 20b. The identification information is, for example, information that can be visually identified by the vehicle 20b. The camera 70 also analyzes the image captured by the camera 70 to obtain the color, age, and height of the clothes of the user 80a. The color, age, and height of the clothes are examples of identification information of the user 80a. The identification information is, for example, information that can be visually identified by the user 80a.

[0045] The camera 70 also analyzes the image captured by the camera 70 to obtain the driving characteristic information of the vehicle 20b. The driving characteristic information includes information indicating whether the vehicle 20b has a tendency to drive dangerously. For example, the camera 70 determines the indication state of the traffic signal of the traffic light 50, and when it is detected from the image that the vehicle 20b is driving in disregard of the traffic signal, it is determined that the vehicle 20b has a tendency to drive dangerously. When the camera 70 turns left or right without operating the direction indicator of the vehicle 20b, it is determined that the vehicle 20b has a tendency to drive dangerously. In this way, the camera 70 determines that the vehicle 20b has a tendency to drive dangerously when it drives without following traffic regulations. In addition, when the vehicle 20b turns left or right without paying attention to the oncoming vehicle, the rear vehicle and / or nearby pedestrians, it is determined that the vehicle 20b has a tendency to drive dangerously. The driving characteristic information is an example of the movement characteristic information of the vehicle 20b.

[0046] The camera 70 also analyzes the image captured by the camera 70 to obtain walking characteristic information of the user 80a. The walking characteristic information is information indicating the action tendency of the user 80a when traveling on the road. For example, the camera 70 specifies the indication state of the traffic signal of the traffic light 50, and when it is detected from the image based on the history of the position information of the user 80a that the user 80a crosses the road 90 without following the indication state of the traffic light 50, it is determined that the user 80a has a tendency to cross the road 90 without following the indication state of the traffic light 50. The camera 70 recognizes the road markings and / or traffic signs on the road 90, and when it is detected from the image based on the history of the position information of the user 80a that the user 80a passes without following the road markings and / or traffic signs, it is determined that the user 80a has a tendency to pass without following the road markings and / or traffic signs. For example, the camera 70 determines whether pedestrians have a tendency to cross the road in a place where a crosswalk is not provided. In addition, the camera 70 determines whether the user 80a tends to cross the road 90 in a place where no crosswalk is provided near the crosswalk. For example, the camera 70 determines whether the user 80a tends to cross the road 90 diagonally near the crosswalk. In addition, the camera 70 determines whether the user 80a tends to start crossing the road in a time shorter than a predetermined time when the indication state of the traffic light 50 is switched from a forward indication to a stop indication. In this way, the camera 70 determines that the user 80a tends to take dangerous actions when walking without following traffic regulations. The walking characteristic information is an example of the movement characteristic information of the user 80a.

[0047] The position information of the vehicle 20b and the user 80a, the identification information of the vehicle 20b and the user 80a, the driving characteristics information of the vehicle 20b, and the walking characteristics information of the user 80a acquired by the camera 70 are transmitted from the camera 70 to the support device 60. The vehicle-mounted processing device 40a can acquire the position information of the vehicle 20b and the user 80a, the identification information of the vehicle 20b and the user 80a, the driving characteristics information of the vehicle 20b, and the walking characteristics information of the user 80a from the image captured by the camera included in the vehicle-mounted processing device 40a, and transmit the acquired information to the support device 60.

[0048] The auxiliary device 60 compares the history of the location information obtained from the camera 70 and / or the vehicle-mounted processing device 40a with the history of the location information obtained from the user terminal 82a, performs matching based on the similarity of the position and displacement, and extracts the identification information and walking characteristic information of the user terminal 82a from the identification information and walking characteristic information obtained from the camera 70 and / or the vehicle-mounted processing device 40a. The auxiliary device 60 stores the extracted identification information and walking characteristic information of the user terminal 82a and the identification information received from the user terminal 82a in a corresponding manner.

[0049] The auxiliary device 60 compares the history of the location information obtained from the camera 70 and / or the vehicle-mounted processing device 40a with the history of the location information obtained from the user terminal 82b, performs matching based on the similarity of the position and displacement, and extracts the identification information and driving characteristic information of the vehicle 20b from the identification information and walking characteristic information obtained from the camera 70 and / or the vehicle-mounted processing device 40a. The auxiliary device 60 stores the extracted identification information and driving characteristic information of the vehicle 20b and the identification information received from the user terminal 82b in a corresponding manner.

[0050] In this way, the auxiliary device 60 can establish a correspondence between the identification information of the user terminal 82b and the identification information and driving characteristic information of the vehicle 20b. Therefore, when the vehicle 20b is traveling in a place where the camera 70 is not installed and there are no other vehicles with ADAS functions around, the auxiliary device 60 can assist the traffic participants based on the identification information and driving characteristic information after receiving the identification information of the user terminal 82b. For example, when the vehicle 20b is a red vehicle and tends to drive dangerously, the auxiliary device 60 outputs an alarm message to the user terminals around the vehicle 20b through sound, etc., which means "a red vehicle is approaching, please pay attention", thereby reminding the users of the user terminals around the vehicle 20b.

[0051] Similarly, the auxiliary device 60 can establish a correspondence between the identification information of the user terminal 82a and the identification information and walking characteristic information of the user 80a. Thus, when the user 80a walks in a place where the camera 70 is not installed or there is no other vehicle with ADAS function, the auxiliary device 60 can assist the traffic participant based on the identification information and the walking characteristic information after receiving the identification information of the user terminal 82b. For example, when the user 80a wears red clothes and tends to cross the road without paying attention to road markings and traffic signs, the auxiliary device 60 outputs an alarm message to the vehicles around the user 80a, which means "There is a pedestrian wearing red clothes nearby, please pay attention", thereby reminding the passengers of the vehicles around the user 80a.

[0052] Figure 2 The following is a functional configuration of the support device 60. The support device 60 includes a support control device 200, a communication device 290, and a storage device 280.

[0053] The communication device 290 is responsible for the communication between the vehicle-mounted processing device 40a, the user terminals such as the user terminal 82a and the user terminal 82b, and the camera 70 and the auxiliary device 60, respectively, based on the control of the auxiliary control device 200. The auxiliary control device 200 is implemented by a circuit including, for example, a calculation processing device including a processor. The auxiliary control device 200 can be implemented by a microcomputer including a CPU, ROM, RAM, I / O, bus, etc. The storage device 280 is implemented by having a non-volatile storage medium. The auxiliary control device 200 performs processing using information stored in the storage device 280. The storage device 280 can store traffic-related information such as map information used in the control of the auxiliary control device 200.

[0054] The auxiliary control device 200 includes an acquisition unit 202, a storage control unit 240, and an auxiliary control unit 260. The acquisition unit 202 includes a first acquisition unit 210 and a second acquisition unit 220. Figure 2 The functional form of a part of the functional structure shown.

[0055] The first acquisition unit 210 acquires the identification information and location information of the user terminal 82 associated with the user 80. The first acquisition unit 210 acquires the identification information and location information of the user terminal 82 transmitted from the user terminal 82 and received by the communication device 290.

[0056] The second acquisition unit 220 acquires the identification information and position information of the moving object acquired by analyzing the image of the moving object acquired by the camera. Here, the camera is the camera 70 or the vehicle-mounted processing device 40a. The second acquisition unit 220 acquires the identification information and position information of the moving object acquired by the camera 70 or the vehicle-mounted processing device 40a.

[0057] When the position information of the user terminal 82 acquired by the first acquisition unit 210 matches the position information of the mobile object acquired by the second acquisition unit 220, the storage control unit 240 performs control for storing the identification information of the mobile object acquired by the second acquisition unit 220 in correspondence with the identification information of the user terminal 82 acquired by the first acquisition unit 210. For example, when the degree of coincidence between the position information of the user terminal 82 and the position information of the mobile object is higher than a predetermined value, the storage control unit 240 performs control for storing the identification information of the mobile object in association with the identification information of the user terminal 82. The storage control unit 240 stores the identification information of the mobile object in the storage device 280 in association with the identification information of the user terminal 82, for example.

[0058] The assistance control unit 260 performs control related to assistance for traffic participants based on the identification information of the user terminal 82 and the identification information of the mobile object that are correspondingly stored under the control of the storage control unit 240. For example, even if the second acquisition unit 220 cannot acquire the identification information of the mobile object, when the identification information of the user terminal 82 is acquired by the first acquisition unit 210, the assistance control unit 260 performs assistance for traffic participants based on the identification information of the user terminal 82 and the identification information of the mobile object that are correspondingly stored under the control of the storage control unit 240. As an example, the assistance control unit 260 performs control for notifying traffic participants other than the user 80 of the identification information.

[0059] The identification information may include information indicating the color of the moving object. The identification information may include information indicating the type of the moving object. The identification information may include information indicating the size of the moving object.

[0060] The storage control unit 240 may perform control for deleting the identification information of the user terminal 82 and the identification information of the mobile object stored in association with each other every time a predetermined time has passed.

[0061] When the location information of the user terminal 82 newly acquired by the first acquisition unit 210 matches the location information of the mobile object newly acquired by the second acquisition unit 220 , the storage control unit 240 may perform control for updating the identification information of the user terminal 82 and the identification information of the mobile object stored correspondingly.

[0062] The second acquisition unit 220 acquires the movement characteristic information of the moving object and the position information of the moving object acquired by analyzing the image of the moving object acquired by the camera. Here, the camera is the camera 70 or the vehicle-mounted processing device 40a. The second acquisition unit 220 acquires the movement characteristic information and the position information of the moving object acquired by the camera 70 or the vehicle-mounted processing device 40a.

[0063] When the position information of the user terminal 82 acquired by the first acquisition unit 210 matches the position information of the mobile object acquired by the second acquisition unit 220, the storage control unit 240 performs control for storing the movement characteristic information of the mobile object acquired by the second acquisition unit 220 in correspondence with the identification information of the user terminal 82 acquired by the first acquisition unit 210. For example, when the degree of coincidence between the position information of the user terminal 82 and the position information of the mobile object is higher than a predetermined value, the storage control unit 240 performs control for storing the movement characteristic information of the mobile object in association with the identification information of the user terminal 82. The storage control unit 240 stores the movement characteristic information of the mobile object in association with the identification information of the user terminal 82 in the storage device 280, for example.

[0064] The assistance control unit 260 performs control related to assistance for traffic participants based on the identification information of the user terminal 82 and the mobile characteristic information of the mobile object correspondingly stored under the control of the storage control unit 240. For example, even in the case where the second acquisition unit 220 cannot acquire the mobile characteristic information of the mobile object, when the identification information of the user terminal 82 is acquired by the first acquisition unit 210, the assistance control unit 260 performs assistance for traffic participants based on the identification information of the user terminal 82 and the mobile characteristic information of the mobile object correspondingly stored under the control of the storage control unit 240. As an example, the assistance control unit 260 performs control for notifying traffic participants other than the user 80 of the mobile characteristic information of the mobile object.

[0065] When the position information of the user terminal 82 newly acquired by the first acquisition unit 210 matches the position information of the mobile object newly acquired by the second acquisition unit 220, the storage control unit 240 may perform control for updating the identification information of the user terminal 82 and the mobile characteristic information of the mobile object stored in correspondence. Whenever a predetermined time has passed, the storage control unit 240 may perform control for deleting the identification information of the user terminal 82 and the mobile characteristic information of the mobile object stored in correspondence with each other.

[0066] The moving object includes at least one of a pedestrian and a vehicle. In this embodiment, the moving object is a user 80a and / or a vehicle 20b.

[0067] In the case where the moving object is a pedestrian, the movement characteristic information may include information indicating the behavior tendency related to the passage of the pedestrian on the road. The movement characteristic information may include information indicating whether the pedestrian has a tendency to cross the road in a place where a crosswalk is not provided. The movement characteristic information may include information indicating whether the pedestrian has a tendency to cross the road in a place where a crosswalk is not provided near a crosswalk. The movement characteristic information may include information indicating whether the pedestrian has a tendency to cross the road without following the indication state of the traffic light. The movement characteristic information may include information indicating whether the pedestrian has a tendency to start crossing the road in a time shorter than a predetermined time when the indication of the traffic light is switched from a go indication to a stop indication.

[0068] In the case where the moving object is a vehicle, the movement characteristic information may include information related to dangerous driving of the vehicle.

[0069] The second acquisition unit 220 may acquire the movement characteristic information of the mobile object and the position information of the mobile object acquired by analyzing the image of the mobile object acquired by the photographing device. When the position information of the user terminal 82 acquired by the first acquisition unit 210 matches the position information of the mobile object acquired by the second acquisition unit 220, the storage control unit 240 may control the identification information and the movement characteristic information of the mobile object acquired by the second acquisition unit 220 to be stored in correspondence with the identification information of the user terminal 82 acquired by the first acquisition unit 210. The assistance control unit 260 may control the assistance for the traffic participant based on the identification information of the user terminal 82, the identification information and the movement characteristic information of the mobile object stored in correspondence under the control of the storage control unit 240.

[0070] When the moving object is a pedestrian, the identification information may include information indicating the color of the pedestrian, and the movement characteristic information may include information indicating the behavior tendency of the pedestrian in relation to the passage of the road.

[0071] In the case where the moving object is a vehicle, the identification information may include information indicating the color of the vehicle, and the movement characteristic information may include information related to dangerous driving of the vehicle.

[0072] Figure 3 The following is an example of a data structure of vehicle attribute information including identification information and movement characteristic information stored under the control of the storage control unit 240. The vehicle attribute information includes terminal ID, time, color, type, size, and travel characteristics.

[0073] The terminal ID is identification information of the user terminal 82 stored in association with the vehicle. The time, color, type, and size are examples of identification information of the vehicle. The driving characteristics are examples of movement characteristic information of the vehicle.

[0074] The time may be, for example, the time when information including the terminal ID, time, color, type, size, and driving characteristics is stored in the storage device 280 .

[0075] The color may be information indicating a representative color for vehicle identification. The color may be information indicating the color of the vehicle body. The category may be information indicating the vehicle type, information indicating the difference between a truck, a bus, and a passenger car, or information indicating the difference between an ordinary car and a large car. The size may be information indicating the length of the vehicle. The size may be information indicating the height of the vehicle, the lateral width of the vehicle, and the like.

[0076] In this embodiment, the storage control unit 240 establishes a correspondence between the user terminal 82b and the vehicle 20b by matching the location information of the user terminal 82b with the location information of the vehicle 20b obtained based on image analysis, and stores the identification information of the user terminal 82b, the color, type, size, and driving characteristics of the vehicle 20b in correspondence as attribute information in the storage device 280.

[0077] The driving characteristics include information indicating whether there is a tendency to perform dangerous driving in the vehicle 20b. The driving characteristics may include a plurality of data items as data items related to dangerous driving. For example, the driving characteristics may include a plurality of data items related to rule compliance and safety, such as not following traffic lights, ignoring pedestrians crossing the road, not operating the direction indicator when turning left or right, ignoring oncoming vehicles when turning left or right, ignoring pedestrians when turning left or right, ignoring the following vehicle when turning left or right, and applying sudden brakes when turning left or right.

[0078] The identification information of the vehicle 20b can be sent from the camera 70 and the vehicle-mounted processing device 40a, respectively. In addition, the identification information of the vehicle 20b can be sent from other camera devices installed on traffic infrastructure at other locations. In addition, the identification information of the vehicle 20b can also be sent from camera devices of other vehicles at other locations. Therefore, there may be a case where the recognition results based on the color, type, and size of each camera device are different.

[0079] Therefore, whenever new identification information of a vehicle corresponding to a specific terminal ID is obtained, the storage control unit 240 can update the identification information stored corresponding to the specific terminal ID. The storage control unit 240 can update the identification information corresponding to the specific terminal ID to the latest identification information. When there are differences in the colors recognized by multiple cameras, the storage control unit 240 can update the color corresponding to the specific terminal ID to the color recognized by the most cameras. Similarly, when there are differences in the categories recognized by multiple cameras, the storage control unit 240 can update the category corresponding to the specific terminal ID to the category recognized by the most cameras.

[0080] When there are differences in the sizes recognized by multiple cameras, the storage control unit 240 may update the size corresponding to the specific terminal ID to the average value of the sizes recognized by the multiple cameras. When there are differences in the sizes recognized by multiple cameras, the storage control unit 240 may update the size corresponding to the specific terminal ID to the maximum value of the sizes recognized by the multiple cameras.

[0081] In the case where the user terminal 82b is a terminal that the user can carry, the user terminal 82b is not always mounted on the vehicle 20b. Therefore, the storage control unit 240 can delete the attribute information according to the time elapsed from the time included in the attribute information. For example, when a predetermined time has elapsed from the time included in the attribute information, the storage control unit 240 deletes the attribute information. As an example, when more than 24 hours have elapsed from the time included in the attribute information, the storage control unit 240 deletes the attribute information.

[0082] Figure 4 The following is an example of the data structure of the user's attribute information including the identification information and the movement characteristic information stored under the control of the storage control unit 240. The user's attribute information includes the terminal ID, time, color, age, height, and walking characteristic.

[0083] The terminal ID is identification information of the user terminal 82 stored in association with the user. The time, color, age, and height are examples of identification information of the vehicle. The walking characteristics are examples of movement characteristic information of the user.

[0084] The time may be the time when the attribute information is stored in the storage device 280 .

[0085] Color may be information indicating a representative color identified by the user. Color may be information indicating the color of the user's clothing, such as the clothes the user is wearing. Age may be information indicating the age group of the user. Age may be information indicating the estimated age group of the user identified by the image. Age may be information indicating the age group of the user pre-registered in the user terminal identified by the terminal ID. Height may be information indicating the height of the user.

[0086] In this embodiment, the storage control unit 240 establishes a correspondence between the user terminal 82a and the user 80a by matching the location information of the user terminal 82a with the location information of the user 80a obtained based on image analysis, and stores the identification information of the user terminal 82a in correspondence with the color, age, height and walking characteristics as attribute information in the storage device 280.

[0087] The walking characteristics are information indicating the action tendency of the user 80a when traveling on the road. The walking characteristics may include a plurality of data items as data items related to the action tendency. For example, the walking characteristics may include: a plurality of data items related to rule compliance and safety, such as whether the user 80a has a tendency to cross the road without following the indication state of the traffic light, whether the user 80a has a tendency to pass without following the road markings and / or traffic signs, whether pedestrians have a tendency to cross the road in places where no crosswalks are set, whether the user 80a has a tendency to cross the road in places where no crosswalks are set near the crosswalks, whether the user 80a has a tendency to cross the road diagonally near the crosswalks, and whether the user 80a has a tendency to suddenly start crossing the road when the indication state of the traffic light is switched from the going indication to the stop indication, etc.

[0088] The identification information of the user 80a can be sent from the camera 70 and the vehicle-mounted processing device 40a respectively. In addition, the identification information of the user 80a can be sent from other cameras installed on traffic infrastructure in other places. In addition, the identification information of the user 80a can also be sent from cameras of other vehicles in other places. Therefore, there may be a case where the recognition results based on the color, age and height of each camera are different.

[0089] Therefore, whenever new identification information of a user corresponding to a specific terminal ID is obtained, the storage control unit 240 may update the identification information stored corresponding to the specific terminal ID. The storage control unit 240 may update the identification information corresponding to the specific terminal ID to the latest identification information. When there are differences in the colors recognized by a plurality of camera devices, the storage control unit 240 may update the color corresponding to the specific terminal ID to the color recognized by the most camera devices.

[0090] When there are differences in the ages recognized by multiple cameras, the storage control unit 240 may update the age corresponding to a specific terminal ID to the average of the ages recognized by the multiple cameras. When there are differences in the ages recognized by multiple cameras, the storage control unit 240 may update the age corresponding to a specific terminal ID to the maximum of the ages recognized by the multiple cameras. Similarly, when there are differences in the heights recognized by multiple cameras, the storage control unit 240 may update the height corresponding to a specific terminal ID to the average of the heights recognized by the multiple cameras. When there are differences in the heights recognized by multiple cameras, the storage control unit 240 may update the height corresponding to a specific terminal ID to the maximum of the heights recognized by the multiple cameras.

[0091] In the case where the user terminal 82a is a terminal that can be carried by the user, the user terminal 82a is not always carried by the specific user 80a. Therefore, the storage control unit 240 can delete the attribute information according to the elapsed time from the time included in the attribute information. For example, when a predetermined time has passed from the time included in the attribute information, the storage control unit 240 deletes the attribute information. As an example, when more than 24 hours have passed from the time included in the attribute information, the storage control unit 240 deletes the attribute information.

[0092] Figure 5 The schematic diagram shows the situation in which the assistance control unit 260 assists the traffic participant. Here, it is assumed that the attribute information of the vehicle 20b has been stored in the storage device 280. That is, it is assumed that the identification information and driving characteristic information of the vehicle 20b acquired by the camera 70 and / or the vehicle-mounted processing device 40a are stored in the storage device 280 in correspondence with the identification information of the user terminal 82b.

[0093] The user terminal 82b transmits the position information and the identification information to the support device 60. Based on the identification information transmitted from the user terminal 82b, the support control unit 260 acquires the identification information and the driving characteristic information stored in the storage device 280 in correspondence with the identification information. Thus, the support control unit 260 determines that the vehicle having the identification information and the driving characteristic information exists at the position transmitted from the user terminal 82b.

[0094] The user terminal 82c transmits the position information and the identification information to the assistance device 60. The assistance control unit 260 performs traffic assistance for the user terminal 82c based on the history of the position information transmitted from the user terminal 82b, the history of the position information transmitted from the user terminal 82c, the identification information corresponding to the identification information of the user terminal 82b, and the driving characteristic information.

[0095] As an example, Figure 5As shown, it is assumed that the user terminal 82c is about to cross the road 90. When the auxiliary control unit 260 determines that the user 80c is about to cross the road 90 based on the history of the position information of the user terminal 82b and the user terminal 82c and the map information, and determines that the user terminal 82b and the user terminal 82c are close to each other within a predetermined distance, when the driving characteristic information corresponding to the identification information of the user terminal 82b contains information indicating a tendency to ignore pedestrians crossing the road, the auxiliary control unit 260 sends auxiliary information to the user terminal 82c to instruct the user terminal 82c to output warning information. The auxiliary information provided to the user terminal 82c may be information instructing to output warning information to the meaning of "please pay attention to the vehicle" by sound. The auxiliary information provided to the user terminal 82c may be information instructing to output warning information to the meaning of "please pay attention to the red vehicle" by sound based on the identification information. The auxiliary information provided to the user terminal 82c may be information instructing to further output warning information to the meaning of "please be careful when crossing the road" based on the driving characteristic information corresponding to the identification information of the user terminal 82b.

[0096] In addition, the auxiliary control unit 260 may estimate the respective positions of the user terminal 82b and the user terminal 82c when the predetermined time has passed, so that the auxiliary control unit 260 may determine whether the user terminal 82b and the user terminal 82c are close to each other within the predetermined time. When the auxiliary control unit 260 determines that the user terminal 82b and the user terminal 82c are close to each other within the predetermined time, the auxiliary control unit 260 may send auxiliary information to the user terminal 82c, and the auxiliary information instructs the user terminal 82c to output the alarm information meaning "the vehicle is approaching, please pay attention" by sound. The auxiliary information may be information that instructs the alarm information meaning "the red vehicle is approaching, please pay attention" to be output by sound based on the identification information. In the case of determining whether the user terminal 82b and the user terminal 82c are close to each other within the predetermined time, the auxiliary control unit 260 may set the "predetermined time" based on the driving characteristic information corresponding to the identification information of the user terminal 82b. For example, in the case where the driving characteristic information corresponding to the identification information of the user terminal 82b indicates a tendency to perform dangerous driving, the auxiliary control unit 260 may set the "predetermined time" to be longer.

[0097] Figure 6 Schematically showing other situations in which the assistance control unit 260 provides assistance to traffic participants. Here, it is assumed that the attribute information of the user 80a has been stored in the storage device 280. That is, it is assumed that the identification information and driving characteristic information of the user 80a acquired by the camera 70 and / or the vehicle-mounted processing device 40a are stored in the storage device 280 in correspondence with the identification information of the user terminal 82a.

[0098] exist Figure 6In the illustrated situation, the traffic participants are the user 80a and the vehicle 20c. The vehicle 20c is equipped with an onboard processing device 40c. The onboard processing device 40c is similar to the onboard processing device 40a of the vehicle 20a and has a function of communicating with the support device 60.

[0099] The user terminal 82a transmits the position information and the identification information to the support device 60. Based on the identification information transmitted from the user terminal 82a, the support control unit 260 acquires the identification information and the walking characteristic information stored in the storage device 280 in correspondence with the identification information. Thus, the support control unit 260 determines that the user having the identification information and the walking characteristic information is present at the position transmitted from the user terminal 82a.

[0100] The vehicle-mounted processing device 40c transmits the location information to the assistance device 60. The assistance control unit 260 performs traffic assistance for the user 80a and the vehicle 20c based on the history of the location information transmitted from the vehicle-mounted processing device 40c, the history of the location information transmitted from the user terminal 82a, the identification information corresponding to the identification information of the user terminal 82a, and the walking characteristic information.

[0101] As an example, Figure 6 As shown, it is assumed that the user terminal 82a is located near a crosswalk. When the auxiliary control unit 260 determines that the user terminal 82a is located near a crosswalk based on the position information of the user terminal 82a and the onboard processing device 40c and the map information, and the user terminal 82a and the onboard processing device 40c are close to each other within a predetermined distance, and when the driving characteristic information corresponding to the identification information of the user terminal 82a includes information indicating a tendency to cross the road in a place where no crosswalk is provided near the crosswalk, the auxiliary control unit 260 sends auxiliary information to the user terminal 82a and the onboard processing device 40c to instruct the user terminal 82a and the onboard processing device 40c to output warning information.

[0102] The auxiliary information provided to the user terminal 82a may be information instructing the user terminal 82a to output, by voice, an alarm message meaning “a vehicle is approaching, please pay attention”. The auxiliary information provided to the user terminal 82a may be information instructing the user terminal 82a to output, by voice, an alarm message meaning “there is a crosswalk, please use the crosswalk” based on the walking characteristic information corresponding to the identification information of the user terminal 82a.

[0103] The auxiliary information provided to the vehicle-mounted processing device 40c may be information instructing the vehicle-mounted processing device 40c to output warning information to the effect of “pay attention to pedestrians” by sound or display. The auxiliary information provided to the vehicle-mounted processing device 40c may be information instructing output of warning information to the effect of “pay attention to pedestrians wearing red clothes” based on identification information corresponding to the identification information of the user terminal 82a. The auxiliary information provided to the vehicle-mounted processing device 40c may be information instructing further output of warning information to the effect of “pay attention to pedestrians crossing diagonally” based on walking characteristic information corresponding to the identification information of the user terminal 82a.

[0104] As combined Figure 5 and Figure 6 As in the examples described above, appropriate assistance can be provided to traffic participants based on the identification information and driving characteristics information of the vehicle 20b associated with the user terminal 82b and the identification information and walking characteristics of the user 80a associated with the user terminal 82a.

[0105] Figure 7 An example of the execution flow of the process related to the support control method performed by the imaging device 70 , the user terminal 82 , the vehicle-mounted processing device 40 a included in the vehicle 20 a , and the support device 60 is shown.

[0106] In S720, the user terminal 82a transmits position information indicating the current position of the user terminal 82a based on the signal received from the GNSS satellite together with the identification information of the user terminal 82a to the assistance device 60. The position information and the identification information are transmitted from the user terminal 82a to the assistance device 60 periodically.

[0107] In S722, the user terminal 82b transmits position information indicating the current position of the user terminal 82b based on the signal received from the GNSS satellite together with the identification information of the user terminal 82b to the assistance device 60. The position information and the identification information are transmitted from the user terminal 82b to the assistance device 60 periodically.

[0108] In S700, the vehicle-mounted processing device 40a sends the position information, identification information and walking characteristic information of the user 80a and / or the position information, identification information and driving characteristic information of the vehicle 20b obtained by analyzing the image captured by the camera possessed by the vehicle-mounted processing device 40a to the support device 60.

[0109] In S730 , the camera 70 transmits the position information, identification information, and walking characteristic information of the user 80 a and / or the position information, identification information, and driving characteristic information of the vehicle 20 b obtained by analyzing the image captured by the camera 70 to the support device 60 .

[0110] In S712, the auxiliary control unit 260 of the auxiliary device 60 performs matching based on the similarity of the position and displacement based on the history of the position information received from the user terminal 82, the camera 70, and the vehicle-mounted processing device 40a in S720, S722, S700, and S730. As a result, the auxiliary control unit 260 determines that the position information of the user terminal 82a matches the position information of the user 80a sent from the camera 70 and / or the vehicle-mounted processing device 40a, and further determines that the position information of the user terminal 82b matches the position information of the vehicle 20b sent from the camera 70 and / or the vehicle-mounted processing device 40a.

[0111] In S714, the storage control unit 240 stores the identification information of the user terminal 82b and the attribute information based on the identification information of the vehicle 20b and the driving characteristic information transmitted from the camera 70 and / or the vehicle-mounted processing unit 40a in the storage device 280. In S716, the storage control unit 240 stores the identification information of the user terminal 82a and the attribute information based on the identification information of the user 80a and the driving characteristic information transmitted from the camera 70 and / or the vehicle-mounted processing unit 40a in the storage device 280.

[0112] Figure 8 An example of the execution flow of the processing related to the assistance control method performed in the user terminal 82b, the assistance device 60, and the user terminal 82c is shown. Figure 8 In particular, this is an example of the execution flow when traffic assistance is performed for the user terminal 82c.

[0113] In S822, the user terminal 82b transmits position information indicating the current position of the user terminal 82b based on the signal received from the GNSS satellite together with the identification information of the user terminal 82b to the assistance device 60. The position information and the identification information are transmitted from the user terminal 82b to the assistance device 60 periodically.

[0114] In S800, the user terminal 82c transmits position information indicating the current position of the user terminal 82c based on the signal received from the GNSS satellite to the support device 60. The position information is transmitted from the user terminal 82c to the support device 60 periodically.

[0115] In S814, the assistance control unit 260 of the assistance device 60 determines whether to perform assistance based on the information sent from the user terminal 82a and the user terminal 82c. Figure 5As described in the above, when it is determined based on the history of the location information of the user terminal 82c and the map information that the user terminal 82c intends to cross the road 90, and when it is determined based on the history of the location information of the user terminal 82b and the user terminal 82c that the user terminal 82b and the user terminal 82c are close to each other within a predetermined distance, the assistance control unit 260 determines to assist the user terminal 82c. Figure 5 As described above, the assist control unit 260 can determine whether to perform assist based on the driving characteristic information stored in the storage device 280 in association with the identification information of the user terminal 82b.

[0116] If it is determined that assistance is to be performed, in S816, the assistance control unit 260 sends assistance information to the user terminal 82c via the communication device 290. Figure 5 As described in the examples above, the auxiliary information may be information that instructs the user terminal 82c to output alarm information. Figure 5 As described in the above description, the auxiliary information may be information based on the identification information and / or the driving characteristic information stored in the storage device 280 in correspondence with the identification information of the user terminal 82b.

[0117] When the user terminal 82c receives the auxiliary information from the auxiliary device 60, in S806, the user terminal 82c outputs alarm information based on the auxiliary information through the HMI (Human Machine Interface) function of the user terminal 82c. The user terminal 82c can output the alarm information by sound.

[0118] Fig. 9 An example of the execution flow of the process related to the support control method performed in the user terminal 82a, the support device 60, and the vehicle-mounted processing device 40c included in the vehicle 20c is shown. Fig. 9 In particular, this is an example of the execution flow when traffic assistance is performed on the in-vehicle processing device 40c and the user terminal 82a.

[0119] In S922, the user terminal 82a transmits position information indicating the current position of the user terminal 82a based on the signal received from the GNSS satellite together with the identification information of the user terminal 82a to the assistance device 60. The position information and the identification information are transmitted from the user terminal 82a to the assistance device 60 periodically.

[0120] In S900, the in-vehicle processing device 40c transmits position information indicating the current position of the vehicle 20c based on the signal received from the GNSS satellite to the support device 60. The position information is transmitted from the in-vehicle processing device 40c to the support device 60 periodically.

[0121] In S914, the assistance control unit 260 of the assistance device 60 determines whether to assist the in-vehicle processing device 40c and / or the user terminal 82a based on the information sent from the user terminal 82a and the in-vehicle processing device 40c. Figure 6 As described in the above, the assistance control unit 260 determines that the user terminal 82a is located near the crosswalk based on the location information of the user terminal 82a and the map information, and determines that the user terminal 82a and the vehicle processing device 40c are close to each other within a predetermined distance based on the history of the location information of the vehicle-mounted processing device 40c and the user terminal 82a, and determines that the user terminal 82a and the vehicle-mounted processing device 40c are close to each other. Figure 6 As described in the above description, the assistance control unit 260 can determine whether to perform assistance based on the walking characteristic information stored in the storage device 280 in association with the identification information of the user terminal 82a.

[0122] If it is determined that assistance is to be performed, the assistance control unit 260 sends assistance information to the vehicle-mounted processing device 40c via the communication device 290 in S916, and sends the assistance information to the user terminal 82a via the communication device 290 in S918. Figure 6 As described in the examples above, the auxiliary information may be information that instructs the vehicle-mounted processing device 40c and the user terminal 82c to output alarm information. Figure 6 As described in the examples above, the auxiliary information may be information based on identification information and / or driving characteristic information stored in the storage device 280 in correspondence with the identification information of the user terminal 82a. If the vehicle-mounted processing device 40c receives the auxiliary information from the auxiliary device 60, in S906, the vehicle-mounted processing device 40c outputs warning information based on the auxiliary information through the HMI function of the vehicle-mounted processing device 40c. If the user terminal 82a receives the auxiliary information from the auxiliary device 60, in S926, the user terminal 82a outputs warning information based on the auxiliary information through the HMI function of the user terminal 82a.

[0123] According to the assistance system 10 described above, the identification information and / or walking characteristic information of the pedestrian acquired by the shooting device on the vehicle with shooting function and communication function and / or the traffic infrastructure can be maintained in correspondence with the identification information of the user terminal carried by the pedestrian. Thus, even if there is no vehicle with shooting function and / or the shooting device on the traffic infrastructure around the user terminal, the identification information and / or walking characteristic information of the user holding the user terminal can be determined based on the identification information of the user terminal carried by the pedestrian, and appropriate and effective assistance can be provided to the traffic participants based on the determined identification information and / or walking characteristic information. In addition, the identification information and / or driving characteristic information of the vehicle acquired by the shooting device on the vehicle with shooting function and communication function and / or the traffic infrastructure can be maintained in correspondence with the identification information of the user terminal moving with the vehicle. Thus, even if there is no vehicle with shooting function and / or the shooting device on the traffic infrastructure around the vehicle, the identification information and / or driving characteristic information of the vehicle can be determined based on the identification information of the user terminal located in the vehicle, and appropriate and effective traffic assistance can be provided to the traffic participants based on the determined identification information and / or driving characteristic information.

[0124] Fig.10 An example of a computer 2000 that can fully or partially embody multiple embodiments of the present invention is shown. The program installed in the computer 2000 can cause the computer 2000 to function as a device such as the auxiliary device 60 or each unit of the device involved in the embodiment, perform operations associated with the device or each unit of the device, and / or perform a process involved in the embodiment or steps of the process. Such a program can be executed by the CPU 2012 in order to cause the computer 2000 to perform the processing flow described in this specification and specific operations associated with some or all of the functional blocks in the block diagram.

[0125] The computer 2000 according to the present embodiment includes a CPU 2012 and a RAM 2014, which are connected to each other via a main controller 2010. The computer 2000 further includes a ROM 2026, a flash memory 2024, a communication interface 2022, and an input / output chip 2040. The ROM 2026, the flash memory 2024, the communication interface 2022, and the input / output chip 2040 are connected to the main controller 2010 via the input / output controller 2020.

[0126] The CPU 2012 operates according to the programs stored in the ROM 2026 and the RAM 2014 , thereby controlling each unit.

[0127] The communication interface 2022 communicates with other electronic devices via a network. The flash memory 2024 stores programs and data used by the CPU 2012 in the computer 2000. The ROM 2026 stores a startup program executed by the computer 2000 when activated, and / or a program dependent on the hardware of the computer 2000. The input / output chip 2040 can also connect various input / output units such as a keyboard, a mouse, and a monitor to the input / output controller 2020 via an input / output port such as a serial port, a parallel port, a keyboard port, a mouse port, a monitor port, a USB port, and an HDMI (registered trademark) port.

[0128] The program is provided via a computer-readable storage medium such as a CD-ROM, DVD-ROM, or a USB flash drive, or a network. RAM 2014, ROM 2026, or flash memory 2024 is an example of a computer-readable storage medium. The program is installed in flash memory 2024, RAM 2014, or ROM 2026 and executed by CPU 2012. The information processing described in these programs is read by computer 2000, and the cooperation between the program and the various types of hardware resources described above is realized. The device or method can be constructed by implementing the operation or processing of information in accordance with the use of computer 2000.

[0129] For example, when the computer 2000 communicates with an external device, the CPU 2012 may execute a communication program loaded into the RAM 2014, and based on the processing described in the communication program, instruct the communication interface 2022 to perform communication processing. Under the control of the CPU 2012, the communication interface 2022 reads the transmission data stored in the transmission buffer processing area provided in the RAM 2014 and the recording medium such as the flash memory 2024, transmits the read transmission data to the network, and writes the reception data received from the network to the reception buffer processing area provided on the recording medium.

[0130] In addition, the CPU 2012 can read all or a necessary part of a file or database stored in a recording medium such as the flash memory 2024 to the RAM 2014, and execute various processes on the data in the RAM 2014. The CPU 2012 then writes the processed data back to the recording medium.

[0131] Various types of information such as programs, data, tables, and databases can be saved to a recording medium and applied to information processing. CPU2012 can perform various operations, information processing, conditional judgment, conditional branching, unconditional branching, information retrieval / replacement, etc., which are specified by the instruction sequence of the program, on the data read from RAM2014, and write the results back to RAM2014. In addition, CPU2012 can retrieve information in files, databases, etc. in the recording medium. For example, when multiple items having attribute values ​​of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, CPU2012 can retrieve an item that specifies the attribute value of the first attribute and is consistent with the condition from the multiple items, read the attribute value of the second attribute stored in the item, and thereby obtain the attribute value of the second attribute associated with the first attribute that meets the pre-set condition.

[0132] The program or software module described above can be stored in a computer-readable storage medium on or near the computer 2000. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium. The program stored in the computer-readable storage medium can be provided to the computer 2000 via a network.

[0133] The program installed in the computer 2000 and causing the computer 2000 to function as the auxiliary device 60 can be run in the CPU 2012 or the like, thereby causing the computer 2000 to function as each unit of the auxiliary device 60 (e.g., the auxiliary control device 200, etc.). The information processing described in these programs is read into the computer 2000, thereby functioning as a specific unit in which software and the various hardware resources described above cooperate, that is, each unit of the auxiliary device 60. And, by using these specific units to realize the calculation or processing of information corresponding to the purpose of use of the computer 2000 in this embodiment, a unique auxiliary device 60 corresponding to the purpose of use is constructed.

[0134] Various embodiments are described with reference to block diagrams, etc. In the block diagram, each functional block may represent (1) a step of a process for performing an operation or (2) each unit of a device having the function of performing an operation. Specific steps and each unit may be implemented by a dedicated circuit, a programmable circuit supplied together with computer-readable instructions stored on a computer-readable storage medium, and / or a processor supplied together with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuit may include digital and / or analog hardware circuits, and may also include integrated circuits (ICs) and / or discrete circuits. The programmable circuit may include reconfigurable hardware circuits including logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc., including memory elements.

[0135] The computer-readable storage medium may include any tangible device capable of storing instructions to be executed by an appropriate device, and as a result, the computer-readable storage medium having the instructions stored therein constitutes at least a part of a product containing instructions executable to implement a unit for performing the operations specified in the process flow or block diagram. As examples of computer-readable storage media, electrical storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. may be included. As more specific examples of computer-readable storage media, floppy disks (registered trademark), floppy disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), electrically erasable programmable read-only memories (EEPROM), static random access memories (SRAM), compact disk read-only memories (CD-ROM), digital versatile disks (DVD), Blu-ray (registered trademark) optical disks, memory sticks, integrated circuit cards, etc. may be included.

[0136] Computer-readable instructions may include assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine delegate instructions, microcode, firmware instructions, state setting data, or any source code or object code described in any combination of one or more programming languages ​​including object-oriented programming languages ​​such as Smalltalk (registered trademark), JAVA (registered trademark), C++, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0137] Computer readable instructions are provided to a processor or programmable circuit of a general-purpose computer, a special-purpose computer, or other programmable data processing device via a local or local area network (LAN), a wide area network (WAN) such as the Internet, and the computer readable instructions can be executed to implement a unit for performing the operations specified in the described processing flow or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0138] The present invention has been described above using the embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various changes or improvements can be made to the above embodiments. It is obvious from the description of the claims that such changes or improvements can also be included in the technical scope of the present invention.

[0139] Regarding the execution order of each process of actions, processes, steps, and steps in the devices, systems, programs, and methods shown in the claims, specifications, and drawings, it should be noted that there is no special indication of "before", "preceding", etc., and in addition, as long as the output of the previous process is not used in the subsequent process, it can be implemented in any order. For the action flow in the claims, specifications, and drawings, even if "first", "next", etc. are used for convenience, it does not mean that it must be implemented in such an order.

[0140] [Description of Reference Numerals]

[0141] 10Assistance Systems

[0142] 20 vehicles

[0143] 40 On-board processing device

[0144] 50Traffic lights

[0145] 60 auxiliary devices

[0146] 70 Camera

[0147] 80 users

[0148] 82 user terminals

[0149] 90 Road

[0150] 200 Auxiliary Control Devices

[0151] 202 Acquisition Department

[0152] 210 First Acquisition Department

[0153] 220 Second Acquisition Unit

[0154] 240 Storage Control Unit

[0155] 260 Auxiliary Control Unit

[0156] 280 storage devices

[0157] 290 communication device

[0158] 2000 Computer

[0159] 2010 Main Controller

[0160] 2012CPU

[0161] 2014 RAM

[0162] 2020 Input / Output Controller

[0163] 2022 Communication Interface

[0164] 2024 Flash

[0165] 2026ROM

[0166] 2040 input / output chips.

Claims

1. An auxiliary control device, characterized in that: have: A first acquisition unit, which acquires identification information of a user terminal associated with a user and location information of the user terminal; a second acquisition unit that acquires movement characteristic information of the moving object and position information of the moving object acquired by analyzing an image of the moving object acquired by a photographing device; a storage control unit that controls, when the position information of the user terminal acquired by the first acquisition unit matches the position information of the mobile object acquired by the second acquisition unit, to store the movement characteristic information of the mobile object acquired by the second acquisition unit in correspondence with the identification information of the user terminal acquired by the first acquisition unit; as well as An assistance control unit performs control related to assistance for traffic participants based on the identification information of the user terminal and the movement characteristic information of the mobile object that are correspondingly stored under the control of the storage control unit.

2. The auxiliary control device according to claim 1, wherein: Even in the case where the second acquisition unit cannot acquire the mobile characteristic information of the mobile object, when the identification information of the user terminal is acquired by the first acquisition unit, the auxiliary control unit provides assistance to traffic participants based on the identification information of the user terminal and the mobile characteristic information of the mobile object stored correspondingly under the control of the storage control unit.

3. The auxiliary control device according to claim 1 or 2, wherein: The assist control unit performs control for notifying traffic participants other than the user of the movement characteristic information of the moving object.

4. The auxiliary control device according to claim 1 or 2, wherein: The moving object includes a pedestrian.

5. The auxiliary control device according to claim 4, wherein: The movement characteristic information includes information indicating a behavior tendency of the pedestrian in relation to travel on a road.

6. The auxiliary control device according to claim 4, wherein: The movement characteristic information includes information indicating whether the pedestrian has a tendency to cross the road in a place where no crosswalk is provided.

7. The auxiliary control device according to claim 4, wherein: The movement characteristic information includes information indicating whether the pedestrian has a tendency to cross the road in a location where no crosswalk is provided near a crosswalk.

8. The auxiliary control device according to claim 4, wherein. The movement characteristic information includes information indicating whether the pedestrian has a tendency to cross the road without following the indication state of the traffic light.

9. The auxiliary control device according to claim 4, wherein: The movement characteristic information includes information indicating whether the pedestrian tends to start crossing the road in a time shorter than a predetermined time when the instruction of the traffic light is switched from a proceed instruction to a stop instruction.

10. The auxiliary control device according to claim 1 or 2, wherein: The moving object includes a vehicle.

11. The auxiliary control device according to claim 10, wherein: The movement characteristic information includes information related to dangerous driving of the vehicle.

12. The auxiliary control device according to claim 1 or 2, wherein: The storage control unit performs control for deleting the identification information of the user terminal and the movement characteristic information of the mobile object stored in correspondence with each other every time a predetermined time elapses.

13. The auxiliary control device according to claim 1 or 2, wherein: When the location information of the user terminal newly acquired by the first acquisition unit matches the location information of the mobile object newly acquired by the second acquisition unit, the storage control unit performs control for updating and establishing the identification information of the user terminal and the movement characteristic information of the mobile object stored correspondingly.

14. The auxiliary control device according to claim 1 or 2, wherein: The second acquisition unit further acquires identification information of the moving object and position information of the moving object acquired by analyzing an image of the moving object acquired by the photographing device, When the position information of the user terminal acquired by the first acquisition unit matches the position information of the mobile object acquired by the second acquisition unit, the storage control unit performs control for storing the movement characteristic information and identification information of the mobile object acquired by the second acquisition unit in correspondence with the identification information of the user terminal acquired by the first acquisition unit. The assistance control unit performs control related to assistance for traffic participants based on the identification information of the user terminal, the movement characteristic information and the recognition information of the mobile object correspondingly stored under the control of the storage control unit.

15. The auxiliary control device according to claim 14, wherein: The moving object includes a pedestrian. The identification information includes information indicating the color of the pedestrian. The movement characteristic information includes information indicating a behavior tendency of the pedestrian in relation to travel on a road.

16. The auxiliary control device according to claim 14, wherein: The moving object includes a vehicle, The identification information includes information indicating the color of the vehicle. The movement characteristic information includes information related to dangerous driving of the vehicle.

17. An auxiliary control method, characterized in that: have: A step of acquiring identification information of a user terminal associated with the user and location information of the user terminal; a step of acquiring movement characteristic information of the moving object and position information of the moving object acquired by analyzing an image of the moving object acquired by a photographing device; When the location information of the user terminal matches the location information of the mobile object, a control step is performed for storing the movement characteristic information of the mobile object in correspondence with the identification information of the user terminal; as well as A step of performing control related to assistance for traffic participants is performed based on the identification information of the user terminal and the movement characteristic information of the mobile object established to be correspondingly stored by the step of performing control for storage.

18. A computer-readable storage medium, characterized in that: A program for causing a computer to function as the auxiliary control device according to any one of claims 1 to 16 is stored.

Images