Monitoring device, monitoring method and monitoring program

Abstract

To provide a monitoring device which notifies a user passing along a road of a road traffic rule or a road traffic violation.SOLUTION: A monitoring device comprises: an estimation section which estimates a road traffic rule regarding the surroundings on the basis of information relating to a user and the surroundings of the user on a road; and a notification section which notifies the user of behavior information corresponding to the traffic road rule.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a monitoring device, a monitoring method, and a monitoring program. [Background technology]

[0002] Patent Document 1 discloses an information providing system that detects traffic violations by analyzing images captured by a plurality of cameras installed on a road. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-157021 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the information provision system described in Patent Document 1 only identifies users who have committed traffic violations in the past, and is unable to notify users traveling on the road of road traffic rules or road traffic violations.

[0005] The present disclosure has an object to provide a monitoring device that solves the above problems. [Means for solving the problem]

[0006] According to a first aspect of the present disclosure, there is provided a monitoring device including an estimation unit that estimates road traffic rules for a user based on information about the user and the user's surroundings on a road, and a notification unit that notifies the user of behavioral information in accordance with the road traffic rules.

[0007] According to a second aspect of the present disclosure, there is provided a monitoring method including an estimation step of estimating road traffic rules for the surroundings based on information about a user and the user's surroundings on a road, and a notification step of notifying the user of behavioral information in accordance with the road traffic rules.

[0008] According to a third aspect of the present disclosure, there is provided a monitoring program comprising an estimation step of estimating road traffic rules for the surroundings based on information about a user and the user's surroundings on a road, and a notification step of notifying the user of behavioral information in accordance with the road traffic rules. Effect of the Invention

[0009] According to the present disclosure, it is possible to notify a user traveling on a road of road traffic rules or road traffic violations. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram illustrating a monitoring system according to one embodiment of the present disclosure. [Diagram 2] FIG. 2 is a block diagram illustrating a terminal according to an embodiment of the present disclosure. [Diagram 3] FIG. 2 is a block diagram illustrating a server according to one embodiment of the present disclosure. [Figure 4] FIG. 2 is a block diagram showing functional units of a terminal according to an embodiment of the present disclosure. [Figure 5A] FIG. 2 is a schematic diagram illustrating an example of a user's road traffic according to one embodiment of the present disclosure. [Figure 5B] FIG. 2 is a schematic diagram illustrating an example of a screen display of a map application according to an embodiment of the present disclosure. [Figure 5C] FIG. 2 is a schematic diagram illustrating an example of a screen display of a terminal according to an embodiment of the present disclosure. [Figure 5D] FIG. 2 is a schematic diagram illustrating an example of a screen display of a terminal according to an embodiment of the present disclosure. [Figure 5E] FIG. 2 is a schematic diagram illustrating an example of a screen display of a terminal according to an embodiment of the present disclosure. [Figure 5F] FIG. 2 is a schematic diagram illustrating an example of a screen display of a terminal according to an embodiment of the present disclosure. [Figure 6A] 1 is a flowchart illustrating a process of a monitoring system according to an embodiment of the present disclosure. [Figure 6B] 1 is a flowchart illustrating a process of a monitoring system according to an embodiment of the present disclosure. [Figure 6C] 1 is a flowchart illustrating a process of a monitoring system according to an embodiment of the present disclosure. [Figure 7] FIG. 1 is a block diagram showing a monitoring device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] A monitoring device according to an embodiment of the present disclosure will now be described with reference to the drawings.

[0012] 1 is a schematic diagram illustrating a monitoring system according to an embodiment of the present disclosure. The monitoring system includes a terminal 1, a server 3, and a server 4. In this embodiment, the monitoring system monitors the behavior of a user with a vehicle 2 on a road.

[0013] The terminal 1 is a mobile terminal owned by a user. The terminal 1 may be, for example, a smartphone, a tablet terminal, a wearable terminal such as smart glasses, or the like. The terminal 1 may be configured by combining a plurality of electronic devices, for example, the terminal 1 may be configured by combining a smartphone and a wearable terminal. The terminal 1 is preferably connected to the network 9 by wireless communication. The terminal 1 is communicatively connected to the server 3 and the server 4 via the network 9. The terminal 1 may be communicatively connected to an in-vehicle device or the like possessed by the vehicle 2 by wired communication or wireless communication. The terminal 1 may be communicatively connected to a facility 5 provided on a road via the network 9, or may be communicatively connected directly to the facility 5 without via the network 9.

[0014] The vehicle 2 may be a kick scooter, bicycle, moped, motorcycle, or the like driven by a user and used as a means of transportation by the user. The vehicle 2 does not have to be a vehicle owned by the user, and may be, for example, a bicycle provided by a bicycle rental service. In the present disclosure, the user does not necessarily have to own the vehicle 2, and the user may travel on foot.

[0015] The server 3 is an information processing device maintained and managed by an administrator who monitors the user's actions. The server 3 may be a hardware server or a software server. When the server 3 is a hardware server, the server 3 may be, for example, a network server or a cloud server. When the server 3 is a software server, the server 3 may be, for example, server software or a server program. The administrator is an organization that manages the area in which the user acts, and may be, for example, a local government, the police, etc.

[0016] The server 4 is an information processing device maintained and managed by a service provider that provides a notification service of road traffic rules and the like to a user. The server 4 may be a hardware server or a software server. When the server 4 is a hardware server, the server 4 may be, for example, a network server or a cloud server. When the server 4 is a software server, the server 4 may be, for example, server software or a server program. The service provider may be, for example, a map application service company, a logistics company, a delivery company, or the like.

[0017] Note that server 3 and server 4 may be the same server. For example, if the user is an employee of a logistics company or a delivery company and is a delivery person, the logistics company or the delivery company maintains and manages server 3 or server 4. If server 3 and server 4 are the same server, the logistics company or the delivery company provides a notification service for road traffic rules, etc. to the user and monitors the user's behavior. In this embodiment, a case will be described where server 3 and server 4 are the same server, and the provider of the notification service for road traffic rules, etc. and the administrator who monitors the user's behavior are the same person.

[0018] The facility 5 may be, but is not limited to, an accessory installed on a road, a traffic light, a road sign, a road lighting, etc. The facility 5 may be, for example, a bus stop sign, a taxi stand guide sign, etc. installed along a road, or a railroad crossing, a barrier, an alarm, etc. installed on a railway that intersects with the road. The facility 5 may also be a road marking, an indication display, a center line, etc. drawn on the road. The facility 5 is not limited to the above examples, and may be, for example, a facility such as a nursery school, a kindergarten, a school, etc. The facility 5 may include information that can identify the road traffic rules in the user's surroundings. If the facility 5 is capable of communicating with the terminal 1, the facility 5 may transmit information such as road traffic rules to the terminal 1 via a beacon signal, etc.

[0019] 2 is a block diagram showing a terminal 1 according to an embodiment of the present disclosure. The terminal 1 includes a CPU 11, a ROM 12, a RAM 13, a storage device 14, a display device 15, a touch panel 16, an imaging device 17, an input / output IF (Interface) 18, and a communication IF 19. The CPU 11, the ROM 12, the RAM 13, the storage device 14, the display device 15, the touch panel 16, the imaging device 17, the input / output IF 18, and the communication IF 19 are connected to each other via a bus so as to be able to communicate with each other.

[0020] CPU 11 is a central processing unit. CPU 11 controls each part of terminal 1 using an application program. ROM 12 is a read only memory. ROM 12 is made up of non-volatile memory, and stores application programs for controlling each part of terminal 1. RAM 13 is a random access memory. RAM 13 provides a memory area necessary for the operation of CPU 11. Storage device 14 is a large-capacity storage device such as a hard disk drive.

[0021] The display device 15 may be a liquid crystal display or an organic EL (Electroluminescence) display. The touch panel 16 is a user interface that accepts input from a user, and may be formed integrally with the display device 15. The imaging device 17 is an image capturing device such as a camera, and captures an image of the user's surroundings according to a user's operation or a command from an application or the like. The imaging device 17 includes an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a control circuit, a signal processing circuit, and the like.

[0022] The input / output IF 18 is an input / output interface that receives voice input from the user, outputs voice to the user, and transmits and receives data between the terminal 1 and other devices. The input / output IF 18 may include a mouse, a trackball, a keyboard, a speaker, and the like. The user may operate the terminal 1 not only through the touch panel 16 but also through the input / output IF 18. The communication IF 19 performs data communication between the terminal 1 and other devices via wired communication and / or wireless communication. The communication IF 19 has a GPS (Global Positioning System) receiver, and the terminal 1 can acquire user position information via the communication IF 19. For example, the terminal 1 may transmit and receive information such as the speed of the vehicle 2 to and from the vehicle 2 via the communication IF 19, and the terminal 1 may transmit and receive information such as the current position of the user to and from the server 3 and the server 4. The receiver of the communication IF 19 is not limited to a GPS, and may be a satellite positioning system such as a general GNSS (Global Navigation Satellite System).

[0023] 3 is a block diagram showing a server 3 according to an embodiment of the present disclosure. The server 3 includes a CPU 31, a ROM 32, a RAM 33, a storage device 34, a display device 35, a touch panel 36, an input / output IF 37, and a communication IF 38. The CPU 31, the ROM 32, the RAM 33, the storage device 34, the display device 35, the touch panel 36, the input / output IF 37, and the communication IF 38 are connected to each other via a bus so as to be able to communicate with each other.

[0024] CPU 31 is a central processing unit. CPU 31 controls each part of server 3 using an application program. ROM 32 is a read-only memory. ROM 32 is made up of non-volatile memory, and stores application programs for controlling each part of server 3. RAM 33 is a random access memory. RAM 33 provides a memory area necessary for the operation of CPU 31. Storage device 34 is a large-capacity storage device such as a hard disk drive.

[0025] The display device 35 may be a liquid crystal display, an organic EL display, etc. The touch panel 36 is a user interface that receives input from an administrator of the server 3, and may be formed integrally with the display device 35.

[0026] The input / output IF 37 is a user interface that accepts input from an administrator of the server 3. The input / output IF 18 may include a mouse, a trackball, a keyboard, a speaker, and the like. A user may operate the server 3 not only through the touch panel 36 but also through the input / output IF 37. The communication IF 38 performs data communication between the server 3 and other devices via wired communication and / or wireless communication. In this embodiment, the configuration of the server 4 is the same as that of the server 3, so a description of the configuration of the server 4 will be omitted.

[0027] Fig. 4 is a block diagram showing the functional unit 100 of the terminal 1 according to an embodiment of the present disclosure. The functional unit 100 realizes the function of a monitoring device in the terminal 1. The functional unit 100 may be implemented by hardware, software, or firmware. When the functional unit 100 is implemented by hardware, the terminal 1 may be provided with a hardware configuration for implementing the functional unit 100 in addition to the configuration shown in Fig. 2, or the configuration shown in Fig. 2 and the hardware configuration for implementing the functional unit 100 may be separated, and the terminal 1 and the separately configured hardware configuration may be connected to each other so as to be able to communicate with each other.

[0028] When the functional unit 100 is implemented by software or firmware, the storage device 14 stores a program for implementing the functional unit 100. The RAM 13 reads out the program, and the CPU 11 executes the program, thereby realizing the function of the functional unit 100. In the following, this embodiment will be described taking as an example a case where the functional unit 100 is implemented by software or firmware. That is, the terminal 1 stores the functional unit 100 as a monitoring application.

[0029] The functional unit 100 includes a control unit 101, an acquisition unit 102, an image processing unit 103, a voice processing unit 104, an estimation unit 105, a notification unit 106, a display unit 107, and a database 108. The control unit 101 controls each unit of the functional unit 100, and manages data transmission and reception between each unit of the functional unit 100, processing order, notification timing, data transmission and reception between the hardware of the terminal 1 and the functional unit 100, etc.

[0030] The acquisition unit 102 acquires images, audio signals, sensor signals, position information, and the like from hardware included in the terminal 1 or other devices connected to the terminal 1. The acquisition unit 102 may also acquire acceleration, biometric information of the user, and the like from a wearable terminal or the like. The acquisition unit 102 may acquire acceleration from the wearable terminal and store it in the storage device 44 or the like as gait information of the user. If the terminal 1 is equipped with an accelerometer, the acquisition unit 102 may acquire acceleration, and the like from the terminal 1. If the vehicle 2 is equipped with a drive recorder or the like, the acquisition unit 102 may acquire images and videos from the drive recorder. In this embodiment, the acquisition unit 102 stores the image captured by the imaging device 17, the audio signal acquired via the input / output IF 18, and the information acquired via the communication IF 19 in the storage device 44 or the like. The acquisition unit 102 may directly output the acquired information to the image processing unit 103, the audio processing unit 104, and the estimation unit 105.

[0031] The image processing unit 103 reads out an image stored in the storage device 44 or the like, and analyzes information contained in the image. The image processing unit 103 extracts features in the image, recognizes characters in the image, recognizes objects in the image, and the like, based on image templates and the like stored in the database 108, for example. The image templates can be, for example, traffic lights, bus stop signs, taxi stand guide signs, barriers, alarms, warning lights, road signs, road markings, instruction displays, and the like. The image processing unit 103 outputs information such as features, recognized characters, and recognized objects to the estimation unit 105. The image processing unit 103 also outputs information that the image template has been detected to the estimation unit 105.

[0032] The voice processing unit 104 reads out the voice signal stored in the storage device 44 or the like, and analyzes information included in the voice signal. For example, the voice processing unit 104 executes extraction of features from the voice signal, separation of the signal source of the voice signal, and determination of similarity with the voice signal template, based on a voice signal template or the like included in the database 108. The voice signal template may be, for example, an ambulance siren, a fire engine siren, a police car siren, a warning sound at a railroad crossing, or a sound signal at a pedestrian crossing. The voice processing unit 104 outputs information such as the features and the separated signal source voice to the estimation unit 105. In addition, when the voice signal and the voice signal template are similar, the voice processing unit 104 outputs information that the voice template has been detected to the estimation unit 105. When the voice processing unit 104 detects a warning sound or the like, the control unit 101 may transmit a command to the CPU 11 to start the imaging device 17, and the image processing unit 103 may execute image processing.

[0033] The estimation unit 105 estimates information on the road on which the user travels, the user's traffic conditions on the road, the road traffic rules applied to the road, the user's state, the road traffic rules violated by the user, the actions the user should take, etc., based on the position information, the information included in the image, and the information included in the voice, and outputs the estimation result to the notification unit 106. Based on the user's current position information in the map application, the road traffic rules violated by other users in the past, and the records of traffic accidents that have occurred in the past, the estimation unit 105 estimates the road traffic rules, etc. that the user may violate. Based on the estimated road traffic rules, the estimation unit 105 may output to the notification unit 106 a route on which the user does not violate the road traffic rules, etc., or a route on which the user is less likely to cause a traffic accident. In addition, when the user repeatedly violates road traffic rules, the estimation unit 105 may evaluate the habituality of the user's violations, etc., based on the road traffic rules violated by the user in the past, and output the evaluation amount of the habituality of the user's violations to the notification unit 106. Furthermore, the estimation unit 105 may determine the user's state based on the user's biometric information, etc. For example, the estimation unit 105 may determine whether or not the user is in a health state suitable for riding in the vehicle 2 based on the pulse rate of the user acquired from a wearable device or the like. It may also determine whether or not the user is riding in the vehicle 2 based on the gait of the user acquired from a wearable device or the like.

[0034] The notification unit 106 outputs notification information to a user, an administrator, a service provider, etc., based on the information received from the estimation unit 105. The notification information may be, for example, information on the road on which the user travels, the user's traffic status on the road, road traffic rules that apply to the road, road traffic rules that the user has violated, actions that the user should take, etc. The notification unit 106 may notify the user of the notification information by voice via the input / output IF 18, or the display unit 107 may generate display information based on the notification information and notify the user by image via the display device 15.

[0035] The display unit 107 receives the notification information from the notification unit 106 and generates display information based on the notification information. The display unit 107 displays the display information on the display device 15. The display unit 107 can also combine an image or video acquired by the imaging device 17 with a display of a map application, and display the combined image or video on the display device 15 together with the notification information.

[0036] The database 108 stores image templates, audio signal templates, guidance voices, maps, installation information of the facilities 5, road traffic rules, records of past traffic accidents, etc. The database 108 may store information on road traffic rules that the user has violated. The functional unit 100 may include the database 108 separately. When the functional unit 100 includes the database 108 separately, the acquisition unit 102 may acquire data included in the database 108 from the server 3, the server 4, or the facility 5. When the facility 5 includes the database 108, the acquisition unit 102 may acquire data such as road traffic rules included in the database 108 via the network 9.

[0037] The terminal 1 notifies the user of notification information according to the road conditions, thereby making it possible to notify the user traveling on the road of road traffic rules or road traffic violations. This makes it possible to prevent the user from violating the road traffic rules and from encountering a traffic accident, etc., and enables the user to travel the road safely.

[0038] FIG. 5A is a schematic diagram showing an example of road traffic of a user according to an embodiment of the present disclosure, and FIG. 5B is a schematic diagram showing an example of a screen display of a map application according to an embodiment of the present disclosure. FIGS. 5C to 5F are schematic diagrams showing an example of a screen display on a display device 15 according to an embodiment of the present disclosure. FIGS. 6A to 6C are flowcharts showing the processing of a monitoring system according to an embodiment of the present disclosure. The processing of the monitoring system of this embodiment will be described with reference to FIGS. 5A to 5F and 6A to 6C. This embodiment will be described assuming that a user rides a bicycle and travels along the road, and temporarily dismounts from the bicycle depending on road traffic conditions and travels along the road on foot.

[0039] 5A to 5F, the road 6 is a one-way, one-lane public road. In one lane, the road 6 includes a roadway 61, a bicycle path (bicycle-only lane) 62, and a sidewalk 63. The roadway 61 is a lane on which ordinary vehicles, large vehicles, motorcycles, mopeds, etc. travel. A white dashed line 512 separates the roadway 61 from the bicycle path 62. The bicycle path 62 is a lane on which bicycles, kick scooters, etc. can travel, and a lane sign 513 clearly indicates that the bicycle path 62 is a bicycle-only lane. A white line 514 separates the bicycle path 62 from the sidewalk 63. The sidewalk 63 is a lane on which pedestrians travel, and is an exceptional lane on which bicycles, kick scooters, etc. can also travel. In this embodiment, the white line 514 separates the sidewalk 63 from the bicycle path 62, but the sidewalk 63 and the bicycle path 62 may also be separated by a step, a guardrail, or the like.

[0040] 5A to 5F, the roadway 61, bicycle path 62, sidewalk 63, white dashed line 512, lane marking 513, and white line 514 located on the left side are called roadway 61L, bicycle path 62L, sidewalk 63L, white dashed line 512L, lane marking 513L, and white line 514L, respectively. In Fig. 5A to 5F, the roadway 61, bicycle path 62, sidewalk 63, white dashed line 512, lane marking 513, and white line 514 located on the right side are called roadway 61R, bicycle path 62R, sidewalk 63R, white dashed line 512R, lane marking 513R, and white line 514R, respectively. The roadway 61L and the roadway 61R are separated by a center line 511. The center line 511, the white dashed line 512L, the white dashed line 512R, the lane markings 513L, the white line 514L, etc. are collectively referred to as road markings 51.

[0041] The track 52 is a lane on which railroad vehicles and the like pass, and is provided so as to intersect with the road 6. The alarm 521 and the crossing gate 522 are provided in the vicinity of the area where the track 52 and the road 6 intersect. Before and after a railroad vehicle passes the track 52, the alarm 521 operates the crossing gate 522 to restrict pedestrians or passing vehicles from crossing the track 52. The alarm 521 also notifies the vicinity of the alarm 521 that a railroad vehicle is approaching the track 52 by emitting an alarm sound 523. It is desirable that at least one alarm 521 is provided on each side of the road 6. The alarm 521 provided on one side of the road 6 and corresponding to the roadway 61L, the bicycle path 62L, and the sidewalk 63L is referred to as the alarm 521L. The alarm 521 provided on the other side of the road 6 and corresponding to the roadway 61R, the bicycle path 62R, and the sidewalk 63R is referred to as the alarm 521R. The alarm device 521L is provided near the area where the roadway 61L and the railroad track 52 intersect, and the alarm device 521R is provided near the area where the roadway 61R and the railroad track 52 intersect.

[0042] The crossing gate 522 is a traffic restriction device. When a train or the like travels on the track 52 and passes through the road 6, the crossing gate 522 uses a barrier to block traffic on the roadway 61, the bicycle path 62, and the sidewalk 63. It is desirable to provide at least one crossing gate 522 on each side of the track 52. The crossing gate 522 provided on one side of the road 6 is referred to as crossing gate 522L.

[0043] In this embodiment, bicycles are classified as light vehicles, and although they can exceptionally ride on the sidewalk 63, the following embodiment will be described assuming that they ride on the bicycle path 62 in principle. When a bicycle rides on the roadway 61L, it must ride on the left side of the center of the roadway 61L. When multiple roadways 61L are provided on the road 6, it must ride on the left side of the roadway 61L from the center of the roadway 61L, of the roadway 61L located on the leftmost side.

[0044] When a bicycle travels on the sidewalk 63, it must travel slowly on the sidewalk 63. Traveling slowly means a speed at which a vehicle or the like can stop immediately, for example, a speed of 10 km / h or less with a stopping distance of 1 m or less. On sidewalks 63 where a "Normal bicycles allowed on sidewalks" sign is posted, bicycles can travel on sidewalk 63L. However, when a bicycle travels on sidewalk 63L or sidewalk 63R, the bicycle must travel close to the roadway 61L on sidewalk 63L, and the bicycle must travel close to the roadway 61R on sidewalk 63R.

[0045] In the road 6 where a sign for "normal bicycle lane" is installed, bicycles must travel on the bicycle lane 62 in the direction of the arrow. In Figs. 5A to 5F, when a bicycle travels from the front to the back, the bicycle must travel on the bicycle lane 62L. In this embodiment, since the road 6 is configured to have a bicycle lane 62 with a sign for "normal bicycle lane", the embodiment will be described assuming that bicycles are not allowed to travel on the roadway 61 and the sidewalk 63. That is, the embodiment will be described assuming that the following road traffic rules are applied. (A) Pedestrians can travel on the sidewalk 63L or 63R regardless of the direction of travel. (B) In Figs. 5A to 5F, when a vehicle travels from the front to the back, it must travel on the roadway 61L, and when a vehicle travels from the back to the front, it must travel on the roadway 61R. (C) In Figures 5A to 5F, when a bicycle travels from the front to the back, it must travel on bicycle path 62L, and when a bicycle travels from the back to the front, it must travel on bicycle path 62R.

[0046] The above-mentioned configuration of roads and road traffic rules is an example of an embodiment in Japan, and the configuration of roads and road traffic rules is not limited to this. The configuration of roads and road traffic rules may be the configuration of roads and road traffic rules in a country other than Japan. For example, in a country where the road traffic rules require vehicles to keep to the right, the configuration of roads and road traffic rules may be different from the above-mentioned configuration.

[0047] First, the user operates the terminal 1 to start the monitoring application (step S101). The user may input information about the wearable terminal, information about the vehicle 2, and the like, to the monitoring application. For example, if the vehicle 2 is a bicycle, the user may input to the monitoring application that the vehicle 2 is a bicycle. The user may operate the terminal 1 to start the monitoring application, or the administrator may start the monitoring application by remote control or the like. When the monitoring application is started, the acquisition unit 102 acquires the user's moving direction, the user's moving speed, the user's location information, the operation status of the terminal 1, and the like (step S102). The acquisition unit 102 acquires the user's current location via a map application or GPS or the like. As shown in FIG. 5A and FIG. 5B, the display unit 107 may display the map application on a part of the display device 15 to show the user's current location. The acquisition unit 102 may acquire acceleration, the user's biological information, and the like from the wearable terminal or the like.

[0048] Based on the moving speed of the user, the acceleration of the user, the biometric information of the user, and the like, the estimation unit 105 determines whether the user is in the vehicle 2 (step S201). For example, if the moving speed of the user is higher than the average walking speed and the moving speed of the user continues for a predetermined time, the estimation unit 105 may estimate that the user is in the vehicle 2. Based on the acceleration of the user or the biometric information of the user, the estimation unit 105 may estimate the gait of the user, and based on the gait, the estimation unit 105 may estimate whether the user is in the vehicle 2. The estimation unit 105 may compare the acceleration of the user with a general acceleration of a bicycle or the like, and estimate whether the user is in the vehicle 2. When the vehicle 2 is equipped with a drive recorder or the like, the acquisition unit 102 may acquire an image or video from the drive recorder, and the estimation unit 105 may estimate whether the user is in the vehicle 2 based on the image or video. For example, the estimation unit 105 may estimate the distance between the user and a road sign in the image or video, and estimate the moving speed of the user based on the change in the distance over time.

[0049] When the estimation unit 105 determines that the user is not riding in the vehicle 2 based on the moving speed of the user, the acceleration of the user, the biometric information of the user, and the like (NO in step S202), the estimation unit 105 assumes that the user is walking without riding in the vehicle 2, and the notification unit 106 notifies the user of notification information for pedestrians (step S203). For example, in FIG. 5A, when the user is traveling on the bicycle path 62L without riding in the vehicle 2, the notification unit 106 notifies the user to travel on the sidewalk 63L. The notification unit 106 may notify the user to travel on the bicycle path 62L by riding in the vehicle 2. For example, when the estimation unit 105 estimates that the current position of the user in the map application is the bicycle path 62L instead of the sidewalk 63L, the acquisition unit 102 may acquire an image or video, and the estimation unit 105 may more accurately estimate the lane in which the user is traveling based on the display position of the lane marking 513L, and the like. If the vehicle 2 is equipped with a drive recorder or the like, the acquisition unit 102 may acquire an image or video from the drive recorder, and the notification unit 106 may notify the user of the traffic information based on the display of the traffic light from the image or video. When the notification unit 106 notifies the user of the notification information, the process returns to step S201.

[0050] When the estimation unit 105 determines that the user is riding on the vehicle 2 (YES in step S202), the notification unit 106 notifies the user of bicycle-oriented notification information (step S204). The estimation unit 105 estimates whether the user is violating road traffic rules or the like based on the user's moving speed, the user's biometric information, the map application, and the like (step S205, first estimation). For example, when the estimation unit 105 estimates that the user's current position is on the right side of the road 6 in the map application and the user's traveling direction is the opposite direction to the direction specified in the road traffic rules or the like, the notification unit 106 notifies the user that the user is violating the road traffic rules or the like. The user's traveling direction may be estimated based on the user's acceleration or the history of the user's position information in the map application. Furthermore, the position of the road on which the user is traveling may be estimated based on the user's position information (coordinate information) and the road information (coordinate information) of the map application. The estimation unit 105 can determine whether the user is located on the right side of the road (a range determined as a violation) by, for example, comparing the coordinate information indicating the range of the road information with the coordinate information indicating the user's current position. The estimation unit 105 can also read the travel direction determined in each range of the coordinate information indicating a predetermined range of the road information (pedestrian lanes, bicycle lanes, vehicle lanes) from a storage unit or the like, and compare the travel direction with the user's travel direction, and can determine that the user's travel direction is the opposite direction (violating the rules) when the directions do not match. In addition, if the user's moving speed is higher than a predetermined moving speed, the notification unit 106 may prompt the user to reduce the moving speed. For example, if the user's moving speed is 40 km / h or more, the estimation unit 105 may estimate that the user is traveling at a dangerous speed. If the user's pulse rate is higher than a predetermined pulse rate, the notification unit 106 may prompt the user to get off the vehicle 2 and move to a safe place.

[0051] If the user does not violate the road traffic rules, etc. (NO in step S206), the display unit 107 does not display the notification information on the display device 15, as shown in FIG. 5C. For example, the estimation unit 105 may determine whether or not a lane marking 513L is displayed in the lane in which the user is traveling in the user's traveling direction, based on the position information of the map application and the image or video information. The acquisition unit 102 may periodically acquire images or videos, and the estimation unit 105 may use the images or videos to determine the lane in which the user is traveling. On the other hand, if the user violates the road traffic rules, etc. (YES in step S206), the estimation unit 105 determines that the user may be violating the road traffic rules, etc.

[0052] The acquisition unit 102 acquires images and sounds (step S301), and the image processing unit 103 and the sound processing unit 104 process information about the user's surroundings. The estimation unit 105 estimates whether the user is violating road traffic rules, etc., based on information such as images and sounds in addition to the user's moving speed, the user's biometric information, and a map application (step S302, second estimation). If the estimation unit 105 determines that the user is unlikely to violate road traffic rules, etc., or if the estimation unit 105 determines that the user is violating road traffic rules, etc. but the violation is minor (NO in step S303), the process returns to step S201. Note that if the user's violation is minor, the display unit 107 may display notification information on the display device 15 to prompt the user to travel on a more appropriate road. For example, if there is an obstacle on the bicycle path 62L and the user temporarily travels on the sidewalk 63L to avoid the obstacle, the display unit 107 may not display notification information on the display device 15. The time during which the user continued to violate the road traffic rules, etc., the location information where the user violated the road traffic rules, etc., and the road traffic rules violated by the user may be adopted as evaluation amounts, and when the evaluation amount exceeds a predetermined value, the notification unit 106 may notify the user of the notification information, and the display unit 107 may display the notification information on the display device 15. The estimation unit 105 may cumulatively add up the evaluation amounts, and when the evaluation amount exceeds a predetermined value, the notification unit 106 may notify the user of the notification information, and the display unit 107 may display the notification information on the display device 15.

[0053] On the other hand, when the estimation unit 105 determines that the user may violate road traffic rules or the like, or when the estimation unit 105 determines that the user has violated road traffic rules or the like (YES in step S303), the display unit 107 displays notification information on the display device 15, and the notification unit 106 notifies the user of information about the violation or the like (step S304). For example, in the map application of FIG. 5B, when the estimation unit 105 compares the coordinates of the user with the coordinates of the railroad tracks 52 and determines that the user is approaching the railroad tracks 52, the acquisition unit 102 may acquire a voice and check whether or not a warning sound of a railroad crossing is being sounded. When the acquisition unit 102 acquires a warning sound of a railroad crossing, the estimation unit 105 may estimate that the user may violate road traffic rules or the like, and the notification unit 106 may notify the user to stop in front of the railroad crossing. As shown in FIG. 5D, when the estimation unit 105 estimates that the user is walking on the sidewalk 63L based on an image or video, the notification unit 106 notifies the user that the user is walking on the sidewalk (sidewalk 63L) and that the user should walk on a bicycle lane (bicycle path 62L). As shown in FIG. 5E, when the estimation unit 105 estimates that the user is walking on the bicycle path 62R based on the fact that a lane sign 513R acquired from an image or video is displayed in the opposite direction to the user's traveling direction, the notification unit 106 notifies the user that the user is violating a road traffic rule and the action the user should take to resolve the violation. When the user is committing a violation that is likely to cause an accident, the notification unit 106 may issue a warning sound or the like to notify the user that the user is violating a road traffic rule. As shown in FIG. 5F, when it is determined based on an image, video, or audio that the user may violate a road traffic rule, that is, when the user is attempting to cross the railroad tracks 52 despite the barrier 522L restricting the user from crossing the railroad tracks 52 at the railroad crossing, the notification unit 106 notifies the user of an appropriate action. Note that the functional unit 100 may be configured so that the acquisition unit 102 always detects an ambulance siren sound, a fire engine siren sound, a police car siren sound, a railroad crossing alarm sound, and the like, regardless of whether or not the user may violate a road traffic rule, and the like. In such a configuration, when the acquisition unit 102 acquires a siren sound, an alarm sound, and the like, the notification unit 106 notifies the user of the notification information.

[0054] If the user follows the road traffic rules as a result of the notification in step S304 (YES in step S305), notification unit 106 notifies that the user is no longer violating the road traffic rules (step S306). On the other hand, if the user continues to violate the road traffic rules (NO in step S305), notification unit 106 notifies not only the user but also the administrator that the user is violating the road traffic rules, etc. The notification information notified to the administrator may be, for example, the location information and date and time when the user violated the road traffic rules, the road traffic rules violated by the user, and an image or video of the user at the time of the violation. Then, the process returns to step S201.

[0055] If the user or administrator has ended the monitoring application (YES in step S307), the processing of the monitoring application ends. On the other hand, if the user or administrator has not ended the monitoring application, the processing returns to step S201. Note that the notification unit 106 may not only notify the user about the road traffic rules, but also notify the user to drive slowly when passing near the pedestrian 71, and may notify the user to be careful of passing vehicles 72 when turning right on the road. For example, when the monitoring application receives a beacon signal or the like emitted by a wearable terminal carried by a pedestrian, the monitoring application may assume that the distance between the pedestrian and the user is such that the beacon signal can be received, and notify the user to drive slowly.

[0056] In the present embodiment, the configuration in which the monitoring application notifies the user whether or not the user violates the road traffic rules has been described, but the notification information notified by the monitoring application is not limited to this. For example, when the user encounters an accident, the monitoring application may obtain images before and after the accident from a recording medium provided in a drive recorder or the like, and store information on whether or not the user violated the road traffic rules and the images. Based on the information and the images, foreign tourists who are not familiar with the road traffic rules can use the information when responding to the accident.

[0057] When applying road traffic rules to the user's passage, the user or the administrator may set the strictness of the rule application by the settings of the monitoring application. For example, in the road 6 having the bicycle path 62, bicycles may be tacitly allowed to pass on the sidewalk 63 under actual road traffic conditions. Therefore, when a bicycle passes on the sidewalk 63, the monitoring application may not notify that the user has violated the road traffic rules. A distinction may be made between road traffic rules that are only notified to the user by referring to the actual road traffic conditions and road traffic rules that are notified to the user and the administrator.

[0058] A facility 5 such as an alarm 521 may transmit notification information to a monitoring application via a beacon signal or the like, and the monitoring application may notify the user of the notification information. For example, the alarm 521 may notify the monitoring application via a beacon signal that a railroad vehicle is approaching, and the monitoring application may notify the user to stop until the railroad vehicle passes. A road sign may transmit road traffic rules to a monitoring application, and the monitoring application may notify the user of the notification information. For example, the alarm 521 may transmit a beacon signal before and after a railroad vehicle passes a railroad crossing, and when the monitoring application no longer receives the beacon signal, the user may be notified that the railroad crossing may be passed. A traffic light may transmit a beacon signal including information such as green (may proceed), yellow (must stop), and red (must stop) to a predetermined traffic lane, and the monitoring application may notify the user of the traffic information based on the information included in the beacon signal.

[0059] The terminal 1 and the server 3 or 4 may communicate with each other, and based on the user's current location information and display information of traffic lights, etc., it may be determined whether the user has violated road traffic rules by ignoring traffic lights, etc. The monitoring application transmits the user's movement record, etc. in the map application to the server 3 or 4, and the server 3 or 4 refers to the movement record and the display information of traffic lights. This enables the server 3 or 4 to detect the user's violation of road traffic rules.

[0060] By the terminal 1 notifying the user of notification information according to the road conditions, the user can be prevented from violating the road traffic rules and from encountering a traffic accident, etc. This enables the user to travel the road safely. In addition to preventing the user from unintentionally violating the road traffic rules, it is also possible to notify the administrator of the violation when the user maliciously violates the road traffic rules.

[0061] [Other embodiments] 7 is a block diagram showing a monitoring device according to an embodiment of the present disclosure. The monitoring device 1000 includes an estimation unit 1001 and a notification unit 1002. The estimation unit 1001 estimates whether or not a user has violated a road traffic rule or the like based on an image, a video, an audio signal, a sensor signal, position information, acceleration, biometric information of the user, and the like, and estimates notification information to be notified to the user. The notification unit 1002 notifies the user and an administrator of the notification information.

[0062] Also included in the scope of each embodiment is a processing method for recording a program that operates the configuration of each embodiment to realize the functions of the above-mentioned embodiments on a recording medium, reading the program recorded on the recording medium as code, and executing the program on a computer. That is, a computer-readable recording medium is also included in the scope of each embodiment. Also included in each embodiment is not only a recording medium on which the above-mentioned computer program is recorded, but also the computer program itself.

[0063] Examples of the recording medium that can be used include a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM (Compact Disc-Read Only Memory), a magnetic tape, a non-volatile memory card, and a ROM. In addition, the scope of each embodiment includes not only those that execute processing by a single program recorded on the recording medium, but also those that execute processing by operating on an OS (Operating System) in cooperation with other software and functions of an extension board.

[0064] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and detailed description of the present disclosure within the scope of the present disclosure. In addition, each embodiment can be appropriately combined with other embodiments.

[0065] According to this embodiment, it is possible to notify a user traveling on a road of traffic rules or traffic violations.

[0066] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0067] (Appendix 1) an estimation unit that estimates road traffic rules for a user and the surroundings of the user on a road based on information about the user and the surroundings of the user; A monitoring device comprising: a notification unit that notifies the user of behavior information in accordance with the road traffic rules.

[0068] (Appendix 2) An acquisition unit for acquiring the information, The monitoring device described in Appendix 1, wherein the information is at least one of an image of the surroundings, sounds from the surroundings, location information from a satellite positioning system, location information of the user on the road, the direction of movement of the user, the movement speed of the user, the acceleration of the user, biometric information of the user, the operating status of a mobile terminal owned by the user, whether or not the user is in a vehicle, and the type of the vehicle.

[0069] (Appendix 3) 3. The monitoring device according to claim 1 or 2, wherein if the user violates the road traffic rules, the monitoring device notifies the user of the details of the violation.

[0070] (Appendix 4) 4. The monitoring device according to claim 1, wherein, if the user violates the road traffic rules, the monitoring device notifies a predetermined third party of the details of the violation.

[0071] (Appendix 5) The monitoring device according to any one of appendix 2 to 4, wherein, when the image includes a predetermined road traffic sign, the behavior information corresponding to the predetermined road traffic sign is notified to the user.

[0072] (Appendix 6) The monitoring device according to any one of appendix 2 to 5, wherein, when the sound includes a predetermined warning sound, the behavior information corresponding to the predetermined warning sound is notified to the user.

[0073] (Appendix 7) A monitoring device according to any one of appendices 1 to 6, further acquiring the road traffic rules and the behavioral information from facilities installed on the road, and notifying the user of the road traffic rules and the behavioral information.

[0074] (Appendix 8) An imaging unit that captures an image of the user's surroundings; A sound collection unit for acquiring sounds from the surroundings; an acquisition unit that acquires information of the user on a road, 2. The monitoring device of claim 1, wherein the estimation unit estimates road traffic rules for the surroundings based on the image and the sound.

[0075] (Appendix 9) an estimation step of estimating road traffic rules for a user and the user's surroundings on a road based on information about the user and the surroundings; a notification step of notifying the user of behavior information in accordance with the road traffic rules.

[0076] (Appendix 10) The method further comprises the step of acquiring the information, The monitoring method described in Appendix 9, wherein the information is at least one of an image of the surroundings, sounds from the surroundings, location information from a satellite positioning system, location information of the user on the road, the direction of movement of the user, the movement speed of the user, the acceleration of the user, biometric information of the user, the operating status of a mobile terminal owned by the user, whether or not the user is in a vehicle, and the type of the vehicle.

[0077] (Appendix 11) The monitoring method according to claim 9 or 10, wherein, if the user violates the road traffic rules, the user is notified of the details of the violation.

[0078] (Appendix 12) The monitoring method according to any one of appendices 9 to 11, wherein, if the user violates the road traffic rules, the content of the violation is notified to a specified third party.

[0079] (Appendix 13) The monitoring method according to any one of appendices 10 to 12, wherein, when the image includes a predetermined road traffic sign, the user is notified of the behavior information corresponding to the predetermined road traffic sign.

[0080] (Appendix 14) The monitoring method according to any one of appendices 10 to 13, wherein, when the sound includes a predetermined warning sound, the user is notified of the behavioral information corresponding to the predetermined warning sound.

[0081] (Appendix 15) A monitoring method according to any one of appendices 9 to 14, further acquiring the road traffic rules and the behavioral information from facilities installed on the road, and notifying the user of the road traffic rules and the behavioral information.

[0082] (Appendix 16) An imaging step of acquiring an image of the user's surroundings; A sound collection step of acquiring sounds from the surroundings; and acquiring information of the user on a road, 10. The surveillance method of claim 9, wherein the estimation unit estimates road traffic rules for the surroundings based on the image and the sound.

[0083] (Appendix 17) an estimation step of estimating road traffic rules for a user and the user's surroundings on a road based on information about the user and the surroundings; and a notification step of notifying the user of behavior information conforming to the road traffic rules.

[0084] (Appendix 18) The method further comprises the step of acquiring the information, The monitoring program described in Appendix 17, wherein the information is at least one of an image of the surroundings, sounds from the surroundings, location information from a satellite positioning system, location information of the user on the road, the direction of movement of the user, the movement speed of the user, the acceleration of the user, biometric information of the user, the operating status of a mobile terminal owned by the user, whether or not the user is in a vehicle, and the type of the vehicle.

[0085] (Appendix 19) 19. The monitoring program according to claim 17 or 18, wherein if the user violates the road traffic rules, the monitoring program notifies the user of the details of the violation.

[0086] (Appendix 20) 20. The monitoring program according to any one of appendices 17 to 19, wherein, if the user violates the road traffic rules, the content of the violation is notified to a specified third party.

[0087] (Appendix 21) 21. The monitoring program according to any one of appendices 18 to 20, wherein, if the image includes a predetermined road traffic sign, the user is notified of the behavior information corresponding to the predetermined road traffic sign.

[0088] (Appendix 22) 22. The monitoring program according to any one of appendices 18 to 21, wherein, if the sound includes a predetermined warning sound, the user is notified of the behavior information corresponding to the predetermined warning sound.

[0089] (Appendix 23) A monitoring program according to any one of appendices 17 to 22, further acquiring the road traffic rules and the behavioral information from facilities installed on the road, and notifying the user of the road traffic rules and the behavioral information.

[0090] (Appendix 24) An imaging step of acquiring an image of the user's surroundings; A sound collection step of acquiring sounds from the surroundings; and acquiring information of the user on a road, 18. The surveillance program of claim 17, wherein the estimation unit estimates a road traffic rule for the surroundings based on the image and the sound. [Explanation of symbols]

[0091] 2: Vehicle 1: Terminal 11: CPU 12:ROM 13: RAM 14:Storage device 15:Display device 151:Display screen 152: Current location icon 16: Touch panel 17: Imaging device 18: Input / Output IF 19: Communication Interface 100: Functional section 101: Control unit 102: Acquisition Department 103: Image processing unit 104: Audio processing unit 105: Estimation part 106: Notification Department 107: Display section 108: Database 3: Server 4: Server 5: Facilities 51:Road markings 511: Chuo Line 512: White dashed line 513: Lane markings 514: White line 52:Railway 521: Alarm 522: Barrier 523: Alarm sound 6: Road 61: Roadway 62: Bicycle path 63: Sidewalk 71: Pedestrian 72: Passing vehicles 9: Network 1000: Monitoring device 1001: Estimation section 1002: Notification department

Claims

1. an estimation unit that estimates road traffic rules for the surroundings based on information about a user and the surroundings of the user on a road; A monitoring device comprising: a notification unit that notifies the user of behavior information in accordance with the road traffic rules.

2. An acquisition unit for acquiring the information, 2. The monitoring device of claim 1, wherein the information is at least one of an image of the surroundings, sounds from the surroundings, location information from a satellite positioning system, location information of the user on the road, the direction of movement of the user, the movement speed of the user, the acceleration of the user, biometric information of the user, the operating status of a mobile terminal owned by the user, whether or not the user is in a vehicle, and the type of the vehicle.

3. The monitoring device according to claim 1 , wherein, when the user violates the road traffic rules, the monitoring device notifies the user of the details of the violation.

4. The monitoring device according to claim 1 , wherein, when the user violates the road traffic rules, the monitoring device notifies a predetermined third party of the details of the violation.

5. The monitoring device according to claim 2 , wherein, when the image includes a predetermined road traffic sign, the user is notified of the behavior information corresponding to the predetermined road traffic sign.

6. The monitoring device according to claim 2 , wherein, when the sound includes a predetermined warning sound, the behavior information corresponding to the predetermined warning sound is notified to the user.

7. The monitoring device according to claim 1 , further acquiring the road traffic rules and the behavior information from a facility installed on the road, and notifying the user of the road traffic rules and the behavior information.

8. An imaging unit that captures an image of the user's surroundings; A sound collection unit for acquiring sounds from the surroundings; an acquisition unit that acquires information of the user on a road, The monitoring device according to claim 1 , wherein the estimation unit estimates a road traffic rule for the surroundings based on the image and the sound.

9. an estimation step of estimating road traffic rules for a user and the user's surroundings on a road based on information about the user and the surroundings; and a notification step of notifying the user of behavior information conforming to the road traffic rules.

10. an estimation step of estimating road traffic rules for a user and the user's surroundings on a road based on information about the user and the surroundings; and a notification step of notifying the user of behavior information conforming to the road traffic rules.

Images