Reminder systems and reminder programs

Abstract

To appropriately alert a road user in accordance with the situation of each vehicle and / or of an actual site.SOLUTION: An alerting system includes: an imaging device 2; an information generation device 3 that determines a content of alert information to be provided to a road user on the basis of image data generated by the imaging device 2; and a display device 4 that displays the content of alert information determined by the information generation device 3.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an attention calling system and an attention calling program, and more particularly to a technique for calling the attention of road users by means of, for example, an electronic signboard. [Background technology]

[0002] When lane restrictions are implemented on an in-use road for road construction, construction signs are placed at the start of the traffic restrictions stating things like "Lane restrictions apply beyond this point" or "Construction in progress, watch your speed" to alert drivers of passing vehicles.

[0003] Additionally, some of the construction signs used throughout the city (called "electronic signboards") use LEDs to project images of guards waving flags to attract the attention of passing drivers.

[0004] In addition, a security sign that can be attached to equipment used at construction sites is known to have at least a foldable main plate with a display section on the front side consisting of letters, drawings, photographs or a combination of these (see Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Utility Model Registration No. 3211690 Summary of the Invention [Problem to be solved by the invention]

[0006] However, conventional signs (including electric signs) only display information that is common to all vehicles passing through the section, such as "Construction ahead," or only display fixed information that has been determined in advance, and therefore do not necessarily display information effectively according to the conditions of each vehicle or the actual conditions at the site (for example, the speed of each vehicle, the weather at the time, moment-to-moment traffic congestion, obstacles, etc.). In addition, when the restricted section is long, there is a problem that drivers' attention to their surroundings tends to decrease if the displayed information is fixed and constant.

[0007] Therefore, an object of the present invention is to provide an attention calling system and an attention calling program that are capable of issuing appropriate attention according to the conditions of each vehicle and the actual conditions at the site. [Means for solving the problem]

[0008] In order to solve the above problems, the present invention provides an attention drawing system including an imaging device, an information generating device that determines the content of information to be provided to road users based on image data generated by the imaging device, and a display device that displays the content of the information determined by the information generating device. death , When a plurality of the display devices are installed and a plurality of vehicles are lined up, the contents of the information on each of the display devices are switched in accordance with the passage of the vehicles so that the contents of the information selected for each of the vehicles are displayed on each of the display devices. It is characterized by:

[0010] In the attention calling system according to the present invention, the image data may include at least one of road conditions, weather conditions, and vehicle conditions.

[0011] In the attention calling system according to the present invention, the display device may display at least one of characters, pictures, and moving content.

[0012] In the warning system of the present invention, the information generation device may use a trained model that has been machine-learned based on past performance data to output the content of the information when the image data is input.

[0013] The attention calling program according to the present invention causes a computer to function as a means for receiving an input of image data generated by an imaging device, a means for determining the content of information to be provided to road users based on the image data, and a means for outputting the content of the information to a display device. and when a plurality of vehicles are connected in a line, the means for determining the content of the information determines the content of the information to be output to each of the plurality of display devices for each of the display devices and for each of the vehicles. It is characterized by:

[0015] In the attention calling program according to the present invention, the image data may include at least one of road conditions, weather conditions, and vehicle conditions.

[0016] In the warning program of the present invention, the means for determining the content of the information may use a trained model that has been machine-learned based on past performance data so as to output the content of the information when the image data is input. Effect of the Invention

[0017] According to the warning system and warning program of the present invention, the content of the information to be provided to road users is determined based on image data generated by an imaging device, so that appropriate warning content can be selected / determined according to the condition of each vehicle and the actual situation at the site and displayed on a display device to provide an appropriate warning.

[0018] According to the warning system and warning program of the present invention, if multiple display devices are installed so that the content of information can be switched according to the passage of vehicles, multiple display devices can be linked at construction sites, for example, to provide appropriate warnings to road users, thereby ensuring safety.

[0019] According to the warning system and warning program of the present invention, when the image data includes at least one of road conditions, weather conditions, and vehicle conditions, it is possible to display appropriate warning content on the display device based on appropriate image data in accordance with the condition of each vehicle and the actual situation on site, thereby providing appropriate warnings.

[0020] According to the warning system of the present invention, when the display device is configured to display at least one of text, pictures, and moving content, it is possible to display and convey the warning content to road users in an easily understandable manner.

[0021] According to the warning system and warning program of the present invention, when a trained model trained by machine learning based on past performance data is used, the warning content is selected / determined and output using the trained model trained by machine learning, making it possible to select / determine more accurate warning content. [Brief description of the drawings]

[0022] [Figure 1] 1 is a block diagram showing a schematic configuration of an attention calling system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a diagram showing an example of deployment of the attention calling system of FIG. 1 at a site, and is a schematic configuration diagram showing an example in which one imaging device is installed. [Diagram 3] FIG. 2 is a diagram showing an example of deployment of the attention calling system of FIG. 1 in a field, and is a schematic configuration diagram showing an example in which a plurality of imaging devices are installed. [Figure 4] 2 is a block diagram showing a schematic configuration of an example of a trained model (neural network) of the attention calling system of FIG. 1. FIG. [Diagram 5] 2 is a diagram showing an example of a display on a display device of the attention calling system of FIG. 1. [Figure 6] 2 is a diagram showing an example of a display when a plurality of display devices of the attention calling system of FIG. 1 are installed. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, the present invention will be described based on the illustrated embodiment.

[0024] FIG. 1 is a block diagram showing a schematic configuration of an attention calling system 1 according to an embodiment of the present invention.

[0025] The attention calling system 1 is a system for calling the attention of road users such as vehicle drivers by using image data, and has an imaging device 2, an information generating device 3, and a display device 4 as main components.

[0026] Information provided to alert road users such as vehicle drivers is called "attention alert information." An area where image data required to generate attention alert information is acquired (specifically, an area including, for example, vehicles traveling on the road, the road surface, features around the road, and the space around the road) is called an "image acquisition area." An area where the display device 4 is installed and where attention alert information is provided is called an "information provision area." Note that the image acquisition area and the information provision area may be set to (approximately) coincide / overlap, or may be set to be different from each other. For example, the image acquisition area may be set to include the information provision area and to include areas before, after, and around the information provision area in the vehicle's traveling direction.

[0027] The warning system 1 in the embodiment includes an imaging device 2, an information generating device 3 that determines the content of warning information to be provided to road users based on image data generated by the imaging device 2, and a display device 4 that displays the content of the warning information determined by the information generating device 3.

[0028] In addition, the warning program 321 in the embodiment causes the computer to function as a means for receiving input of image data generated by the imaging device 2, a means for determining the content of warning information to be provided to road users based on the image data, and a means for outputting the content of the warning information to the display device 4.

[0029] The imaging device 2 and the information generating device 3 are connected via a communication network 9 so as to be able to communicate with each other / transmit signals, and the information generating device 3 and the display device 4 are connected via the communication network 9 so as to be able to communicate with each other / transmit signals.

[0030] The communication network 9 is a variety of wireless or wired communication network / signal transmission lines capable of sending and receiving signals, including various communication forms / signal transmission forms such as LAN (Local Area Network), VPN (Virtual Private Network), WAN (Wide Area Network), the Internet, an intranet, a mobile communication network, a dedicated line, a bus, and a cable.

[0031] At least one of the imaging device 2 and the information generating device 3 may be attached to the display device 4 or may be configured integrally with the display device 4.

[0032] The imaging device 2 is a mechanism for generating image data of an area including an image acquisition area and transmitting the image data to the information generating device 3, and mainly includes an imaging section 21 and a communication section 22.

[0033] The imaging unit 21 has a function of repeatedly capturing an image of an area including an image acquisition area at a predetermined time interval (called an "imaging interval"), generating and outputting image data, and is configured by, for example, a digital camera. The imaging unit 21 may capture a still image, generate and output image data of the still image, or may capture a moving image, generate and output image data of the moving image.

[0034] The imaging interval is not limited to a specific time length, but may be set to any time length within a range of, for example, 10 to 60 seconds. When the imaging unit 21 captures a moving image, it may capture a moving image (in other words, still images that are continuous in time series) with a capture time within a range of, for example, 1 to 60 seconds.

[0035] The communication unit 22 is configured by a communication interface / communication module having a function of connecting the imaging device 2 to the communication network 9 so as to be able to communicate / transmit signals, and transmits and receives signals to and from the information generating device 3.

[0036] The imaging device 2 is installed on a road, roadside, or in a position near the road, and transmits still or moving image data captured by the imaging unit 21 to the information generating device 3 via the communication unit 22 and the communication network 9.

[0037] The imaging device 2 is installed after its location has been selected, taking into consideration the acquisition (specifically, the taking of photographs) of, for example, the following information, which is required for creating the trained model 322 described below. a) Road conditions Specifically, whether there is road congestion or traffic jams, whether there are obstacles, whether the road is damaged, whether the road is flooded, whether the road surface is frozen, etc. b) Weather conditions Specifically, whether there will be rain or snow, the amount of sunshine, whether there will be fog, etc. c) Vehicle status Specifically, the vehicle's speed, vehicle spacing (distance between vehicles), whether the lights are on, whether there are motorcycles, bicycles, pedestrians, etc. approaching the vehicle, and whether the driver is drowsy or distracted.

[0038] The imaging device 2 may be configured to acquire information relating to all of the above road conditions, weather conditions, and vehicle conditions, or may be configured to acquire information relating to one or two of the above road conditions, weather conditions, and vehicle conditions.

[0039] The imaging device 2 may be installed in a single unit if it is possible to image the entire area including the image acquisition area (see Figure 2), or multiple units may be installed when required to image the entire area including the image acquisition area or when required to image information required for creating the trained model 322 described later (see Figure 3).

[0040] The information generating device 3 is a mechanism for generating (in other words, selecting or deciding) warning information using image data generated by the imaging device 2 and transmitting it to the display device 4, and mainly has a control unit 31, a memory unit 32, a communication unit 33, an information decision unit 34, and a learning unit 35.

[0041] The information generating device 3 may be configured by an information processing device designed specifically for the device, or may be configured by using a general-purpose computer / server.

[0042] The control unit 31 has the function of controlling the operation of each part that constitutes the information generating device 3, and is configured as a mechanism having, for example, a central processing unit (CPU: abbreviation for Central Processing Unit) that performs calculations related to the selection / determination of warning information.

[0043] The control unit 31 coordinates and controls the start, content, and end of processing of each part constituting the information generating device 3 in accordance with the attention program 321 by the central processing unit executing the attention program 321 for controlling the operation of the information generating device 3, which is stored in the memory unit 32.

[0044] The memory unit 32 has the function of serving as a working area for temporarily storing data and other information generated when the central processing unit performs arithmetic processing related to the selection / decision of warning information, and as a memory area for storing and storing various information, programs, and data, and is configured as a mechanism having at least one of, for example, a ROM (short for Read Only Memory), which is a readable memory device, a RAM (short for Random Access Memory), which is a readable and writable memory device, a hard disk drive (HDD: short for Hard Disk Drive), and a solid state drive (SSD: short for Solid State Drive).

[0045] Specifically, the memory unit 32 stores an alert program 321, a trained model 322, and a performance database 323.

[0046] The communication unit 33 is configured by a communication interface / communication module having a function of connecting the information generating device 3 to the communication network 9 so as to be able to communicate / transmit signals, and transmits / receives signals between the imaging device 2 and the display device 4.

[0047] Specifically, the communication unit 33 receives image data generated by the imaging device 2 via the communication network 9, and also transmits the contents of the warning information selected / determined by the information determination unit 34 to the display device 4 via the communication network 9.

[0048] The information determination unit 34 selects / determines the content of the attention-calling information based on the image data generated by the imaging device 2 at each imaging interval.

[0049] The information determination unit 34 selects / determines the content of the warning information using a trained model 322 that has been machine-learned based on past performance data so as to output the content of the warning information when image data generated by the imaging device 2 is input. The trained model 322 is created by the learning unit 35 and stored in the storage unit 32.

[0050] The learning unit 35 uses past performance data recorded and accumulated in the performance database 323 to create a trained model 322 by a known machine learning algorithm such as a neural network.

[0051] The performance database 323 is a database in which performance data including a combination of image data and the contents of warning information corresponding to the image data is recorded and accumulated as learning data. Of the combinations, the image data is used as input data, and the contents of the warning information are used as teacher data. The performance database 323 is stored in the storage unit 32.

[0052] The performance database 323 is a collection of performance data that includes combinations of image data at multiple points in time in the image acquisition area (in other words, at each of various conditions in the image acquisition area, such as road conditions, weather conditions, and vehicle conditions) and the content of warning information corresponding to the image data.

[0053] The learning data is prepared taking into consideration, for example, the following correspondence relationship between the image data as input data and the contents of the warning information as teaching data. a) When "road congestion has occurred" is indicated in the image data, "watch your speed" and "watch out for vehicle distance" are selected as the contents of the attention alert information. a) When "traffic jam occurs" in the image data, "traffic jam / stop" is selected as the content of the attention information. c) When "an obstacle is present" is displayed in the image data, "Watch out for obstacles" is selected as the content of the attention-drawing information. d) When "road damage exists" is indicated in the image data, "road damage" and "watch your speed" are selected as the contents of the warning information. e) When "road flooded" is indicated in the image data, "beware of flooding" is selected as the content of the warning information. f) When the image data indicates that "there is frozen road surface," "Watch out for frozen roads" and "Watch out for slipping" are selected as the contents of the attention alert information. g) When "rainfall or snowfall is present" in the image data, "watch out for slippage" and "watch out for distance between vehicles" are selected as the contents of the attention alert information. H) When the image data shows that the level of sunlight is low (for example, at dusk or at night), the content of the attention-calling information is selected as "lights on (for vehicles without lights)." Whether or not the vehicle has lights off is determined based on, for example, the image data generated by the imaging device 2. i) When "fog has occurred" in the image data, "poor visibility" and "watch your speed" are selected as the contents of the attention-calling information. J) When the image data indicates that the vehicle speed exceeds the speed limit, the warning information selects "Slow down." The vehicle speed is measured, for example, based on the image data generated by the imaging device 2. The speed limit of each road is stored in advance, for example, in the storage unit 32. C) When the image data indicates that "the vehicle interval is narrow," "be careful of the vehicle interval" is selected as the content of the attention-calling information. C) When the image data indicates that "there are motorcycles, bicycles, pedestrians, etc. approaching the vehicle," "beware of surroundings" is selected as the content of the attention alert information. m) When the image data indicates that "the driver is dozing off" or "the driver is distracted," "concentrate on driving" is selected as the content of the attention-calling information.

[0054] When image data is input, the learning unit 35 outputs an output value corresponding to the content of the attention-calling information, and trains the trained model 322 by comparing the output value with the teacher data.

[0055] The learning unit 35 creates a model by, for example, machine learning or deep learning using a neural network, and as a result, the trained model 322 is configured by a neural network.

[0056] As shown in Fig. 4, the neural network N has an input layer X, an intermediate layer Y (also called a "hidden layer"), and an output layer Z. The input layer X has a plurality of nodes x1, x2, x3,..., the intermediate layer Y has a plurality of nodes y1, y2, y3,..., and the output layer Z has a plurality of nodes z1, z2, z3,... (or one node). Note that although the example shown in Fig. 4 has one intermediate layer Y, the neural network N may be configured to have a plurality of intermediate layers.

[0057] Each connection between nodes in the input layer X and the hidden layer Y has a weight, and each connection between nodes in the hidden layer Y and the output layer Z has a weight.

[0058] The learning unit 35, for example, uses image data as an input layer X and the contents of the attention information (specifically, output values ​​corresponding to the contents of the attention information) as an output layer Z, and uses the performance data recorded and accumulated in the performance database 323 as learning data to learn various parameters (including the weights of each connection between nodes) related to the intermediate layer Y to create a neural network N. In other words, the learning unit 35 creates a trained model 322 by learning various parameters related to the intermediate layer Y so that appropriate information contents corresponding to the situation in the image data are selected / determined from the image data.

[0059] In addition, when multiple imaging devices 2 are installed, when selecting / determining the content of the warning information to be transmitted to a display device 4, image data generated by one imaging device 2 may be used as input data, or image data generated by multiple imaging devices 2 may be used as input data.

[0060] In addition, in cases where it is necessary to appropriately select / determine the content of appropriate information corresponding to the situation in the image data, image data focusing on a specific part of the image data (specifically, a still image or a video) or features related to the specific part may be used as input data, or processed image data in which processing related to hue, saturation, brightness, and lightness has been applied to the image data may be used as input data.

[0061] The information determination unit 34 uses image data generated by the imaging device 2 and input via the communication network 9 and the communication unit 33, as well as the trained model 322 stored in the memory unit 32 to select / determine the content of the warning information.

[0062] Then, the information determination section 34 outputs / transmits the contents of the attention drawing information selected / determined as above to the display device 4 via the communication section 33 and the communication network 9 for each imaging interval.

[0063] The display device 4 is a mechanism for displaying the contents of the attention-calling information selected / determined by the information generating device 3, and mainly includes a communication unit 41 and a display unit .

[0064] The communication unit 41 is configured by a communication interface / communication module having a function of connecting the display device 4 to the communication network 9 so as to be able to communicate / transmit signals, and transmits / receives signals to / from the information generating device 3.

[0065] Specifically, the communication unit 41 receives the contents of the attention-attracting information transmitted from the information determining unit 34 via the communication network 9, and transmits the contents to the display unit 42.

[0066] The display unit 42 has a function of displaying the contents of the attention-drawing information transmitted from the communication unit 41, and is configured as a mechanism having a panel on which light-emitting diodes are arranged vertically and horizontally and capable of displaying characters and pictures, for example.

[0067] The display device 4 is installed on the road, roadside, or near the road, and displays and provides attention-calling information to road users, such as drivers of vehicles traveling in the information provision area, thereby calling the attention of road users.

[0068] The display device 4 may be configured to display characters, or may be configured to display pictures, or may further be configured to display moving content using characters and pictures, or may even be configured to display a combination of characters, pictures, and moving content.

[0069] Specifically, the display device 4 may be configured to display text as shown in FIG. 5(A), or may be configured to display moving content using pictures as shown in FIGS. 5(B) and (C).

[0070] FIG. 5(B) is an example of "Watch your speed" (image: a flag waving up and down; the dotted-dash line in the figure represents the up and down movement of the flag) selected as the content of the warning information when "there is road congestion" in the image data, and FIG. 5(C) is an example of "concentrate on driving" (image: a siren turns on and rotates horizontally; the dotted-dash line in the figure represents the horizontal rotation movement of the siren) selected as the content of the warning information when "the driver is dozing off or distracted" in the image data.

[0071] The display device 4 may be installed in one information provision area, or multiple display devices 4 may be installed at a predetermined distance. When multiple display devices 4 are installed, the information determination unit 34 selects / determines the content of the attention-calling information to be output / transmitted to each of the multiple display devices 4 for each display device 4.

[0072] The display device 4 switches the display to display the content of the attention information every time the content of the attention information is transmitted from the information determining unit 34 (for example, at each imaging interval).

[0073] When multiple display devices 4 are installed, each display device 4 can be configured to display warning information for the nearest vehicle heading toward that display device 4, and when multiple vehicles are lined up, the contents of the warning information selected / decided for each vehicle can be displayed on the nearest display device 4 in front of each vehicle.

[0074] Specifically, for example, when a line of vehicles enters an information provision area and the image data indicates that "road congestion has occurred," the display device 4 at the front of the information provision area may display "Watch your speed" as the content of the attention alert for the leading vehicle (see FIG. 3), and then the next display device 4 in the information provision area may display "Watch your speed" as the content of the attention alert for the leading vehicle, and the display device 4 at the front of the information provision area may display "Watch your distance" as the content of the attention alert for the following vehicles (i.e., the display device 4 at the front may switch the display from "Watch your speed" to "Watch your distance"; see FIG. 6). That is, a plurality of display devices 4 may be linked to switch and display the content of the attention alert information in accordance with the passage of vehicles.

[0075] In addition, in a case where a plurality of display devices 4 are installed, if the content of the attention calling information displayed on the display device 4 on the front side in the traveling direction of the vehicle is not followed, the same content as the content of the attention calling information may be displayed on the next display device 4 on the traveling direction of the vehicle. In this case, a trained model obtained by machine learning may be configured to determine whether or not the content of the attention calling information displayed on the display device 4 on the front side has been followed.

[0076] As described above, according to the warning system 1 and the warning program 321, the content of the information to be provided to road users (i.e., the content of the warning information) is determined based on the image data generated by the imaging device 2, so that appropriate warning content (i.e., the content of the warning information) can be selected / determined according to the condition of each vehicle and the actual situation at the scene, and displayed on the display device 4 to provide an appropriate warning.

[0077] According to this attention calling system 1 and attention calling program 321, moreover, since a plurality of display devices 4 are installed and the contents of the attention calling information are switched according to the passage of vehicles, it is possible to link a plurality of display devices 4 at a construction site or the like to appropriately call the attention of road users and ensure safety. Also, in road construction with a long restricted area, workers may be deployed as auxiliary monitors, but according to this attention calling system 1, workers are not required, so it is possible to save labor, and further, since workers do not stand on the side of the road, it is possible to significantly reduce the risk of personal injury accidents.

[0078] According to this warning system 1 and warning program 321, the image data includes at least one of road conditions, weather conditions, and vehicle conditions, so that appropriate warnings can be displayed on the display device 4 according to the conditions of each vehicle and the actual conditions at the site based on appropriate image data, thereby making it possible to give appropriate warnings.

[0079] According to this warning system 1, the display device 4 is configured to display at least one of text, pictures, and moving content, making it possible to display and convey warning content to road users in an easily understandable manner.

[0080] According to this warning system 1 and warning program 321, a trained model 322 that has been machine-learned based on past performance data is used. Therefore, the warning content is selected / determined and output using the trained model 322 that has been machine-learned, making it possible to select / determine more accurate warning content.

[0081] Although the embodiment of the present invention has been described in detail above, the specific configuration is not limited to this embodiment, and even if there are design changes and the like within the scope of the gist of the present invention, they are included in the present invention.

[0082] For example, the correspondence relationships a) to c) between image data as input data and the contents of warning information as teacher data in the above embodiments are merely examples, and the contents of image data and the contents of warning information in this invention are not limited to the explanations / examples in the above embodiments. [Explanation of symbols]

[0083] 1. Warning system 2. Imaging device 21 Imaging unit 22 Communications Department 3 Information generation device 31 Control Unit 32 Storage section 321 Awareness Program 322 trained models 323 Performance Database 33 Communications Department 34 Information Decision Department 35 Learning Department 4 Display device 41 Communications Department 42 Display section 9. Communication Networks

Claims

1. An imaging device; an information generating device that determines the content of information to be provided to road users based on the image data generated by the imaging device; a display device that displays the content of the information determined by the information generating device, When a plurality of the display devices are installed and a plurality of vehicles are lined up, the contents of the information on each of the display devices are switched in accordance with the passage of the vehicles so that the contents of the information selected for each of the vehicles are displayed on each of the display devices. The alert system is characterized by:

2. The image data includes at least one of road conditions, weather conditions, and vehicle conditions.

2. The warning system according to claim 1 .

3. The display device displays at least one of text, pictures, and moving content.

3. The attention calling system according to claim 1 or 2.

4. The information generating device uses a trained model that has been machine-learned based on past performance data so as to output the content of the information when the image data is input. An attention calling system according to any one of claims 1 to 3.

5. Computer, A means for receiving input of image data generated by an imaging device; A means for determining the content of information to be provided to road users based on the image data; and a means for outputting the contents of the information to a display device; the means for determining the content of the information determines the content of the information to be output to each of the plurality of display devices for each of the vehicles when a plurality of vehicles are connected in a line; The present invention relates to a warning program.

6. The image data includes at least one of road conditions, weather conditions, and vehicle conditions.

6. The warning program according to claim 5,

7. The means for determining the content of the information uses a trained model that has been machine-learned based on past performance data so as to output the content of the information when the image data is input.

7. The warning program according to claim 5 or 6.

Images