Safe driving support system, safe driving support method and program

Abstract

This safe driving assistance system comprises a detection means that detects a person or animal crossing or likely to cross a road on the basis of an image acquired from a camera capable of photographing the road, a determination means that determines whether the crossing position of the detected person or animal is within a crosswalk, and an output means that outputs the result of the detection and the result of the determination.

Description

[Technical field]

[0001] The present invention relates to a safe driving support system, a safe driving support device, a safe driving support method, and a program recording medium. [Background technology]

[0002] Patent Document 1 discloses a roadside device capable of determining the sidewalk conditions. According to the document, the roadside device includes a camera, which is a detection unit that detects information that can determine the sidewalk conditions around the roadside device, a communication unit that communicates with other devices (vehicle-mounted devices or portable electronic devices (e.g., smartphones)), and a control unit. The control unit determines the sidewalk conditions around the roadside device based on the information detected by the camera, and transmits the determined sidewalk conditions to other devices. Paragraph 0063 of the document also describes that the "sidewalk conditions" notified by the roadside device may include the presence and number of pedestrians and bicycles, information on whether the pedestrians include children or elderly people, and conditions such as pedestrians and bicycles jumping out. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 061975 Summary of the Invention [Problem to be solved by the invention]

[0004] The following analysis is provided by the present inventor. According to the invention of Patent Document 1, it is possible to notify other devices (vehicle-mounted devices or portable electronic devices (e.g., smartphones)) of the sidewalk conditions, but there is a problem in that it is not possible to notify of pedestrians or bicycles that are outside the sidewalk, for example, that are stepping onto or crossing the roadway.

[0005] In the future, as autonomous driving technology becomes more widespread, there is a possibility that the accuracy of detecting pedestrians and bicycles using in-vehicle cameras will improve, but since in-vehicle cameras have a low viewpoint and a limited viewing angle, there is a possibility that they will be late in discovering pedestrians and bicycles. In addition, there are some roads where people routinely cross the road outside of crosswalks or diagonally, which are prohibited by the Road Traffic Act, and immediate measures are needed. In addition, in some areas, there have been accidents in which vehicles collide with wild animals.

[0006] An object of the present invention is to provide a safe driving support system, a safe driving support device, a safe driving support method, and a program recording medium that can contribute to the early detection of people or animals crossing outside a crosswalk or crossing diagonally. [Means for solving the problem]

[0007] According to a first aspect, there is provided a safe driving support system including: a detection means for detecting a person or animal that is crossing or may cross a road based on an image acquired from a camera capable of photographing the road; a determination means for determining whether the crossing position of the detected person or animal is within a pedestrian crossing; and an output means for outputting the result of the determination.

[0008] According to a second aspect, there is provided a safety driving support device connected to a server that includes a detection means for detecting a person or animal that is crossing or may cross a road based on an image acquired from a camera capable of photographing the road, and a transmission means for transmitting the detection result of the person or animal, and the safety driving support device includes a means for determining whether the crossing position of the person or animal is within a pedestrian crossing based on the detection result of the person or animal received from the server, and a means for outputting the result of the determination.

[0009] According to a third aspect, there is provided a safe driving support method for detecting a person or animal that is crossing or may cross a road based on an image acquired from a camera, determining whether the crossing position of the detected person or animal is within a pedestrian crossing, and outputting the result of the determination.

[0010] According to a fourth aspect, there is provided a safe driving support method which receives a detection result of a person or animal from a server equipped with a detection means for detecting a person or animal that is crossing or may cross a road based on an image acquired from a camera capable of photographing the road, and a transmission means for transmitting the detection result of the person or animal, determines whether the crossing position of the person or animal is within a pedestrian crossing based on the detection result of the person or animal received from the server, and outputs the result of the determination.

[0011] According to a fifth aspect, there is provided a program recording medium having recorded thereon a program for causing a computer capable of acquiring images from a camera to execute the following processes: detecting a person or animal that is crossing or may cross the road based on the image acquired from the camera; determining whether the crossing position of the detected person or animal is within a pedestrian crossing; and outputting the results of the detection and the determination. Effect of the Invention

[0012] The present invention can contribute to the early detection of people or animals crossing the road outside of a pedestrian crossing or crossing diagonally. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing a configuration of an embodiment of the present invention. [Diagram 2] 2 is a flow chart showing the flow of operations of an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram for explaining the operation of one embodiment of the present invention. [Figure 4] 1 is a diagram showing a schematic configuration of a first embodiment of the present invention; [Diagram 5] 2 is a functional block diagram illustrating a configuration of a server according to the first embodiment of the present invention. FIG. [Figure 6] FIG. 2 is a sequence diagram illustrating an operation of the first exemplary embodiment of the present invention. [Figure 7]FIG. 4 is a diagram for explaining the operation of the server according to the first embodiment of the present invention. [Figure 8] FIG. 4 is another diagram for explaining the operation of the server according to the first exemplary embodiment of the present invention. [Figure 9] FIG. 4 is another diagram for explaining the operation of the server according to the first exemplary embodiment of the present invention. [Figure 10] FIG. 11 is a sequence diagram showing a modified operation of the first exemplary embodiment of the present invention. [Figure 11] FIG. 13 is a diagram showing a schematic configuration of a second embodiment of the present invention. [Figure 12] FIG. 11 is a functional block diagram illustrating a configuration of a server according to a second embodiment of the present invention. [Figure 13] FIG. 11 is a sequence diagram illustrating an operation of the second exemplary embodiment of the present invention. [Figure 14] FIG. 13 is a diagram showing a schematic configuration of a third embodiment of the present invention. [Figure 15] FIG. 11 is a functional block diagram illustrating a configuration of a server according to a third embodiment of the present invention. [Figure 16] FIG. 11 is a sequence diagram showing the operation of the third exemplary embodiment of the present invention. [Figure 17] FIG. 11 is a diagram for explaining the operation of a server according to the third exemplary embodiment of the present invention. [Figure 18] FIG. 11 is another diagram for explaining the operation of the server according to the third exemplary embodiment of the present invention. [Figure 19] FIG. 1 is a diagram showing the configuration of a computer that can function as a server of the present invention. [Figure 20] FIG. 4 is a functional block diagram showing a modified configuration of the first embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] First, an overview of one embodiment of the present invention will be described with reference to the drawings. Note that the reference numerals in the drawings attached to this overview are attached to each element for convenience as an example to aid in understanding, and are not intended to limit the present invention to the illustrated form. Furthermore, the connection lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional. A unidirectional arrow is a schematic representation of the flow of a main signal (data), and does not exclude bidirectionality. Furthermore, although there are ports or interfaces at the connection points of input and output of each block in the drawings, they are not shown.

[0015] In one embodiment, the present invention can be realized in a safe driving support system 10 including a detection means 11, a determination means 12, and an output means 13, as shown in FIG.

[0016] The detection means 11 detects a person or animal that is crossing or may cross a road based on an image acquired from a camera capable of photographing the road. A camera that grasps traffic flow and the situation at an intersection can be used as the camera 14. Of course, a security camera or a camera mounted on another mobile object can also be used as the camera 14. As a method for the detection means 11 to detect a person or animal from an image, a method using a class identifier that identifies the class of an object in the image or an object detection technology used in the fields of autonomous driving and security can be used.

[0017] The determination means 12 determines whether the crossing position of the detected person or animal is within a crosswalk. Hereinafter, in this specification, the term "crossing position" includes the crossing position, which is the current position of the person or animal detected from the image when crossing the road, and the planned crossing position, which is the predicted future position when crossing. If the position of the crosswalk in the image obtained by the camera 14 is known, the determination means 12 can determine whether the crossing position of the person or animal is within a crosswalk based on the position of the person or animal captured in the image (coordinates in the image). In addition, when determining whether the crossing position of the person or animal is within a crosswalk based on the position of the person or animal captured in the image (coordinates in the image), the size of the image of the person or animal in the image may be taken into consideration in addition to the position of the image of the person or animal in the image. In addition, if the position of the crosswalk is unknown or the camera 14 is moving, a method of determining whether the crossing position of the person or animal is within a crosswalk based on the image of the crosswalk captured in the image may be adopted. For example, a model that has previously learned the pattern of a pedestrian crossing may be used to detect an area in an image where a pedestrian crossing is captured, and if the crossing position of a person or animal in the image overlaps or is close to the area of ​​the pedestrian crossing, it may be determined that the crossing position is within a pedestrian crossing. Furthermore, if the position of the pedestrian crossing can be obtained as position information on map information, a method may be adopted in which the crossing position of a person or animal is found on the map information, and thereby it may be determined whether the crossing position of a person or animal is within a pedestrian crossing or not.

[0018] The output means 13 outputs the detection results and the judgment results to a predetermined output destination. The output destination of the information from the output means 13 can be a vehicle that may encounter (collide with) the person or animal. In another embodiment, the output means 13 can be configured to output the information to an electronic bulletin board or a large display installed in a position that is easily visible to the driver of the vehicle, such as near the road.

[0019] Fig. 2 is a flow chart showing the flow of the operation of the safe driving support system 10. As shown in Fig. 2, the safe driving support system 10 detects a person or an animal that is crossing or may cross the road based on an image acquired from the camera 14 (step S001). For example, the safe driving support system 10 detects pedestrians P11 and P13, a bicycle P12, and an animal P14 in Fig. 3 based on an image acquired from the camera 14.

[0020] Next, the safe driving support system 10 judges whether the crossing position of the detected person or animal is within the crosswalk or not. For example, the safe driving support system 10 judges that the pedestrian P11 in Fig. 3 is in a position to cross the crosswalk. On the other hand, the safe driving support system 10 judges that the bicycle P12, pedestrian P13, and animal P14 in Fig. 3 are in a position to cross the crosswalk outside.

[0021] Finally, the safe driving support system 10 outputs the above detection results and judgment results to a predetermined output destination. The safe driving support system 10 notifies vehicles traveling on the road of the presence of the bicycle P12, pedestrian P13, and animal P14 in Fig. 3 and that they are in positions to cross outside the crosswalk. This allows the driver of the vehicle traveling on this road to take appropriate measures such as paying attention to the road ahead or slowing down.

[0022] As described above, according to this embodiment, it is possible to prevent accidents by early detection of people or animals crossing outside the crosswalk or crossing diagonally and notifying vehicles, etc.

[0023] [First embodiment] Next, the first embodiment of the present invention will be described in detail with reference to the drawings. Fig. 4 is a diagram showing a schematic configuration of the first embodiment of the present invention. Referring to Fig. 4, a configuration including a server 100, a base station 200, a terminal 210, and a camera 140 is shown.

[0024] The camera 140 is installed near the traffic lights at the intersection and captures vehicles passing through the intersection where a crosswalk is installed. In this embodiment, the camera 140 is used as a camera for detecting people or animals crossing the road at a location other than a crosswalk. Note that in this embodiment, the case of detecting pedestrians and automobiles as specific examples of people will be mainly described. Also, in the example of FIG. 4, an example is shown in which two cameras are arranged diagonally opposite each other at the intersection, but there is no limit to the number of cameras. For example, one or more cameras may be installed on each of the signal poles of the traffic lights at the intersection. Also, in the example of FIG. 4, the camera 140 is oriented toward the inside of the intersection, but a camera oriented toward the outside of the intersection may be used.

[0025] The terminal 210 is a terminal equipped with a function of wirelessly connecting to the base station 200. In this embodiment, the terminal 210 functions as a transmitting device that transmits an image captured by the camera 140 to the server 100 via the base station 200. The terminal 210 may be a terminal for traffic control purposes that transmits an image captured by the camera 140 to a traffic control center, a server, or the like via the base station 200.

[0026] The base station 200 transmits the camera image received from the terminal 210 to the server 100. This base station 200 may be a base station that provides services not only to the terminal 210 but also to an in-vehicle terminal of a vehicle passing near an intersection or a pedestrian terminal. For example, the base station 200 may be a base station of a fifth-generation mobile communication system installed by a telecommunications carrier, or may be a base station of a network called local 5G operated by a party other than the telecommunications carrier.

[0027] Server 100 detects pedestrians and bicycles attempting to cross outside the crosswalk from images captured by camera 140 and notifies vehicles of the detection. Server 100 may be a server installed on a cloud platform or the Internet, or may be an MEC server installed physically close to a base station. Note that "MEC" is an abbreviation for Multi-access Edge Computing or Multi-access Edge Computing.

[0028] Fig. 5 is a functional block diagram showing the configuration of the server 100. Referring to Fig. 5, the server 100 includes a detection unit 101, a determination unit 102, and a transmission unit 103.

[0029] The detection unit 101 corresponds to the above-mentioned detection means 11, and receives an image captured by the camera 140 from the base station 200. The detection unit 101 detects pedestrians, bicycles, etc. that may be crossing a road in the image captured by the camera 140.

[0030] The determination unit 102 corresponds to the above-mentioned determination means 12, and determines whether a pedestrian or bicycle detected by the detection unit 101 is crossing a crosswalk or is about to cross. Simply put, the determination unit 102 can determine whether a pedestrian or bicycle is crossing a crosswalk or is about to cross, based on the position at which the pedestrian or bicycle is captured in the image of the camera 140. Furthermore, in this embodiment, the determination unit 102 specifies the position of the pedestrian or bicycle. The position information here may be absolute coordinates such as latitude and longitude, or may be position information specified by intersection information and a relative position from the intersection.

[0031] The transmitting unit 103 corresponds to the output means 13 described above, and transmits the positions of the detected pedestrians, bicycles, etc., and the determination results of whether or not they are crossing the pedestrian crossing or about to cross, to a vehicle approaching the intersection. Various methods are possible for transmitting this information by the transmitting unit 103. For example, the transmitting unit 103 may broadcast this information via the base station 200. In addition, if the server 100 can obtain the IP address, etc., of the vehicle-mounted terminal of each vehicle, the server 100 may select a vehicle and transmit this information by multicast or unicast. Furthermore, the transmitting unit 103 may transmit this information using a roadside unit, etc. installed on the road. In addition to the above information, the transmitting unit 103 may add various information and transmit it to a vehicle passing through the intersection. For example, the transmitting unit 103 may transmit to the vehicle, images of pedestrians and bicycles in images obtained from the camera 140, attribute information obtained from these images, etc.

[0032] Next, the operation of this embodiment will be described in detail with reference to the drawings. Fig. 6 is a sequence diagram showing the operation of the first embodiment of the present invention. Referring to Fig. 6, the terminal 210 transmits an image received from the camera 140 to the server 100 via the base station 200 (step S101).

[0033] The server 100 that receives the image detects pedestrians or bicycles that are crossing the road or that may cross the road from the received image. The possibility of crossing the road may be determined based on the previous behavior of the pedestrian or bicycle, in addition to the case where an image of the pedestrian actually starting to cross the road is obtained. For example, if a pedestrian is on the curb of the road, or if a pedestrian is heading toward the road in a place without a crosswalk and checking both sides, it can be said that there is a high possibility that the pedestrian will cross the road. Also, if a bicycle is wobbling while traveling or has changed the direction of the body and started to cross the road, it can be determined that there is a high possibility that the pedestrian will cross the road. Or, if a cyclist looks back toward the roadway, it can be said that there is a high possibility that the cyclist will cross the road diagonally.

[0034] Next, the server 100 determines the current position of the detected pedestrian or bicycle and whether or not the crossing position is within a pedestrian crossing (step S103).

[0035] Next, the server 100 transmits to the vehicle the position of the detected pedestrian or bicycle and information as to whether or not the crossing position is within a pedestrian crossing (step S104).

[0036] The on-board terminal of the vehicle that has received the information performs an operation based on the information received from the server 100 (step S105). Specifically, the on-board terminal that has received the information determines whether or not to use the information received from the server 100, referring to the position, moving direction, and moving speed of the vehicle itself. For example, even if there is a pedestrian trying to cross outside the crosswalk, if the vehicle itself is moving away from the pedestrian, there is no possibility of crossing (colliding) with the pedestrian, so the on-board terminal that has received the information may discard this information. Also, even if the distance between the vehicle and the pedestrian is short, if the vehicle itself is stopped or moving slowly, there is little possibility of crossing (colliding) with the pedestrian, so the on-board terminal that has received the information may discard this information. Also, if the pedestrian is trying to cross within the crosswalk, there is no immediate danger, so the on-board terminal that has received the information may discard this information.

[0037] On the other hand, if the vehicle is approaching a pedestrian who is crossing outside the crosswalk, the in-vehicle terminal uses this information to warn the driver. For example, the in-vehicle terminal uses the display of the car navigation device or AR (Augmented Reality) display to display the pedestrian and its position notified from the server 100. This allows the driver to take action to avoid crossing (collision) with the pedestrian who is crossing outside the crosswalk.

[0038] Next, a specific operation example of the server 100 of this embodiment will be described with reference to Figs. 7 to 9. For example, it is assumed that an image as shown in Fig. 7 is sent from the camera 140. In this case, the server 100 detects pedestrians P1 and P3 who may be crossing the road from the image in Fig. 7. It is assumed that the pedestrian P2 who has already crossed the crosswalk has already been detected at a previous timing.

[0039] Next, the server 100 judges whether the crossing positions of the detected pedestrians P1 and P3 are within the crosswalk. For example, in the example of FIG. 7, the pedestrian P1 is at a waiting position in front of the crosswalk, so the crossing position is judged to be within the crosswalk. On the other hand, the pedestrian P3, who is photographed in the distance, is not at the waiting position of the crosswalk, so the crossing position is judged to be outside the crosswalk. When the server 100 transmits such a judgment result together with the presence of the pedestrians P1 and P3, the in-vehicle terminal or the like can perform an operation to alert the driver of the vehicle to the pedestrian P3. In addition, at this time, the in-vehicle terminal or the like may omit to alert the driver of the vehicle to the pedestrian P1 in order to enhance the effect of alerting the pedestrian P3. This is because the pedestrian P1 is at a waiting position of the crosswalk and there is little possibility of crossing (collision) with a vehicle.

[0040] FIG. 8 is a diagram showing an image transmitted from the camera 140 after a certain time has elapsed from the point in time shown in FIG. 7. In the example of FIG. 8, the pedestrian signal that pedestrian P1 has been waiting for turns green, and pedestrian P1 starts crossing the road. Also, in the distance, a bicycle P4 is seen heading toward the intersection. In this case, the server 100 detects bicycle P4 that may be crossing the road from the image in FIG. 8. Note that pedestrian P1 who has started crossing the crosswalk is not subject to warning drivers of vehicles, etc., because the crossing position is within the crosswalk.

[0041] Next, the server 100 judges whether the crossing position of the detected bicycle P4 is within the crosswalk. For example, in the example of Fig. 8, the bicycle P4 is not at the waiting position of the crosswalk, so the crossing position is judged to be outside the crosswalk. When the server 100 sends such a judgment result together with the presence of the bicycle P4, the in-vehicle terminal etc. can perform an operation to alert the driver etc. of the vehicle to the bicycle P4. Of course, if a pedestrian other than the bicycle P4 is found who is actually crossing the road and is located outside the crosswalk, the pedestrian will also be the target of the warning to the driver etc. of the vehicle.

[0042] In the above examples of Figs. 7 and 8, the server 100 is described as sending the presence of pedestrians and bicycles and the result of the determination of their crossing positions. However, the server 100 may create information to be output to the vehicle-mounted terminal. For example, if the server 100 knows the position and direction of each vehicle and can send information to each vehicle individually, the server 100 can create a message for each vehicle. For example, as shown in Fig. 9, if there is a vehicle that enters the intersection from the left side of Fig. 9 and turns left, the server 100 can create a message for the vehicle saying "Watch out for pedestrians outside the crosswalk ahead of the left turn" and send it to the vehicle-mounted terminal. Then, the vehicle-mounted terminal that receives the message outputs the message received from the server in voice or text. In this way, it is possible to reduce the load on the vehicle-mounted terminal and increase the promptness of the attention message.

[0043] In the above description, the server 100 is described as determining the crossing position of a pedestrian or bicycle, but the vehicle side may determine the crossing position of a pedestrian or bicycle (see FIG. 10). In this case, a configuration as shown in FIG. 20 can be adopted. The server 100c includes a detection means (corresponding to the detection unit 101 in FIG. 20) for detecting a person or animal that is crossing or may cross the road based on an image acquired from a camera capable of photographing the road, and a transmission means (corresponding to the transmission unit 103c in FIG. 20) for transmitting the detection result to the vehicle side. The vehicle-mounted terminal 400 of the vehicle serves as a safe driving support device, and includes a means (corresponding to the determination unit 402 in FIG. 20) for determining whether the crossing position of the person or animal is within the crosswalk based on the detection result of the person or animal received from the server 100, and a means (corresponding to the output unit 403 in FIG. 20) for outputting the result of the determination. The server 100c detects pedestrians and bicycles based on images acquired from the camera, and then transmits information about the detected pedestrians and bicycles to the vehicle (steps S202 and S203 in FIG. 10). The vehicle then determines the crossing position of the detected pedestrians and bicycles (step S203 in FIG. 10). This makes it possible to reduce the load on the server.

[0044] [Second embodiment] Next, a second embodiment in which a server provides information on the light status (signal aspect) of traffic lights at an intersection will be described in detail with reference to the drawings. FIG. 11 is a diagram showing a schematic configuration of the second embodiment of the present invention. The configuration differs from the first embodiment shown in FIG. 4 in that a server 100a is connected to a signal control device 300 and is capable of acquiring information on the light status of traffic lights. Other configurations are the same as those of the first embodiment, so descriptions will be omitted and the following description will focus on the differences.

[0045] The signal control device 300 is a device that controls the lights of traffic lights installed at an intersection based on preset control information. For example, the signal control device 300 controls the lighting state of each of the traffic lights at the intersection based on information such as cycle, split, and offset.

[0046] Fig. 12 is a functional block diagram showing the configuration of a server 100a according to the second embodiment of the present invention. The difference from the server 100 according to the first embodiment shown in Fig. 5 is that a signal control information acquisition unit 104 is added to the server 100a, and that a transmission unit 103a transmits the lighting status of the traffic lights at the intersection in addition to the positions and crossing positions of pedestrians, etc.

[0047] The signal control information acquisition unit 104 acquires information on the lighting state of a traffic signal or control information on which the information is based (hereinafter, these will be collectively referred to as "signal control information") from the signal control device 300.

[0048] Next, the operation of this embodiment will be described in detail with reference to the drawings. Fig. 13 is a sequence diagram showing the operation of the second embodiment of the present invention. Referring to Fig. 13, first, the terminal 210 transmits an image received from the camera 140 to the server 100a via the base station 200 (step S301).

[0049] Thereafter, steps S302 and S303 are the same as steps S102 and S103 in the first embodiment. Thereafter, the server 100a transmits the signal control information, the positions of the detected pedestrians, etc., and the crossing positions to the vehicle side (step S304).

[0050] The on-board terminal of the vehicle that receives the information performs an operation based on the information received from the server 100a. At that time, what is significantly different from the first embodiment is that the on-board terminal judges whether or not to use the information received from the server 100a by referring to the position, moving direction, moving speed, and also the signal control information of the vehicle. For example, even if there is a pedestrian trying to cross the road ahead of the vehicle, if the light state of the traffic light turns red at the time of passing the intersection and the vehicle is expected to stop, the on-board terminal that received the information discards this information. Also, for example, there is a case where the vehicle is trying to turn left and there is a pedestrian trying to cross the road ahead of the left turn, but not the crosswalk. In this case, if the light state of the traffic light is expected to allow the vehicle to turn left, the on-board terminal issues a warning to the driver. Note that whether or not the vehicle is trying to turn left can be determined, for example, by whether or not the blinker for turning left is on. In this way, the on-board terminal starts a warning to the driver only when it is expected to cross a pedestrian trying to cross the road outside the crosswalk, based on the light state of the traffic light.

[0051] As described above, according to the present embodiment, it is possible to cause the vehicle-mounted terminal of the vehicle to perform an attention-calling operation for pedestrians, taking into consideration the lighting state of the traffic lights. In addition, the control of whether or not to perform an attention-calling operation for the driver using the above-mentioned signal control information is not limited to the above example. For example, the vehicle-mounted terminal can be caused to refer to the signal control information and calculate the degree of intersection between the vehicle and the pedestrian who is trying to cross the road outside the crosswalk. Then, the vehicle-mounted terminal may perform an attention-calling operation for different drivers depending on this degree. For example, when the degree is large, the driver can be notified in a more noticeable manner. In addition, when the degree is large, the degree information may be used to control the vehicle itself, such as to decelerate the vehicle.

[0052] In the above embodiment, the in-vehicle terminal determines whether to activate the driver's attention based on the information received from the server 100a, but the server 100a may determine whether to activate the attention. In this case, the server 100a determines whether the vehicle is expected to cross a pedestrian who is trying to cross outside the crosswalk based on the position, moving direction, moving speed, and the lighting status of the signal of the target vehicle. For example, even if there is a pedestrian who is trying to cross outside the crosswalk on the road ahead of the target vehicle, if the target vehicle is expected to stop according to the signal light, the server 100a suppresses the transmission of the position and crossing position of the detected pedestrian, etc. This makes it possible to transmit the result of the determination to a vehicle that may cross a person, bicycle, animal, etc. Similarly, when the server 100a determines whether to activate the attention, the server 100a can refer to the signal control information to calculate the degree of crossing between the vehicle and the pedestrian who is trying to cross outside the crosswalk, and use it to warn the driver and control the vehicle.

[0053] [Third embodiment] Next, a third embodiment in which the server provides the location of pedestrians and bicycles as well as the direction and speed of movement will be described in detail with reference to the drawings. Figure 14 is a diagram showing a schematic configuration of the third embodiment of the present invention. The basic configuration is the same as that of the first embodiment shown in Figure 4, so the following description will focus on the differences.

[0054] Fig. 15 is a functional block diagram showing the configuration of a server 100b according to the third embodiment of the present invention. The difference from the server 100 according to the first embodiment shown in Fig. 5 is that an analysis unit 105 is added to the server 100a, and that a transmission unit 103b transmits the moving direction and moving speed of pedestrians, etc. in addition to the positions and crossing positions of pedestrians, etc.

[0055] More specifically, the analysis unit 105 analyzes the moving direction and moving speed of a pedestrian or bicycle based on multiple images captured by the camera 140. The moving direction and moving speed of a pedestrian or bicycle can be estimated using the movement and the amount of movement of a subject captured in two or more images that are consecutive in time.

[0056] Next, the operation of this embodiment will be described in detail with reference to the drawings. Fig. 16 is a sequence diagram showing the operation of the third embodiment of the present invention. The operation shown in steps S401 to S405 in Fig. 16 corresponds to S101 to S105 in the first embodiment.

[0057] The operation added in this embodiment is the analysis process of the moving direction and moving speed of pedestrians, etc. in step S406. Note that the process of step S406 may be performed before the determination of the crossing position in step S403.

[0058] After completing the analysis of the movement direction and movement speed of the pedestrians, etc., the server 100b transmits the position, crossing position, movement direction and movement speed of the detected pedestrians, etc. to the vehicle side (step S404).

[0059] The on-board terminal of the vehicle that receives the information performs an operation based on the information received from the server 100b. For example, as shown in FIG. 17, when a pedestrian P3 or a bicycle P4 is detected trying to cross the road ahead of the vehicle outside the crosswalk, the on-board terminal displays their positions using AR display or the like. Furthermore, the on-board terminal displays the moving direction and moving speed of the pedestrian, etc., using arrows. In the example of FIG. 17, the moving speed is represented by the length of the arrow, and the moving direction is represented by the orientation of the arrow, but the display form of the moving direction and moving speed is not limited to this. For example, the moving speed can be represented by a numerical value, the thickness of the arrow, or a color. Similarly, the moving direction can be represented by a symbol indicating a direction.

[0060] According to this embodiment, which records the moving direction and moving speed of pedestrians and bicycles in addition to their positions, it is possible to visually display objects that are likely to cross paths with the vehicle and that require more attention. For example, in the case of FIG. 17, there are pedestrian P3 and bicycle P4 outside the crosswalk, but it can be seen that bicycle P4 is moving faster and is approaching the vehicle. Based on this information, the driver of the vehicle can slow down or take evasive action.

[0061] Furthermore, according to this embodiment, it is possible to identify the movements of pedestrians waiting in the median strip or zebra zone, as shown in Fig. 18. For example, in the case of Fig. 18, there is a pedestrian P5 crossing the road outside the crosswalk, but since the pedestrian is waiting in the median strip, it is possible to inform the driver that there is little risk of an immediate crossing.

[0062] In addition, the moving direction and moving speed of the pedestrians etc. obtained in this embodiment may be used not only for display but also for determining the presence or absence of intersection and the degree of intersection on the in-vehicle terminal or server 100b side. Also, the in-vehicle terminal or server 100b may control the vehicle (automatic deceleration, warning notification, etc.) according to the moving direction and moving speed.

[0063] Although each embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiments, and further modifications, substitutions, and adjustments can be made without departing from the basic technical idea of ​​the present invention. For example, the system configuration, the configuration of each element, and the representation form of data, etc. shown in each drawing are examples to help understand the present invention, and are not limited to the configurations shown in these drawings. For example, in the above-mentioned first to third embodiments, the camera 140 is described as being installed at an intersection, but the arrangement of the camera 140 is not limited to this. For example, a camera installed on a street or a camera installed for crime prevention purposes can also be used as the camera 140.

[0064] In addition, in the above-mentioned embodiments, the server side is described as determining whether or not the crossing position of the pedestrian is within the crosswalk, but it is also possible to adopt a configuration in which the vehicle (vehicle-mounted terminal) determines whether or not the pedestrian is within the crosswalk. In this case, the server detects the pedestrian and transmits the position to the vehicle (vehicle-mounted terminal).

[0065] Moreover, the procedures shown in each of the above-mentioned embodiments can be realized by a computer that functions as a device constituting the safe driving support system or the vehicle-side safe driving support device. Specifically, the procedures can be realized by a program that causes a computer (9000 in FIG. 19) to realize the functions of these devices. Such a computer is exemplified by a configuration including a CPU (Central Processing Unit) 9010, a communication interface 9020, a memory 9030, and an auxiliary storage device 9040 in FIG. 19. That is, an object recognition program and a crossing position determination program may be executed by the CPU 9010 in FIG. 19. The computer executes the safe driving support method by the program.

[0066] That is, each unit (processing means, function) of the above-mentioned server etc. can be realized by a computer program that causes a processor mounted on these devices to execute each of the above-mentioned processes using its hardware. The program can be recorded on a computer-readable program recording medium. The recording medium can be a non-transitory one such as a semiconductor memory, a hard disk, a magnetic recording medium, an optical recording medium, etc.

[0067] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. [Appendix 1] A detection means for detecting a person or an animal that is crossing or may cross a road based on an image acquired from a camera capable of photographing the road; A determination means for determining whether the crossing position of the detected person or animal is within a pedestrian crossing; An output means for outputting the result of the determination; A safe driving support system equipped with [Appendix 2] The safety driving support system according to claim 1, wherein the camera is a camera installed near a traffic light at an intersection. [Appendix 3] The safe driving support system according to claim 2, wherein the output means outputs the result of the determination to a vehicle located in the vicinity of the intersection. [Appendix 4] moreover, means for acquiring information indicating a state of a traffic light at the intersection; The safe driving support system of claim 2 or 3, wherein the output means provides information indicating the state of traffic lights at the intersection in addition to the result of the judgment. [Appendix 5] moreover, means for acquiring information indicating a state of a traffic light at the intersection; A safe driving support system as claimed in claim 2 or 3, wherein the output means transmits the result of the judgment to a vehicle that may cross the person or animal based on information indicating the state of traffic lights at the intersection. [Appendix 6] moreover, A means for analyzing at least one of a moving direction and a moving speed of the person or animal, 6. The safe driving support system according to any one of claims 1 to 5, wherein the output means provides at least one of a moving direction and a moving speed of the person or animal in addition to the result of the judgment. [Appendix 7] 6. The safety driving support system according to any one of Appendices 2 to 5, wherein the detection means detects a person or an animal that may be crossing a road connected to the intersection from an image captured by the camera. [Appendix 8] 8. The safe driving support system according to claim 1, wherein the determination means determines whether the crossing position of the person or animal is within a pedestrian crossing based on a position of an image captured by the camera in the image. [Appendix 9] A safe driving support device connected to a server including: a detection means for detecting a person or an animal that is crossing a road or that may cross the road based on an image acquired from a camera capable of photographing the road; and a transmission means for transmitting a detection result of the person or the animal, a means for determining whether or not a crossing position of the person or animal is within a pedestrian crossing based on the person or animal detection result received from the server; A means for outputting the result of the determination; A safety driving support device equipped with: [Appendix 10] receiving information indicating a state of a traffic light at an intersection from the server; 10. A safe driving support device according to claim 9, which outputs a result of the judgment when there is a possibility that the vehicle will cross the person or the animal, based on information indicating a state of traffic lights at the intersection. [Appendix 11] Detecting a person or an animal that is crossing or may cross a road based on an image acquired from the camera; determining whether the crossing position of the detected person or animal is within a pedestrian crossing; outputting the result of the determination; Safe driving support methods. [Appendix 12] receiving a detection result of a person or animal from a server including a detection means for detecting a person or animal that is crossing or may cross a road based on an image acquired from a camera capable of photographing the road, and a transmission means for transmitting the detection result of the person or animal; determining whether or not the crossing position of the person or animal is within a pedestrian crossing based on the person or animal detection result received from the server; outputting the result of the determination; Safe driving support methods. [Appendix 13] A computer that can acquire images from the camera. A process for detecting a person or an animal that is crossing or may cross a road based on an image acquired from the camera; A process of determining whether the crossing position of the detected person or animal is within a pedestrian crossing; A process of outputting the results of the detection and the determination; A program recording medium on which a program for executing the above is recorded. The embodiments of Supplementary Notes 9 to 13 can be expanded into the embodiments of Supplementary Notes 2 to 8, similarly to the embodiment of Supplementary Note 1.

[0068] The disclosures of the above patent documents are incorporated herein by reference. Within the framework of the entire disclosure of the present invention (including the scope of claims), modifications and adjustments of the embodiments and examples are possible based on the basic technical ideas. Furthermore, within the framework of the disclosure of the present invention, various combinations or selections (including partial deletions) of various disclosed elements (including each element of each claim, each element of each embodiment or example, each element of each drawing, etc.) are possible. In other words, the present invention naturally includes various modifications and corrections that a person skilled in the art would be able to make in accordance with the entire disclosure including the scope of claims and the technical ideas. In particular, with regard to the numerical ranges described in this specification, any numerical value or subrange included in the range should be interpreted as being specifically described even if not otherwise specified. [Explanation of symbols]

[0069] 10 Safe Driving Support Systems 11 Detection methods 12 Judgment means 13 Output Method 14, 140 Camera 100, 100a, 100b, 100c servers 101 Detection unit 102 Judgment section 103, 103a, 103b Transmitting section 104 Signal control information acquisition unit 105 Analysis Department 200 base stations 210 Terminals 300 Signal control device 400 Vehicle-mounted terminal 402 Judgment section 403 Output section 9000 Computers 9010 CPU 9020 Communication Interface 9030 Memory 9040 Auxiliary storage device P1~P3, P5, P11, P13 Pedestrians P4, P12 Bicycle P14 Animals

Claims

1. A detection means for detecting a person or an animal that is crossing or may cross a road based on an image acquired from a camera installed near a traffic light at an intersection, the camera being capable of photographing the road from a position higher than a vehicle; A determination means for determining whether the crossing position of the detected person or animal is within a pedestrian crossing; means for acquiring information indicating a state of a traffic light at the intersection; and an output means for outputting the result of the determination indicating whether the crossing position is within a pedestrian crossing and information indicating the state of the traffic lights at the intersection to a vehicle located near the detected person or animal.

2. A detection means for detecting a person or an animal that is crossing or may cross a road based on an image acquired from a camera installed near a traffic light at an intersection, the camera being capable of photographing the road from a position higher than a vehicle; A determination means for determining whether the crossing position of the detected person or animal is within a pedestrian crossing; means for acquiring information indicating a state of a traffic light at the intersection; and an output means for outputting the result of the determination indicating whether or not the crossing position is within a pedestrian crossing to a vehicle located near the detected person or animal, The output means is a safe driving support system that outputs the result of the judgment to a vehicle that may cross the person or animal based on information indicating the state of traffic lights at the intersection.

3. A detection means for detecting a person or an animal that is crossing or may cross a road based on an image acquired from a camera installed near a traffic light at an intersection, the camera being capable of photographing the road from a position higher than a vehicle; A determination means for determining whether the crossing position of the detected person or animal is within a pedestrian crossing; and an output means for outputting the result of the determination indicating whether or not the crossing position is within a pedestrian crossing to a vehicle located near the detected person or animal, The detection means is a safe driving support system that detects people or animals that may be crossing a road connected to the intersection from images captured by the camera.

4. 4. The safe driving support system according to claim 1, wherein the output means outputs the result of the determination to a vehicle located near the intersection.

5. a detection means for detecting a person or an animal that is crossing or may cross the road based on an image acquired from a camera that can photograph the road from a position higher than the vehicle; A determination means for determining whether the crossing position of the detected person or animal is within a pedestrian crossing; A means for analyzing at least one of a moving direction and a moving speed of the person or animal; and an output means for outputting the result of the determination indicating whether or not the crossing position is within a pedestrian crossing to a vehicle located near the detected person or animal, A safe driving support system, wherein the output means provides at least one of the moving direction and moving speed of the person or animal in addition to the result of the judgment.

6. The determining means determines whether or not the crossing position of the person or the animal is within a pedestrian crossing based on a position in the image acquired from the camera.

6. A safe driving support system according to claim 1.

7. Detecting people or animals that are crossing or may cross a road based on images acquired from a camera installed near a traffic light at an intersection that photographs the road from a position higher than the vehicle; determining whether the crossing position of the detected person or animal is within a pedestrian crossing; Obtaining information indicating a state of a traffic light at the intersection; outputting the result of the determination indicating whether the crossing position is within a pedestrian crossing and information indicating a state of a traffic light at the intersection to a vehicle located near the detected person or animal; Safe driving support methods.

8. Detecting people or animals that are crossing or may cross a road based on images acquired from a camera installed near a traffic light at an intersection that photographs the road from a position higher than the vehicle; determining whether the crossing position of the detected person or animal is within a pedestrian crossing; Obtaining information indicating a state of a traffic light at the intersection; outputting a result of the determination indicating whether or not the crossing position is within a pedestrian crossing to a vehicle located near the detected person or animal; In the output, the result of the determination is output to a vehicle that may cross the person or animal based on information indicating a state of a traffic light at the intersection. Safe driving support methods.

9. A computer that can acquire images from a camera installed near the traffic lights at intersections that takes pictures of the road from a higher position than the vehicle. A process for detecting a person or an animal that is crossing or may cross a road based on an image acquired from the camera; A process of determining whether the crossing position of the detected person or animal is within a pedestrian crossing; means for acquiring information indicating a state of a traffic light at the intersection; a process of outputting the result of the detection, the result of the determination indicating whether the crossing position is within a pedestrian crossing, and information indicating the state of traffic lights at the intersection to a vehicle located near the detected person or animal; A program that executes the following.

10. A computer that can acquire images from a camera installed near the traffic lights at intersections that takes pictures of the road from a higher position than the vehicle. A process for detecting a person or an animal that is crossing or may cross a road based on an image acquired from the camera; A process of determining whether the crossing position of the detected person or animal is within a pedestrian crossing; obtaining information indicating a state of a traffic light at the intersection; outputting the result of the detection and the result of the determination indicating whether the crossing position is within a pedestrian crossing to a vehicle located near the detected person or animal; Run the command, In the output process, the program outputs the result of the judgment to a vehicle that may cross the person or animal based on information indicating the state of the traffic lights at the intersection.

Images