Systems etc.

Abstract

It provides a system that differs from conventional systems, such as a mechanism to determine whether the vehicle being photographed is highly likely to be engaging in dangerous driving. [Solution] The system has a control unit that determines the possibility of dangerous driving by a target vehicle, which has been identified based on image information acquired by one or more cameras, based on the number of times the target vehicle has entered a predetermined proximity area.

Description

Technical Field

[0001] The present invention relates to a system and the like.

Background Art

[0002] Japanese Patent Application Laid-Open No. 2013-134590 (Patent Document 1) discloses mounting a rear camera and a front camera on a vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Document 1, no consideration has been given to a mechanism for determining whether the photographed target vehicle is likely to be engaged in dangerous driving. Therefore, the present invention provides a mechanism different from the conventional ones, such as a mechanism for determining whether the photographed target vehicle is likely to be engaged in dangerous driving.

[0005] [[ID=Furthermore, the applicant intends to obtain rights through divisional applications, amendments, etc., for configurations that aim to achieve the effects derived from the components disclosed in this specification and the drawings. For example, problems disclosed in this specification can be reinterpreted as "the problem is" where phrases like "can do" or "is possible" are replaced with "the problem is." Each problem is described independently, and the applicant intends to obtain rights to the configurations for solving each of these problems individually through divisional applications, amendments, etc. Even if a problem is implicitly understood from the description in the specification, the applicant intends to include a part of the configuration described in this specification in the claims through amendment or divisional application. In addition, configurations that solve problems by combining these independent problems are also disclosed, and the applicant intends to obtain rights to them. [Means for solving the problem]

[0006] To solve the above problems, for example, the configuration described in the claims may be adopted. The present invention includes several means for solving the above-mentioned problems, but one example is a system having a control unit that determines the possibility of dangerous driving by a target vehicle, which has been identified based on image information acquired by one or more cameras, based on the number of times the target vehicle has entered a predetermined proximity area. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a mechanism different from conventional ones, such as a mechanism for determining whether a photographed vehicle is likely to be in a state of dangerous driving. Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments.

[0008] Furthermore, the effects of the present invention are not limited to those described herein. Effects derived from the components disclosed in this specification and the drawings are also disclosed, and the applicant intends to obtain rights to such components through divisional applications, amendments, etc. For example, phrases such as "can do" or "is possible" in this specification are descriptions that clearly indicate the effects to be achieved, and there are components that demonstrate effects even without such descriptions. Moreover, there are effects that can be grasped by the component even without such descriptions. [Brief explanation of the drawing]

[0009] [Figure 1] This is an example of an explanatory diagram illustrating the installation location of the drive recorder 101 inside the vehicle in this embodiment. [Figure 2] This is an example of a camera configuration diagram for the drive recorder 101. [Figure 3] This is another example of a camera configuration diagram within the drive recorder 101. [Figure 4] This is an example of a system connection diagram including the drive recorder 101. [Figure 5] This is an example of a schematic hardware configuration diagram for drive recorder 101 (F, R). [Figure 6] This is an example of a schematic configuration diagram for management server 106. [Figure 7] This is an example of a hardware configuration diagram for user terminal 107. [Figure 8] This is an example of a block diagram of the control unit 303. [Figure 9] This is an example of a processing flow for detecting aggressive driving. [Figure 10] This is an example of a vehicle detection processing flow. [Figure 11] This is an example of a diagram illustrating the detection of a vehicle behind the vehicle. [Figure 12] This is an example of an explanatory diagram for the hype index judgment table. [Figure 13] This is an example of an explanatory diagram for the "Incitement Index" judgment graph. [Figure 14] This is an example of a processing flow based on the level of provocation. [Figure 15] This is an example of a configuration diagram of a camera in the drive recorder 101. [Figure 16] This is an example of an explanatory diagram of a first operation example of the drive recorder 101. [Figure 17] This is an example of an explanatory diagram of a first operation example of the drive recorder 101. [Figure 18] This is an example of an explanatory diagram of a configuration for measuring the inter-vehicle distance obtained from perspective (perspective view). [Figure 19] This is an example of an explanatory diagram of a configuration for measuring the inter-vehicle distance obtained from perspective (perspective view). [Figure 20] This is an example of an explanatory diagram of a configuration related to the detection of a person outside the vehicle 10. [Figure 21] This is an example of an explanatory diagram of a configuration related to the detection of a person outside the vehicle 10.

Mode for Carrying Out the Invention

[0010] Hereinafter, modes for carrying out the present invention will be described with reference to the drawings. Note that the following embodiments are one of the embodiments providing the present invention, and the content of the present invention of the present application is not limitedly interpreted based on the following description.

[0011] [First Embodiment] In this embodiment, a dangerous driving determination system that determines the possibility of a dangerous driving act that may cause a risk of traffic danger on the road to other vehicles or the like will be described. As an example of dangerous driving, the determination of the possibility of aggressive driving will be described, but it is not limited thereto. Note that at the time of filing the present application, the following 10 types of obstructive driving that is aggressive driving are defined. 1. Violation of traffic lane 2. Violation of the prohibition of sudden braking 3. Failure to maintain inter-vehicle distance 4. Violation of the prohibition of changing driving route 5. Violation of overtaking 6. Violation of the obligation of dimming etc. 7. Violation of the restriction on using a siren 8. Violation of the obligation of safe driving 9. Minimum speed violation (high-speed motorway national highway) 10.Violation of parking on national highways etc.

[0012] Figure 1 is an example of an explanatory diagram illustrating the installation location of the drive recorder 101 inside the vehicle in this embodiment. Figure 1(a) is a side view of the vehicle, and Figure 1(b) is a top view of the vehicle.

[0013] As shown in Figures 1(a) and 1(b), the drive recorders (DVRs) 101 are installed at the front and rear of the passenger compartment of the vehicle 10. The drive recorder for the front of the vehicle is referred to as the front drive recorder 101F, and the drive recorder for the rear of the vehicle is referred to as the rear drive recorder 101R. Alternatively, the vehicle may be configured to include only the front drive recorder 101F or only the rear drive recorder 101R.

[0014] As shown in Figure 1(b), the front-facing drive recorder 101F is installed, for example, on the passenger side of the left side of the ceiling, and the rear-facing drive recorder 101R is installed in the center of the rear ceiling. Other installation locations are also acceptable. The installation locations of the drive recorders 101 inside the vehicle shown in Figure 1 are displayed on the screens of the management terminal 105, management server 106, user terminal 107, etc., indicating the locations where the drive recorders 101(F, R) are installed, and it is possible to change the settings of each drive recorder 101(F, R) from these screens.

[0015] Next, we will describe the camera inside the drive recorder 101. In this embodiment, we will explain two types as examples using Figures 2 and 3. Figure 2(a) is an example of a camera configuration diagram within the drive recorder 101. This is a top view of the drive recorder 101, with two cameras positioned to the left and right of the front or rear of the vehicle 10.

[0016] The drive recorder 101(F, R) shown in Figure 2(a) has a vehicle camera 102 and a license plate detection camera 103 mounted approximately parallel to the ground at the same height. In this embodiment, the drive recorder 101(F, R) has a vehicle imaging camera 102 and a license plate detection camera 103 housed in the same housing. Therefore, the positional relationship between the vehicle imaging camera 102 and the license plate detection camera 103 is fixed, making it easy for the user to position these cameras. The housing of the drive recorder 101 may be rectangular, cylindrical, or have other shapes. For mounting the drive recorder 101 to the vehicle, known methods or other methods available at the time of filing this application may be used.

[0017] The drive recorder 101 (F, R) may be configured so that the vehicle camera 102 and the license plate detection camera 103 are separable (e.g., detachable), or they may be configured as separate components. In this case, the degree of freedom in the placement position and shooting direction of the vehicle camera 102 and the license plate detection camera 103 is improved. The drive recorder 101 may also obtain a spherical image (hemisphere or full sphere) such as a 360-degree camera.

[0018] The vehicle camera 102 has a wide-angle 130-degree field of view because it needs to capture a wide area in front of or behind the vehicle 10. This configuration is preferable for the vehicle camera 102 in order to record images that capture a wide area in front of, behind, or to the side of the vehicle 10.

[0019] On the other hand, the license plate detection camera 103 is a relatively telephoto camera with a narrow angle of 30 to 40 degrees (for example, 30 degrees) in order to analyze the license plate numbers of vehicles in front of or behind vehicle 10 and obtain those numbers. The license plate detection camera 103 is a camera that obtains license plate numbers as information for identifying the identity of a vehicle, and an angle of view suitable for detecting the target vehicle and its license plate is adopted.

[0020] In this embodiment, the license plate detection camera 103 is used for vehicle detection for forward vehicle collision warning systems (FCWS) or rear vehicle collision warning systems (RCWS). Thus, the camera for the forward vehicle collision warning system or rear vehicle collision warning system can also be used as a camera for determining dangerous driving.

[0021] Of course, the camera used for the forward collision warning system or the rear collision warning system may be different from the camera used to determine dangerous driving. Regarding the angle, a field of view other than the values ​​mentioned above is acceptable, however, the vehicle-photographing camera 102 is a wide-angle camera, while the license plate detection camera 103 has a narrower field of view. By using multiple cameras with different field-of-view angles in this configuration, the accuracy of identifying the vehicle's license plate number and other identifying details is improved compared to using only a wide-angle vehicle camera, allowing for a more accurate determination of the possibility of aggressive driving.

[0022] In the example shown in Figure 2(a), the lenses of the vehicle imaging camera 102 and the license plate detection camera 103 are aligned parallel to each other. However, the lens of the license plate detection camera 103 may be configured to point downwards compared to the lens of the vehicle imaging camera 102, in order to more easily photograph the license plate located near the bumper at the bottom of the vehicle.

[0023] Furthermore, the configuration may allow the direction of the lens (direction of the optical axis) of the license plate detection camera 103 to be changed depending on the installation location of the drive recorder 101 (F, R). For example, the vehicle photography camera 102 has a wide angle and can capture a wide area in front of or behind the vehicle 10, so there is no need to move the direction of the lens, and it can be fixed to the housing of the drive recorder 101 (F, R). On the other hand, the license plate detection camera 103 has a narrow field of view, so it can be configured to change the direction of the lens so that it can reliably target the license plates of vehicles in front of or behind it.

[0024] For example, if the drive recorder 101 (F, R) is installed on the left passenger seat facing the front of the vehicle 10, as shown in Figure 1(b), the optical axis of the lens of the license plate detection camera 103 is changed so that it is directed towards the center of the vehicle 10. This modification, which directs the lens of the license plate detection camera 103 toward the center of the vehicle 10, can be done manually when installing the drive recorder 101(F, R), or the actuators provided by the drive recorder 101(F, R) can change the lens direction in conjunction with the installation position of the drive recorder 101 inside the vehicle as shown in Figure 1.

[0025] Furthermore, a camera direction control unit can be provided to control the orientation of the vehicle number detection camera 103, for example, as follows. 1. It will always follow a position approximately 30m ahead of the center of vehicle 10, either in front of or behind it. 2. The vehicle 10 will always follow the direction of the center of the lane in front of or behind it. 3. Always follow any other vehicles in front of or behind Vehicle 10, and their license plates. The orientation of the lens can be checked on the screen showing the installation position of the drive recorder 101 inside the vehicle, as shown in Figure 1, which is displayed on the user terminal 107, the management terminal 105, and the management server 106. The orientation can also be changed from the user terminal 107, the management terminal 105, and the management server 106.

[0026] The license plate detection camera 103 is positioned so as not to interfere with the field of view of the vehicle imaging camera 102. In other words, the distance between the cameras is sufficiently large so that the end 104E of the license plate detection camera 103 does not enter the field of view of the vehicle imaging camera 102. By positioning the vehicle number camera in a location that does not interfere with the field of view of the vehicle camera, it is possible to prevent the inconvenience of a portion of the image from the wide-angle vehicle camera 102 being lost.

[0027] Figure 2(b) shows an example of a configuration diagram of a drive recorder 101 with two cameras arranged vertically. In the example shown in Figure 2(a), the two cameras were positioned left and right, facing either the front or rear of the vehicle 10. However, in Figure 2(b), these two cameras are positioned vertically. In this case, it is preferable to place the license plate detection camera 103 on the upper side and the vehicle imaging camera 102 on the lower side. In the example shown in Figure 2(b), the distance between the vehicle imaging camera 102 and the license plate detection camera 103 can be narrower than in Figure 2(a). Although the lens of the license plate detection camera 103 is visible within the field of view of the vehicle imaging camera 102, the amount of information in the sky portion of the normally captured video is small, so even if the lens of the license plate detection camera 103 is visible and obstructs the image, it does not pose a significant problem.

[0028] In this configuration, where the two cameras are positioned vertically, the distance between the two lenses can be shortened compared to the case shown in Figure 2(a), making the drive recorder 101(F, R) smaller. Even with a configuration where the two lenses are arranged vertically, as explained in Figure 2(a), it is preferable to direct the vehicle number detection camera 103 towards the lower part of the vehicle in front or behind to facilitate the capture of license plates. The configurations shown in Figures 2(a) and 2(b) may be adopted in drive recorder 101R only, in drive recorder 101F only, or in both.

[0029] Figure 3 shows an example of another configuration diagram of the camera inside the drive recorder 101. In the example shown in Figure 3, instead of arranging the lens tips 104A and 104B of the two cameras on approximately the same plane, the imaging units 104C and 104D of each camera are positioned at the back of the housing. This configuration eliminates the problem of interference caused by the lens of the license plate detection camera 103, and allows the two cameras to be placed even closer together compared to the case shown in Figure 2(a).

[0030] Even with this configuration, it is possible to change the orientation of the lens of the license plate detection camera 103, similar to Figure 2. For example, the tip portion 104A of the lens may be covered with an elastic material, and the lens may be configured to move up, down, left, and right. Conversely, the position of the lens tip 104A may be fixed, and the position of the imaging part 104C of the vehicle number detection camera 103 may move up, down, left, and right, thereby changing the direction of the optical axis (the direction of the lens). With the configuration shown in Figure 3, the dimensions of the drive recorder 101's lens in the optical axis direction become smaller compared to the configuration shown in Figure 2. Furthermore, this is expected to improve the degree of freedom in selecting the installation position and direction of the drive recorder 101.

[0031] Figure 4 shows an example of a connection diagram for a dangerous driving detection system, including, for example, a drive recorder 101. The dangerous driving detection system 1 comprises a drive recorder 101, a management terminal 105, a management server 106, and a user terminal 107, each connected via a network 110. The network 110 can be wired or wireless, and each terminal can send and receive information via the network 110. The network 110 is not limited to the internet and may be a combination of networks using different protocols.

[0032] The drive recorder 101 is a device that is installed in a vehicle, for example, to record video while driving. The drive recorder 101 is an example of an in-vehicle device. However, while the vehicles to which the device is installed in this embodiment are four-wheeled automobiles, it is not limited to four-wheeled automobiles; any vehicle on which a drive recorder 101 can be installed is acceptable. For example, large transport vehicles with four or more wheels such as buses and trucks, two-wheeled vehicles such as motorcycles and bicycles, and other vehicles may be included. For example, the seven types of violations excluding those in 6, 9, and 10 mentioned above can also be considered obstructive driving even if committed by a bicycle. In addition, vehicles such as trains, monorails, and maglev trains may also be included as vehicles on which the device is installed. The drive recorder 101 can transmit various acquired information, including information related to the results of aggressive driving detection, to the management server 106 via the network 110. The management terminal 105 is a terminal that operates and manages the management server 106.

[0033] The management server 106 is a server that stores and manages various types of information, such as recording information, event information, and information related to the results of aggressive driving detection, corresponding to the image information acquired by the drive recorder 101. The various types of information acquired by the drive recorder 101 may be sent directly from the drive recorder 101 to the management server 106, or they may be sent to the management server 106 via a data center.

[0034] The user terminal 107 is a terminal used by, for example, a user driving a car. The user terminal 107 is connected to the drive recorder 101 via an in-vehicle network such as a mobile data communication network or the car's Wi-Fi (registered trademark), and can operate the drive recorder 101. Furthermore, the user terminal 107 is connected to the management server 106 via these networks 110 and can display various information stored in the management server 106.

[0035] Each terminal and management server 106 of the dangerous driving judgment system 1 may be a mobile device such as a smartphone, tablet, mobile phone, or personal digital assistant (PDA), or a wearable device such as glasses, a wristwatch, or clothing. Alternatively, it may be a stationary or portable computer, or a server located in the cloud or on a network. Functionally, it may also be a VR (Virtual Reality) terminal, an AR (Augmented Reality) terminal, or an MR (Mixed Reality) terminal. Alternatively, it may be a combination of multiple such terminals. For example, a combination of one smartphone and one wearable device can logically function as a single terminal. Other types of information processing terminals may also be used.

[0036] Each terminal and management server 106 of the dangerous driving judgment system 1 is equipped with a processor that runs an operating system, applications, and programs, a main memory such as RAM (Random Access Memory), an auxiliary memory such as an IC card, a hard disk drive, an SSD (Solid State Drive), and flash memory, a communication control unit such as a network card, a wireless communication module, and a mobile communication module, input devices such as a touch panel, keyboard, mouse, voice input, and motion detection input from a camera, and output devices such as a monitor or display. The output device may also be a device or terminal that transmits information for output to an external monitor, display, printer, or other device.

[0037] The main memory stores various programs and applications (referred to as modules), and the processor executes these programs and applications to realize each functional element of the overall system. These modules may be implemented in hardware, such as through integration. Furthermore, each module may be an independent program or application, or it may be implemented as a subprogram or function within a single integrated program or application.

[0038] In this specification, each module is described as the entity (subject) that performs the processing; however, in reality, the processor that processes various programs and applications (modules) executes the processing. The auxiliary storage device stores various databases (DBs). A "database" is a functional element (storage unit) that stores a collection of data so that it can be manipulated by the processor or an external computer (e.g., extraction, addition, deletion, overwriting, etc.). The implementation method of a database is not limited; for example, it may be a database management system, spreadsheet software, or text files such as XML or JSON.

[0039] Figure 5 shows an example of a schematic hardware configuration diagram for drive recorder 101 (F, R). The DVR controller 304 is connected to and controls a vehicle camera 102, a license plate detection camera 103, an acceleration sensor 305, an SD card 306, a GNSS sensor 307, and other sensors (not shown). Based on the information received from these elements, the DVR controller 304 generates status information indicating its state. The DVR controller 304 also transmits the generated status information to the microcontroller 301.

[0040] The microcontroller 301 controls the communication processing unit 302. The microcontroller 301 also controls the communication processing unit 302 to transmit status information obtained from the DVR controller 304 to the management server 106. The microcontroller 301 is programmable, and it stores programs for performing the various processes described above. The processing unit of the microcontroller 301 executes these programs to realize the various processes described above.

[0041] The DVR controller 304, microcontroller 301, and communication processing unit 302 can each be implemented as integrated circuits (chips) such as SoCs (System on a Chip), or multiple of these can be implemented together on a single chip. In particular, in this embodiment, the DVR controller 304 and microcontroller 301 work together, so they can be implemented together on a single chip.

[0042] Furthermore, the DVR controller 304 may be configured to have the microcontroller 301 perform some of its functions, or the DVR controller 304 may perform some of the functions of the microcontroller 301. In this embodiment, the DVR controller 304 and the microcontroller 301 together are referred to as the control unit 303. The control unit 303 may be a single chip, or it may consist of multiple chips that separate the functions of the DVR controller 304 and the microcontroller 301.

[0043] Furthermore, by equipping the image sensors of the vehicle camera 102 and the license plate detection camera 103 with an AI processing function for pre-processing, the following outputs can be performed in addition to, or instead of, the generation of normal captured images. The AI ​​processing function has the capability to perform image recognition based on the images captured by these cameras. • Output the target object from the image as metadata. • Outputs images in ISP (Image Signal Processor) output formats, such as YUV or RGB. • Outputs an image with only a specific region extracted.

[0044] By performing preprocessing on the image sensor side in this way, it becomes possible to reduce the amount of data and enable real-time tracking of the target object using high-speed AI processing. In particular, there is an advantage in reducing the amount of data (i.e., communication volume) from the drive recorder 101 to the management server 106.

[0045] The acceleration sensor 305 detects acceleration and generates acceleration information. For example, if a sudden change in acceleration occurs in response to an impact, sudden steering, sudden braking, etc., the control unit 303 detects the occurrence of an event. By analyzing this change in acceleration information, it is possible to infer the circumstances of an accident or other incident that occurred in a vehicle. The SD card 306 is a storage device that records various types of information, such as image information and corresponding recording information. While it is not limited to an SD card, any configuration capable of storing data is acceptable, but a flash memory with high vibration resistance is preferred for in-vehicle use.

[0046] The communication processing unit 302 transmits information from the drive recorder 101 to the management server 106 either directly or via the data center 104 through the wireless communication antenna 308. The GNSS sensor 307 has a position information acquisition unit implemented using an integrated circuit, and acquires position information based on signals received from a GNSS antenna compatible with GNSS (Global Navigation Satellite System), such as GPS (Global Positioning System). The position information acquisition unit can also acquire time information, movement speed information, and movement direction information.

[0047] The drive recorder 101 has a power supply 311 that is connected to the vehicle power supply 310. Power supply 311 receives power from the vehicle power supply 310 (sometimes called receiving power) and supplies it to the DVR controller 304, and also charges the secondary battery 312.

[0048] The secondary battery 312 is a rechargeable battery such as a lithium-ion battery, nickel-cadmium battery, nickel-metal hydride battery, or lead-acid battery. However, a non-rechargeable primary battery is also acceptable. If the accessory power (ACC power) is turned off, or if the power supply from the vehicle power 310 to the power supply 311 of the drive recorder 101 is interrupted due to a malfunction such as a broken wire or disconnection of wiring due to an accident, or if the power supply 311 malfunctions and is unable to supply power, the secondary battery will transmit power (sometimes called power transmission or supply) to the microcontroller 301 or the communication processing unit 302, etc., and maintain at least some of the functions of the drive recorder 101.

[0049] The control unit 303 acquires image information from the vehicle recording camera 102 and the license plate detection camera 103 of the drive recorder 101 (F, R), and determines aggressive driving based on this image information, using the count value when the vehicle enters a predetermined proximity area and the proximity time, which is the time the vehicle is in the predetermined proximity area. Whether or not the vehicle has entered the proximity area can also be determined using the functions of the forward collision warning system (FCWS) or the rear collision warning system (RCWS). The proximity area is the area adjacent to the drive recorder 101 (F, R) or the vehicle 10 equipped with the drive recorder 101 (F, R), but more details will be described later.

[0050] The control unit 303 stores recorded video information and other data in folders divided into multiple levels corresponding to the potential for aggressive driving, according to the level of aggressive driving it has determined, on the storage means, the SD card 306. The recorded video information is image information captured by the camera and stored on the storage means such as the SD card 306. For example, if the level of aggressive driving is high, the control unit 303 stores all recorded information, including the proximity time, in the high-level aggressive driving folder on the SD card 306. If the level of aggressive driving is low, it stores the recording information for the proximity time and a predetermined time before and after the proximity time (e.g., 20 seconds) in the low-level aggressive driving folder on the SD card 306. The level of aggressive driving is determined using a threshold value, which will be described later.

[0051] Furthermore, the control unit 303 of the front drive recorder 101F has a function to receive rear vehicle images, vehicle numbers, etc. from the rear drive recorder 101R using the communication processing unit 302 in order to determine whether the vehicle in front is the same as the vehicle behind.

[0052] Figure 6 shows an example of a schematic configuration diagram of the management server 106. The management server 106 is composed of, for example, a server located on the cloud.

[0053] The main memory 601 stores programs and applications such as the DVR management module 610 and the recording information management module 611. The processor 603 executes these programs and applications to realize each functional element of the management server 106.

[0054] The DVR management module 610 acquires event information from the drive recorder 101(F, R), stores it in the event information 620 of the auxiliary storage device 602, and manages it.

[0055] Furthermore, the DVR management module 610 transmits some or all of the information obtained by decoding the event information 620 to the user terminal 107 or management terminal 105 via push delivery, or in response to requests from the user terminal 107 or management terminal 105.

[0056] The recording information management module 611 acquires the recording information stored in the drive recorder 101(F, R) from the drive recorder 101(F, R), stores it in the recording information 630 of the auxiliary storage device 602, and manages it.

[0057] Furthermore, the recording information management module 611 sends recording information 630 (for example, images of the vehicle behind) to the user terminal 107 or management terminal 105 via push notification or in response to requests from these terminals. The auxiliary storage device 602 stores event information 620, recording information 630, and the like.

[0058] Figure 7 shows an example of a hardware configuration diagram for user terminal 107. The user terminal 107 consists of devices such as smartphones, tablets, laptops, and desktop PCs.

[0059] The main memory 701 stores programs and applications such as the management server linkage module 710 and the DVR linkage module 711, and the processor 703 executes these programs and applications to realize each functional element of the user terminal 107.

[0060] The management server linkage module 710 works in conjunction with the management server 106 to acquire event information 620 and recording information 630 stored in the management server 106, and displays this information on an output device 705 such as a display. Furthermore, the management server integration module 710 can interact with the management server 106 and perform various settings on the management server 106.

[0061] The DVR linkage module 711 operates the drive recorder 101 (F, R) and configures the drive recorder 101 (F, R) when the drive recorder 101 and the user terminal 107 are connected via an in-vehicle network such as Wi-Fi (registered trademark).

[0062] Furthermore, the management terminal 105 can have the same configuration as the user terminal 107, and the DVR linkage module and management server linkage module stored in the main memory can be used to operate the drive recorder 101 (F, R) and the management server 106, as well as configure their settings.

[0063] Figure 8 is an example of a block diagram of the control unit 303. As shown in Figure 8, the control unit 303 includes an image processing unit 321, a counting unit 322, a timing unit 324, a determination unit 326, an information input / output unit 328, an output unit 330, and so on. The SD card 306 stores folders such as a low-teasing level folder 342, a medium-teasing level folder 344, and a high-teasing level folder 346.

[0064] The image processing unit 321 analyzes and processes the image information acquired by the license plate detection camera 103 (R, F) to extract vehicle images of vehicles located in front of or behind vehicle 10. The license plate detection camera 103 constantly acquires moving images of the front or rear of the vehicle 10, and the vehicle image is extracted from this image information. Furthermore, the system may be configured to extract vehicle images when another vehicle enters the immediate vicinity in front of or behind vehicle 10.

[0065] The proximity area is the area close to the drive recorder 101 (F, R) or the vehicle 10 on which it is installed. More specifically, the proximity area is defined as the area within a predetermined distance range from the drive recorder 101 (F, R) or the vehicle 10 on which it is installed. The license plate detection camera 103 can detect the license plate of a target vehicle when the distance to the target vehicle is approximately 30m or less, or 20m to 25m, which is within a detectable distance.

[0066] Alternatively, the proximity zone may be defined as a distance less than or equal to the detectable range of the vehicle number. The proximity zone is defined, for example, when the distance to vehicle 10 is 30m or less, preferably 25m or less. The proximity zone may be any other distance, and the distance to the front and rear of vehicle 10 may be different. The distance defining the proximity zone may be set by the user. The control unit 303 determines that another vehicle has entered the proximity zone by analyzing the captured image or by measurement using other distance measuring sensors.

[0067] If another vehicle is present in the vicinity, the image processing unit 321 extracts an image of the license plate portion from the image information captured by the license plate detection camera 103, and performs character recognition on the image to extract the license plate number (vehicle registration number or vehicle number) of the nearby vehicle.

[0068] The counting unit 322 measures the number of times another vehicle enters a predetermined proximity area (proximity count) using a counter (not shown). More specifically, the count is measured by detecting when the target vehicle enters the proximity area based on image information processed by the image processing unit 321. This configuration eliminates the need for special sensors, allowing for detection of intrusion into the adjacent area solely through image processing by the camera. However, other sensors may be used in parallel with or in conjunction with image processing to detect intrusion into the adjacent area.

[0069] The timing unit 324 measures the proximity time, which is the time spent in the proximity area, using a timer (not shown) each time it counts that another vehicle has entered the proximity area. The determination unit 326 calculates a tailgating index (tailgating point) P based on the number of times other vehicles have entered the proximity zone and the average time spent in the proximity zone (average proximity time), and determines (also called determining) the tailgating level based on the degree of this tailgating index P. This tailgating index P and tailgating level will be described later using a flowchart.

[0070] If the harassment level (degree of harassment index P) is high, the information input / output unit 328 stores all captured images of other vehicles that have entered the proximity area, including the time they were in close proximity, in the high harassment level folder 346 on the SD card 306. Information stored in the high harassment level folder is not overwritten. On the other hand, if the level of provocation is low, the image information for the proximity time and a predetermined time before and after the proximity time (for example, 20 seconds) is stored in the low-provocation level folder 342. Furthermore, the low-provocation level folder 342 is overwritten based on a certain cycle, a certain period of time, etc.

[0071] The information input / output unit 328 stores image information for the proximity time and a predetermined time before and after the proximity time (for example, 20 seconds) in the "Medium" folder 344 when the distortion level is set to "Medium". This stored data may also be overwritten.

[0072] The control unit 303 retrieves the recorded information stored in the high-trolling-level folder 346, the medium-trolling-level folder 344, and the low-trolling-level folder 342 via the information input / output unit 328, and displays or outputs it externally. The output unit 330 can also generate a warning sound or voice from the speaker 309 if the trolling level is high. The alarm output is not limited to sound or voice output; the output unit 330 may generate light from a light-emitting diode or other light-emitting device in place of or in addition to sound or voice, or it may output the alarm in a manner perceptible to a user inside the vehicle 10. The alarm output may also be directed to a person outside the vehicle 10.

[0073] Next, we will explain the details of how to determine aggressive driving using the flowcharts in Figures 9 and 10. Figure 9 shows an example of a processing flow for detecting aggressive driving. The control unit 303 performs the following processing by having the above-described configuration of each part. This processing can be activated, for example, when entering a highway, but it may also be activated when traveling at a speed of approximately 20 km / h to 50 km / h. In other words, aggressive driving can be detected even on ordinary roads.

[0074] The control unit 303 starts acquiring image information (including date, time, camera identification number, etc.) from the vehicle cameras 102 (R, F) (S10). Next, the control unit 303 measures driving information such as the speed and acceleration of the vehicle 10 using the acceleration sensor 305 (S12), and stores this speed and acceleration information (S14). The acceleration information is the longitudinal component of the vehicle 10's direction of travel, obtained from the acceleration sensor 305, but it is also possible to use a composite value of the three axes other than gravitational acceleration.

[0075] More specifically, the control unit 303 combines the acquired speed and acceleration data with the information acquired from various sensors of the drive recorder 101 (F, R) to generate recording information. The control unit 303 associates the recorded video information with the image information acquired from the camera and stores it in the SD card 306 (storage unit). For example, the control unit 303 stores the date, time, camera identification number, type (front or rear), and driving information (speed, acceleration, position coordinates, etc.) in the storage means in association with each other.

[0076] The control unit 303 analyzes the acquired image information and performs a vehicle determination process to determine whether a vehicle is present in front of or behind the vehicle 10 (S16). This vehicle determination process will be described later. The control unit 303 determines, based on the acquired image information, whether there is a vehicle (nearby vehicle) approaching the front or rear of the vehicle 10 (S18).

[0077] The determination of whether a vehicle is in close proximity is based on, but not limited to, detection by the Forward Collision Warning System (FCWS) or the Rear Collision Warning System (RCWS). For example, if the size of a vehicle in an image captured by the license plate detection camera 103 or the vehicle imaging camera 102 exceeds a predetermined size, a proximity determination can be made, for example, if the vehicle is within 30 meters. Alternatively, for example, proximity detection can be performed by measuring distance using sensors that utilize radio waves or sound waves, rather than through image processing.

[0078] Furthermore, the proximity range used for detecting aggressive driving can be made variable. For example, the proximity range can be changed according to the speed of your own vehicle (10). For example, if vehicle 10 is traveling at a high speed, the distance between vehicles increases, so the proximity range can be set to be wider. Conversely, if vehicle 10 is traveling at a low speed, the distance between vehicles decreases, so the proximity range can be set to be narrower.

[0079] In step S18, if the determination unit 326 determines that there are no vehicles in front or behind that have entered the proximity area, the process returns to image acquisition and driving information acquisition (S18 is No). In step S18, if the determination unit 326 determines that there is a vehicle in front or behind that has entered the proximity area, the counting unit measures the number of proximity detections (proximity count) (S20), and the timing unit measures the time the vehicle is in proximity (proximity time) (S22).

[0080] The control unit 303 stores the measured information as count information. The count information includes the date, time, number of proximitys, proximity time, and vehicle number information of the nearby vehicle extracted from the image information. To reduce memory capacity, video information captured by the license plate detection camera 103 is not stored, and only the license plate information of nearby vehicles extracted is stored. However, if there is sufficient memory capacity, video information captured by the license plate detection camera 103 may also be stored.

[0081] The determination unit 326 performs an aggressive driving determination process to determine the aggressive driving index P based on the number of times the vehicle is in close proximity, the time the vehicle is in close proximity, acceleration (and / or speed), etc. (S24). This aggressive driving determination process will be described later. The control unit 303 executes processing according to the determined level of aggression and changes the method of storing the acquired video information (S26). The control unit 303 determines whether there is a recording termination instruction. If it is not a recording termination instruction, it returns to step S12; if it is a recording termination instruction, it terminates the process (S28). For example, if the car stops and the engine is turned off, the control unit 303 determines that recording has ended and terminates the process.

[0082] Figure 10 shows an example of a vehicle detection processing flow. Let's use the rear-facing license plate detection camera 103R as an example. The image processing unit 321 identifies the target vehicle for aggressive driving detection from the image information captured by the license plate detection camera 103R (S31). In this case, if multiple vehicles are shown in the image information, it is preferable for the image processing unit 321 to identify the target vehicle as the other vehicle in the same lane that is closest to the vehicle 10. Even when multiple vehicles are visible in an image, the accuracy of the aggressive driving detection can be improved by configuring the system to determine the possibility of aggressive driving only for the extracted target vehicle. Furthermore, by configuring the system to only determine the possibility of aggressive driving for target vehicles in the same lane as the vehicle in question, the accuracy of the aggressive driving detection can be improved.

[0083] When the license plate detection camera 103 captures an image at a distance of approximately 30m, where proximity detection is performed using a 30-degree field of view, about three vehicles will be captured in the image, as shown in Figure 11. The image processing unit 321 identifies the vehicle to be targeted for aggressive driving detection from among these three vehicles.

[0084] There are various methods for identifying the target vehicle, but for example, the image processing unit 321 can identify the largest vehicle among multiple vehicles visible in the image as the target vehicle. In the example in Figure 11, vehicles 12, 13, and 15 are visible in the same image frame, but vehicle 15, which is the largest among them, is identified as the target vehicle.

[0085] Alternatively, the target vehicle may be identified as a vehicle traveling behind the current vehicle 10 in the same lane. In this case, the captured video footage is analyzed to extract the lanes (white lines, dotted lines, median lines, etc.) to the left and right of vehicle 10, and the vehicle behind within these lanes is identified as the target vehicle.

[0086] Assuming that vehicle 15 was not present in the example in Figure 11, vehicles 11, 12, and 13 are visible in the same image frame. The image processing unit 321 traces the left and right lanes 20 and 21 of its own vehicle 10 backward through image analysis and identifies vehicle 11, which is located in the same lane, as the target vehicle. For example, if the lane (road) in which your vehicle is traveling is curved, the car directly behind your vehicle 10 may be in the adjacent lane. Therefore, it is effective to trace the lanes to the left and right of your vehicle 10 and identify the target vehicle that is in the same lane as your vehicle 10.

[0087] Alternatively, a gyro sensor (not shown) mounted on the drive recorder 101 is used to acquire angular velocity information or attitude information of the vehicle 10. The image processing unit 321 then uses the acquired angular velocity information or attitude information to correct the position in which the target vehicle is identified within the image information. For example, if the vehicle is turning to the left, the correction is made based on its angular velocity, and the vehicle located on the right side of the image frame is identified as the target vehicle. In this way, when the lane is curved, the system can improve the accuracy of its aggressive driving detection by determining the possibility of aggressive driving only for vehicles located behind or in front of the vehicle along the lane, rather than directly behind or in front of the vehicle itself.

[0088] Alternatively, several of the above examples can be combined to identify the target vehicle from multiple vehicle images within the image frame. Furthermore, in the example described above, the process of identifying the target vehicle was described as being performed by the license plate detection camera 103R located behind the vehicle 10, but it is also possible to configure the system so that the vehicle photography camera 102R is responsible for identifying the target vehicle.

[0089] Furthermore, for other vehicles located in front of vehicle 10, the vehicle-capturing camera 102F or the license plate detection camera 103F of the drive recorder 101F can perform the same processing as described above to identify the target vehicle. Furthermore, in the example described above, the license plate detection camera 103 was described as being fixed to the drive recorder 101R. However, for drive recorders 101R with a license plate detection camera 103 whose lens orientation can be changed, the control unit 303 may change the orientation of the lens of the license plate detection camera 103 to face the target vehicle, so that the lens follows the target vehicle.

[0090] Next, the image processing unit 321 extracts the portion of the target vehicle from the camera's image information (S33). Furthermore, the image processing unit 321 extracts the license plate area from the image captured by the vehicle license plate detection camera 103, and the control unit 303 performs character recognition on the image to extract the license plate number (vehicle registration number or vehicle number) of the nearby vehicle (S35). The control unit 303 stores the extracted vehicle number in the SD card 306, associating it with the image information (S37).

[0091] Next, we will explain the details of the aggressive driving detection process (S24) shown in Figure 9. The aggressive driving detection process calculates an aggressive driving index P, which indicates the degree of likelihood of aggressive driving, based on acceleration information from the acceleration sensor 305 and changes in position information acquired by the GNSS sensor 307, or based on speed information acquired from the vehicle, the number of detected nearby vehicles, and the average detection time. The index Pf indicates the likelihood of aggressive driving by a vehicle approaching from the front of vehicle 10, calculated by the front drive recorder 101F, and the index Pr indicates the likelihood of aggressive driving by a vehicle approaching from behind vehicle 10, calculated by the rear drive recorder 101R.

[0092] The determination unit 326 calculates the tailgating index Pr by the rear vehicle, the tailgating index Pf by the front vehicle, and the overall tailgating index P when the rear vehicle and the front vehicle are the same, for example, using the following formula. Pr = avnrptrq (Equation 1) Pf = avnfptfq (Equation 2) P = Pr + Pf (Equation 3) a is the vehicle's acceleration coefficient. v is the vehicle speed coefficient. nr is the number of times a vehicle behind has been detected to have entered the proximity zone. NF is the number of times the vehicle in front has been detected to have entered the proximity zone. tr is the average detection time when the vehicle behind was in the proximity zone. tf is the average detection time when a vehicle ahead was in the proximity zone. p is a power exponent (e.g., 1.5) q is an exponent (for example, 1.3)

[0093] The determination unit 326 sets a coefficient based on acceleration information from the acceleration sensor 305 (hereinafter referred to as acceleration coefficient a), and also sets a coefficient based on velocity information (hereinafter referred to as velocity coefficient v). The acceleration coefficient a is set as follows, for example. In relation to the rear-end pressure index Pr: If the vehicle's acceleration is -0.05G, then a=1. If the vehicle's acceleration is ≤ -0.05G, then a = 0 Let's assume that. For the forward pressure index Pf: If the vehicle's acceleration is <0.05G, then a=1. If the vehicle's acceleration is 0.05G or less, then a = 0 Let's assume that.

[0094] When the vehicle's acceleration is -0.05G or less relative to the tailgating index Pr from behind, it means that the vehicle 10 is decelerating rapidly, and in such a case, the distance to the vehicle behind may decrease and the vehicle may get too close. In order to prevent such rapid deceleration of the vehicle 10 from being judged as tailgating, the coefficient is set to zero when the vehicle's acceleration is -0.05G or less. In this case, since the coefficient is multiplied by zero, the tailgating coefficient calculated in Equation 1 becomes zero, and it is not detected as tailgating. Note that even during normal driving, noise of approximately ±0.03G is generated. A value of -0.05G represents an acceleration equivalent to lightly applying the brakes, and is low enough to detect the effect of braking, even if the noise is not significantly affected. In the case of sudden braking, the acceleration will be approximately -0.2G. For example, the threshold could be set to a value of around -0.1G, so that the system only avoids detecting aggressive braking during sudden deceleration when the brakes are applied more firmly.

[0095] When the vehicle's acceleration is 0.05G or higher relative to the forward tailgating index Pf, it means that the vehicle 10 is accelerating rapidly, and in such a case, the distance to the vehicle in front may decrease and become close. In order to not classify such rapid acceleration of the vehicle 10 as tailgating (to determine that there is no possibility of tailgating), the coefficient is set to zero when the vehicle's acceleration is 0.05G or higher. In this case, since the coefficient is multiplied by zero, the tailgating coefficient calculated in Equation 2 becomes zero, and it is not detected as tailgating.

[0096] For example, the threshold could be set to a value of around 0.1G, and the system could be configured so that the "revving" detection is only performed when the accelerator is pressed more firmly during sudden acceleration. As described above, by configuring the system to determine that there is no possibility of aggressive driving when the vehicle's acceleration meets predetermined conditions, it is possible to prevent the system from mistakenly determining that a vehicle approaching from a vehicle that changes in acceleration due to sudden braking or acceleration is aggressive driving.

[0097] Furthermore, when vehicle 10 is on an uphill or downhill slope, the acceleration generated when the accelerator or brake is pressed will be different from when driving on a flat road. When driving on a slope in this manner, the determination unit 326 corrects the acceleration information obtained from the acceleration sensor based on the information obtained from the gyro sensor, and then makes a determination of aggressive driving to cancel out the effect of the slope. By configuring it in this way, for example, when driving uphill, if the acceleration differs from that during normal driving, the system can correct that value before determining whether aggressive driving has occurred, thereby improving the accuracy of the determination.

[0098] The velocity coefficient v is set as follows, for example. If your vehicle speed is less than 20 km / h, then v=1 If the vehicle speed is ≤ 20 km / h, then v = 0 Let's assume that. For example, during low-speed driving such as in traffic jams, the distance between your vehicle and the vehicle in front or behind will decrease. To prevent such situations from being classified as aggressive driving, the coefficient is set to zero when your vehicle speed is 20 km / h or less. In other words, when driving at 20 km / h or less, the coefficient is multiplied by zero, so the aggressive driving coefficient calculated by Equation 1 or Equation 2 becomes zero, and it is not detected as aggressive driving.

[0099] The 20 km / h threshold can be changed. For example, the threshold could be set to 60 km / h or higher, and the system could be configured to only detect aggressive driving when driving on a highway. By configuring the system so that it does not detect aggressive driving (determines that there is no possibility of aggressive driving) when the vehicle's speed is below a predetermined speed, it is possible to prevent the system from mistakenly detecting aggressive driving when, for example, a vehicle is approaching from in front of or behind during low-speed driving such as in traffic jams.

[0100] The vehicle's speed can be obtained by methods such as calculating it from changes in position information from the GNSS sensor 307, obtaining it from the speed information of the vehicle 10, calculating it by analyzing the flow of scenery around the vehicle 10 through image analysis, or calculating it by analyzing the movement of other vehicles visible in relation to the vehicle 10 through image analysis. Furthermore, whether vehicle 10 is in motion can be determined using the same multiple calculation and acquisition methods as described above, and it can also be determined by analyzing the values ​​of the acceleration sensor and gyroscope sensor.

[0101] p and q are power exponents of the number of detections (proximity count) and the average detection time (average proximity time) spent in the proximity area, respectively. By treating the change in the trolling index, which is derived from the number of proximitys and the mean proximity time, as an exponential function rather than a linear one, the likelihood of it being identified as trolling increases with a higher number of proximitys, and the likelihood of it being identified as trolling also increases with a longer mean proximity time.

[0102] Furthermore, the exponent for the number of proximitys is set to, for example, 1.5, and the exponent for the mean proximity time is set to, for example, 1.3, so that in the detection of harassment, the influence of the number of proximitys is valued more highly than the influence of the mean proximity time. However, the exponents may also be adjusted to value the mean proximity time more highly. Furthermore, the thresholds for the number of close encounters and the average proximity time can be changed according to the speed of the vehicle 10. For example, if the vehicle 10 is traveling at a high speed, changes in relative speed with other vehicles become less frequent, and movement in and out of the close-range area becomes less frequent, so the threshold for the average proximity time can be set to be longer. Conversely, if the vehicle is traveling at a low speed, the threshold for the average proximity time can be set to be shorter.

[0103] Furthermore, the threshold for the number of close encounters can be changed. For example, if your vehicle 10 is traveling at a high speed, changes in relative speed with other vehicles become less frequent, and it becomes less likely for vehicles to enter or exit the close encounter range, so the threshold for the number of close encounters can be set lower. Conversely, if your vehicle is traveling at a low speed, the threshold for the number of close encounters can be set higher. The above coefficients and threshold values ​​can be changed from the drive recorder 101, management terminal 105, management server 106, user terminal 107, etc.

[0104] If vehicle 10 is being tailgated, it is possible that the same vehicle may tailgate from behind and then move in front of it and tailgate from the front. If the determination unit 326 determines that the rear vehicle and the front vehicle are the same vehicle, it calculates an overall tailgating index P by adding the tailgating index Pr from the rear vehicle and the tailgating index Pf from the front vehicle, as shown in equation 3 above. If the vehicle is not being tailgated from both the front and the rear by the same vehicle, the tailgating index Pr from the rear vehicle or the tailgating index Pf from the front vehicle will be used as the final tailgating index P. By configuring the system to aggregate repeated instances of aggressive driving by the same vehicle in front of or behind another vehicle, it becomes possible to perform more accurate aggressive driving detection.

[0105] In this embodiment, whether the proximity action was performed by the same vehicle is confirmed by analyzing the images captured by the license plate detection camera 103 and checking whether the license plate information extracted matches. However, this is not the only method; any other method of verification is acceptable as long as it can be confirmed that the vehicles are the same, or that there is a high probability that they are the same. For example, by analyzing the images, it may be confirmed that there is a high probability that the vehicles are the same if any or any combination of the following elements match: vehicle shape, vehicle size, vehicle height, vehicle width, vehicle color, vehicle type, etc.

[0106] The front drive recorder 101F and the rear drive recorder 101R are connected directly via a wired or wireless in-vehicle network, or via a communication network through the management server 106. The control unit 303 of either drive recorder 101 acts as the master, acquiring aggressive driving index information from the other drive recorder 101, summing them up, and calculating an overall aggressive driving index P.

[0107] For each vehicle targeted for aggressive driving detection, the four-digit vehicle number, the number of proximity counts by the counting unit 322, the proximity time for each proximity detection, and the vehicle's acceleration information and vehicle speed information at the time of proximity detection are stored in the SD card 306 or the control unit's temporary storage area for about 30 minutes. The determination unit 326 reads this information, calculates the average proximity time, and calculates the aggressive driving index P.

[0108] Furthermore, since activating the recognition algorithm after detecting the proximity of a vehicle will cause processing delays, it is better to recognize the vehicle's license plate number when the proximity of a vehicle is detected. The processing order is license plate recognition followed by vehicle detection, but the license plate number only needs to be memorized at the time the vehicle is detected.

[0109] In this embodiment, a dangerous driving determination system was described in which the control unit determines the possibility of dangerous driving by the target vehicle based on the number of times the target vehicle identified based on the image information entered a predetermined proximity area. By configuring it in this way, it is possible to provide a system that can determine whether the vehicle being photographed is likely to be engaging in dangerous driving.

[0110] Furthermore, in this embodiment, a dangerous driving determination system was described in which the control unit determines the possibility of dangerous driving by the target vehicle based on the number of times the target vehicle identified based on the image information entered a predetermined proximity area and the proximity time, which is the time the target vehicle is in the predetermined proximity area. By configuring it in this way, it is possible to provide a system that more accurately determines whether the photographed target vehicle is likely to be engaging in dangerous driving based on two pieces of information: the number and duration of close-range actions.

[0111] Thus, the inventors have found that the number of times a target vehicle outside vehicle 10 enters a predetermined proximity area and the proximity time spent in the proximity area serve as indicators of the possibility that the target vehicle is engaging in dangerous driving. In this embodiment, based on this finding, the configuration for determining the possibility of dangerous driving as described above is adopted. In this embodiment, the detection of aggressive behavior is performed using both the number of proximitys and the proximity time, but it is also possible to configure the system to use only one of these two factors for the detection of aggressive behavior. Furthermore, while we used the mean proximity time for the distance between devices, a configuration using, for example, total proximity time, would also be acceptable.

[0112] Figure 12 is an example of an explanatory diagram for the sensationalism index determination table. Figure 12 shows the number of times the vehicle behind was detected in close proximity (n) and the tailgating index P, which is calculated from the average of the proximity detection times (average proximity time t (seconds)). The judgment threshold is as follows: if the provocation index P is less than 30, there is no level of provocation; if it is between 30 and 50, the level of provocation is low; if it is between 50 and 100, the level of provocation is medium; and if it is 100 or more, the level of provocation is high. The lightest gray values ​​represent the portion where the agitation index P exceeds the low agitation level. The second lightest gray values ​​represent the portion where the agitation index P exceeds the medium agitation level. The darkest gray values ​​represent the portion where the agitation index P exceeds the high agitation level.

[0113] Figure 13 is an example of an explanatory diagram for the hype index judgment graph. In Figure 13, the vertical axis shows the agitation index P, and the horizontal axis shows the number of detections n. The time spent within the proximity range is 1 second, 4 seconds, 6 seconds, 11 seconds, 17 seconds, and 20 seconds, respectively, as shown in the graph. The determination unit 326 determines each agitation level based on thresholds (high agitation level is 100 or more, medium agitation level is 50 or more, and low agitation level is 30 or more).

[0114] For example, if a vehicle enters the proximity zone a total of three times, and the average proximity time exceeds 11 seconds, the determination unit 326 determines that the aggressive driving index P exceeds the threshold of 100 and that the level of aggressive driving is high. As shown in the graph in Figure 13, even if the average proximity time is long, the system is set so that the "high level of aggressive driving" is not triggered if the number of proximity occurrences is only one. In other words, the system is set so that the "high level of aggressive driving" is triggered only when the number of proximity occurrences is two or more. By configuring the system in this way, it is possible to prevent a single instance of close proximity, which does not have a high probability of being aggressive driving, from being mistakenly classified as high-level aggressive driving.

[0115] Furthermore, the average proximity time is capped at 20 seconds. This is because aggressive driving typically involves repeatedly approaching and then moving away from the vehicle within the proximity range, but proximity exceeding 20 seconds is likely to be due to traffic congestion or other situations where the distance between vehicles is narrow, and is therefore unlikely to be aggressive driving. The control unit 303 executes processing based on the level of instability determined by the determination unit 326, classifies the acquired video information, and stores it in the SD folder, which is the storage means.

[0116] Figure 14 shows an example of a processing flow based on the level of provocation. In this flow, the control unit 303 changes the post-determination processing according to the level of aggressive driving. Specifically, depending on the determined level of aggressive driving, it creates different event folders for high, medium, and low levels of aggressive driving and executes different event recordings. By configuring it in this way, it is possible to allocate appropriate actions according to the level of aggressive driving.

[0117] The control unit 303 checks whether aggressive driving has been determined based on the aggressiveness level of the determination unit 326 (S50). If the control unit 303 does not determine that aggressive driving occurred ("no aggressive driving level"), it terminates the process (S50 is No). If aggressive driving is determined to have occurred, the control unit 303 executes folder sorting processing according to the level of aggressive driving (S50 is Yes).

[0118] If the "aggressive driving level is high" (S52 is Yes), the control unit 303 stores the image information captured by the vehicle camera 102 in the "aggressive driving level high" folder. If the image information is already temporarily stored in another area of ​​the SD card 306 or in the memory of the control unit 303, it is stored by moving or copying it to the "aggressive driving level high" folder. In addition, a warning sound indicating the possibility of aggressive driving is output from the speaker 309 (S54).

[0119] If the "trolling level is medium" (S56 is Yes), the control unit 303 stores the image information captured by the vehicle camera 102 in the "trolling level medium" folder. If the image information is already temporarily stored in another area of ​​the SD card 306 or in the memory of the control unit 303, it is stored by moving or copying it to the "trolling level medium" folder (S58).

[0120] If the "trolling level is low" (S60 is Yes), the control unit 303 stores the image information captured by the vehicle camera 102 in the "trolling level low" folder. If the image information is already temporarily stored in another area of ​​the SD card 306 or in the memory of the control unit 303, it is stored by moving or copying it to the "trolling level low" folder (S62).

[0121] The folders categorized as high, medium, and low in terms of provocation level each have different data storage policies, with the high-provocation level folders having a policy to store more important data for longer periods. For example, in the case of a high level of aggressive driving, all image information from the time proximity detection begins until the vehicle finally approaches and moves away is stored in a high-level aggressive driving folder. This image information stored in the high-level aggressive driving folder is not overwritten and is saved for a long period of time.

[0122] If the proximity detection level is moderate, the images stored in the "Moderate Proximity Detection" folder will contain, for example, 20 seconds before and after the last proximity detection time. The image information stored in the "Moderate Proximity Detection" folder will not be overwritten and will be saved long-term. In cases of low-level harassment, the image information stored in the "low-level harassment" folder is, for example, 20 seconds before and after the last proximity detection time. This image information is then stored for, for example, 3 months, after which it is overwritten.

[0123] This configuration allows for a longer storage period for recorded images of vehicles likely to be engaging in aggressive driving, enabling longer-term analysis and display. Simultaneously, recorded images of vehicles less likely to be engaging in aggressive driving can be overwritten and deleted after a certain period following the completion of analysis, thus reducing storage capacity. The control unit 303 processes the following: if it determines that the level of aggressive driving is high, it stores new recording information in the high-level aggressive driving folder, separate from the recording information already stored therein. If it determines that the level of aggressive driving is low, it deletes the recording information already stored in the low-level aggressive driving folder and overwrites it with new recording information.

[0124] In addition to the examples above, it is also possible to increase the memory period in the order of low, medium, and high for each type of "trolling" folder. For example, the storage periods could be set to 3 months, 6 months, and permanently, respectively. This configuration allows for a longer storage period for recorded images of vehicles likely to be engaging in aggressive driving, enabling longer-term analysis and display. Simultaneously, recorded images of vehicles less likely to be engaging in aggressive driving can be deleted after a certain period following analysis, reducing storage capacity.

[0125] This configuration allows for the sequential saving of newly recorded images of vehicles likely to be engaging in aggressive driving, enabling their use for analysis and display over a longer period. Simultaneously, recorded images of vehicles less likely to be engaging in aggressive driving can be deleted after a certain period has elapsed since the analysis was completed, thereby reducing storage capacity.

[0126] Furthermore, it is possible to increase the recording period of saved video information in the order of Low, Medium, and High for the "Trolling Folder." For example, in the order of Low, Medium, and High for the "Trolling Folder," it is possible to store video information for the period of one last proximity detection, video information for the three past proximity detection periods since the last proximity detection, and video information for all of multiple tilting detection periods.

[0127] Furthermore, in the above example, the system is configured to output a warning sound only when the aggressive driving level is high. However, the control unit 303 can also control the output unit 330 according to the determined aggressive driving level and change the warning sound or voice output from the speaker 309. For example, in cases of high-level aggressive driving, a more urgent warning sound or voice message can be emitted compared to cases of low-level aggressive driving to alert the driver.

[0128] The control unit 303 stores information on the location and date / time when the disturbance occurred, along with the disturbance detection level, in the SD card 206, and also transmits this information to the management terminal 105, management server 106, user terminal 107, etc. Furthermore, the drive recorder 101 in this implementation is equipped with a GNSS sensor 307, and when aggressive driving is detected and the level of aggressive driving is determined to be high, medium, or low, the location where the aggressive driving was detected can be displayed on a map shown on the management terminal 105, management server 106, and user terminal 107.

[0129] Furthermore, to show the level of harassment at each location, for example, locations with a high level of harassment can be displayed in red, locations with a medium level of harassment in yellow, and locations with a low level of harassment in blue. Furthermore, if the system accepts the selection to display the location in question, it can play back recorded information that has been associated with the possibility of aggressive driving. In this embodiment, recorded information is stored in different memory areas depending on the level of aggressive driving. This configuration allows for the allocation of recorded information to different memory areas according to the level of aggressive driving, making management and display easier.

[0130] Furthermore, if the camera's image sensor has an AI processing function for pre-processing, the image processing unit 321 may also send data indicating the results of image recognition based on the video captured by the camera to the management server 106.

[0131] Furthermore, the front-facing dashcam 101F can be equipped with a button to determine whether the driver is actually being subjected to aggressive driving and to confirm whether the aggressive driving determination is correct. If aggressive driving is detected, a warning sound can be emitted as described above. When a warning sound is emitted, the control unit 303 accepts input from the driver via a button to indicate whether the aggressive driving detection was correct.

[0132] In this way, by receiving confirmation from the driver each time an aggressive driving detection is made, training data of correctly detected aggressive driving detections can be accumulated, allowing the AI ​​to learn and improve the accuracy of aggressive driving detection. By using the number of proximitys, mean proximity time, and correctly determined trolling judgment results as training data, a trolling judgment model is generated using machine learning, with the number of proximitys and mean proximity time as input and the trolling judgment result as output. By inputting the number of proximitys and mean proximity time into this judgment model and training it, it becomes possible to output trolling judgment results.

[0133] Furthermore, by adjusting the thresholds for the number of close encounters and average proximity time used in correctly identified instances of aggressive driving using statistical methods or AI, it becomes possible to perform more accurate detection of aggressive driving.

[0134] Furthermore, aggressive driving can be detected through the cooperation of the front and rear drive recorders 101. For example, if drive recorder 101R detects the proximity of a vehicle behind, even though drive recorder 101F has detected no vehicle in front, it can be determined that aggressive driving has occurred, the likelihood of aggressive driving can be increased, or the value of the weighting coefficient multiplied by Equation 1 or Equation 2 can be increased. Similarly, if drive recorder 101F detects the proximity of a vehicle in front, even though drive recorder 101R does not detect a vehicle behind it, it is possible to determine that it is aggressive driving, increase the likelihood of aggressive driving, or increase the value of the weighting coefficient applied to Equation 1 or Equation 2.

[0135] In the examples shown in Figures 2 and 3, a configuration was described in which a vehicle camera 102 for recording video and a license plate detection camera 103 for detecting aggressive driving are housed within the same drive recorder 101 housing. However, these cameras can also be configured to be separate and removable. Alternatively, they can be implemented in separate housings from the outset. In this case, the cameras are connected to each other via a wired or wireless network.

[0136] Alternatively, one housing may be equipped with a large-capacity storage device for recording video, such as an SD card 306, while the other housing contains only the camera. The video information captured by the camera-only housing is then transmitted via the network to the housing equipped with the storage device and stored there. In this configuration, which allows the two cameras to be separated, the vehicle camera 102 for recording and the license plate detection camera 103 for detecting aggressive driving can be mounted in optimal locations, improving the accuracy of aggressive driving detection and making it easier to adjust the recording range.

[0137] Furthermore, if the vehicle camera 102 for recording video and the license plate detection camera 103 for detecting aggressive driving are integrated and can be separated, integrating them has the effect of reducing the installation location of the drive recorder 101 to one location, thus limiting the obstruction of the forward or rear view to one location. In this case, it is particularly desirable to conceal the wiring between the two.

[0138] [Second Embodiment] In the embodiment described above, the drive recorder 101 was equipped with a vehicle imaging camera 102 and a license plate detection camera 103. In this embodiment, the drive recorder 101 switches the cameras used for distance measurement, vehicle detection, and license plate recognition processing depending on the distance to the target vehicle. The distance to the target vehicle may be the distance from vehicle 10 to the target vehicle, and this point is the same as in the first embodiment described above. In this embodiment, the drive recorder 101 will be described on the premise that it is equipped with two cameras, a first camera 102C and a second camera 103C, in the same housing, as shown in Figure 15. The field of view of the first camera 102C is the same as that of the vehicle imaging camera 102, which is 130 degrees. The field of view of the second camera 103C is the same as that of the license plate detection camera 103, which is 30 degrees. The external configuration and positional relationship of the first camera 102C and the second camera 103C may be the same as the positional relationship between the vehicle imaging camera 102 and the vehicle number detection camera 103 described in Figure 2(A), and the first camera 102C and the second camera 103C may be installed at the same height as the ground and approximately parallel to each other. However, the first camera 102C and the second camera 103C may be configured to be separable (for example, detachable) or as separate components, and this point is the same as in the first embodiment described above.

[0139] <2-1> First example of operation For the first operation example, it will be described with reference to FIG. 16. The control unit 303 of the drive recorder 101 switches a camera used for at least one of measurement of the distance to the target vehicle (for example, the vehicle directly in front) or the rear vehicle (for example, the vehicle directly behind) as the target vehicle, according to the distance to the target vehicle. In the present embodiment, the distance L to the target vehicle may be the distance measured based on the image information acquired by one or both of the first camera 102C and the second camera 103C.

[0140] Assume that the value of the distance L that is the condition for switching the camera is set to the switching distance L1 = 5 m. The switching distance L1 is an example of the first distance. When the target vehicle is approaching the vehicle 10 and the distance L between the vehicle 10 and the other vehicle is greater than L1 and less than L2 (in the present embodiment, when L1 = 5 m < L < L2 = 100 m), the control unit 303 measures the distance L based on the image information acquired by the second camera 103C. When the distance L approaches to be less than or equal to L1 (in the present embodiment, when approaching to be less than or equal to L1 = 5 m), the control unit 303 switches the camera used for distance measurement from the second camera 103C to the first camera 102C. When the distance L once approaches within L1 = 5 m but then separates to be more than L1 = 5 m, the control unit 303 determines that the distance L is more than L1 = 5 m in the state where the distance is measured by the first camera 102C, and switches the camera used for distance measurement from the first camera 102C to the second camera 103C. When the target vehicle repeatedly approaches and separates from the vehicle 10 around L1 = 5 m, the control unit 303 frequently switches between the first camera 102C and the second camera 103C, but there is no problem. This is because when the distance L is around L1 = 5 m, both the first camera 102C and the second camera 103C have a margin for distance measurement and vehicle detection (for example, RCW). Regarding the recognition of the license plate number, it may be configured such that there is no problem in the recognition limit of license plate number recognition by each camera.

[0141] According to this first example of operation, when the distance L between vehicle 10 and the target vehicle is large, the second camera 103C performs distance measurement, vehicle detection, and license plate recognition. When the target vehicle is approaching vehicle 10 and the distance L between vehicle 10 and the other vehicle is small, the wide-angle camera performs distance measurement, vehicle detection, and license plate recognition. This improves the accuracy of each process and enables a more accurate determination of the possibility of dangerous driving.

[0142] <2-2> Second example of operation Next, a second example of operation will be explained with reference to Figure 17. In the first example of operation described above, depending on the configuration of the drive recorder 101, even if there is sufficient capacity for distance measurement and vehicle detection using the first camera 102C and the second camera 103C, at a distance where the distance L is around L1 = 5m, both the first camera 102C and the second camera 103C may reach their recognition limits for license plate recognition. In this case, the control unit 303 may not be able to recognize the license plate when switching cameras.

[0143] Therefore, the control unit 303 is better configured to switch between the cameras alternately at predetermined intervals (for example, every 50ms) rather than switching between the first camera 102C and the second camera 103C. In this second example of operation, the control unit 303 may choose to use only the second camera 103C, or to alternately switch between the second camera 103C and the first camera 102C, depending on the distance measured by the cameras.

[0144] For example, let's say the switching distance L3 is set to 10m. The switching distance L3 is an example of a second distance. The distance L1 = 5m mentioned above is the limit of recognition distance at which both the first camera 102C and the second camera 103C can recognize the vehicle number, but the distance L3 = 10m is a distance at which the first camera 102C cannot recognize the vehicle number or it is difficult, but it is a distance at which the second camera 103C can recognize the vehicle number. Furthermore, the switching distance L3 is a distance at which both cameras can adequately measure distance and detect vehicles. Here, in the range of distance L1 to L3 (in this embodiment, distance L is 5m to 10m), the value measured by the second camera 103C is expected to be more reliable, and in the range of 0m to less than L1 (in this embodiment, 0m to 5m), the value measured by the first camera 102C is expected to be more reliable. Therefore, the control unit 303 should use the value from the camera from which a more reliable value is expected to be obtained.

[0145] When the distance L is in the range of L1 (approximately 5m in this embodiment), both cameras can measure the distance, detect the vehicle, and recognize the license plate number, and values ​​can be obtained from both cameras, so the control unit 303 can choose one of the values. Since the license plates recognized by both cameras may be different, for example, the control unit 303 can compare the license plate number recognized when L1=5m or more and decide which value to adopt, for example, by adopting the value from the camera that recognized the same license plate number.

[0146] Another issue is the time required for distance measurement, vehicle detection, and license plate recognition. If the cameras are switched alternately every 50ms, each camera needs to process the data within 100ms. Even in this case, since the distance L between vehicle 10 and other vehicles is within 10m, the relative speed is unlikely to be 20km / h or higher. If the relative speed is 20km / h, the change in distance over 100ms would be approximately 0.56m, so a judgment time (processing time) of 100ms is considered acceptable.

[0147] Thus, when the distance L is greater than or equal to the switching distance L3 = 10m, the control unit 303 may perform vehicle detection and license plate recognition using the second camera 103C every 50ms. When the second camera 103C determines that the distance is less than or equal to L3 = 10m, the control unit 303 switches between the first camera 102C and the second camera 103C alternately every 50ms. This switching ensures that the first camera 102C and the second camera 103C perform vehicle detection and license plate recognition processing effectively every 100ms. In this embodiment, the case where vehicle detection and license plate recognition are performed by alternately switching between the first camera 102C and the second camera 103C has been described, but at least performing license plate recognition in this manner can reduce the possibility of failing to recognize the license plate.

[0148] [Third Embodiment] In this embodiment, the following configuration is adopted for measuring the distance L, which is the distance between vehicles determined from perspective (perspective view). The configuration of this embodiment will be described with reference to Figures 18 and 19. The configuration of this embodiment can be applied, for example, to the configuration of the second operation example of the second embodiment described above.

[0149] The control unit 303 measures the distance to the target vehicle using image information acquired by the camera, using a measure calculated from the perspective view of that image information. The reference measure will differ depending on the camera's height from the ground, the camera's orientation, and the lens's focal length. Therefore, the control unit 303 determines the reference measure setting line by aligning it with the horizon, assuming that the camera's orientation is horizontal to the ground.

[0150] The operation of this embodiment will be described with reference to Figures 18 and 19. The measure setting line is, for example, a line parallel to the horizon as shown in Figures 18 and 19. The measure setting line is an example of a reference position set for image information for distance measurement. The measure setting lines are different for the first camera 102C and the second camera 103C, but the control unit 303 sets the measure setting line for each of the image information acquired by the first camera 102C and the second camera 103C. Assuming that the height from the ground is the same for both the first camera 102C and the second camera 103C, and that the orientation of the cameras is horizontal to the ground, it is necessary to align the horizons of the two cameras. Therefore, the control unit 303 first determines the height of the camera from the ground. For example, the camera height may be input by the user, but other methods may also be used. The camera height input by the user may be selected by the user from several options such as 100cm / 110cm / 120cm...250cm. The control unit 303 identifies the difference in the images from both the first camera 102C and the second camera 103C from the selected height, corrects for aligning the horizon, and then measures the distance using the images from each camera.

[0151] The above method assumes, for example, that two cameras are at the same height from the ground and that their orientation is horizontal to the ground. However, in reality, it is possible to align the cameras horizontally to the ground (i.e., perpendicular to the direction of gravity), but this can be troublesome for the user, so the control unit 303 should automatically correct this. The control unit 303 should also detect the camera angle and reflect it in the settings of the measure setting line, even if the camera orientation is slightly upward or downward.

[0152] Furthermore, it may be transformed as follows: <3-1> If the target vehicle is a vehicle behind, the control unit 303 may determine whether it is the vehicle directly behind or not by drawing a virtual line and determining whether it is inside or outside that line, without specifying the lane behind. Similarly, if the target vehicle is a vehicle in front, the control unit 303 may determine whether it is the vehicle directly in front or not by drawing a virtual line and determining whether it is inside or outside that line, without specifying the lane ahead.

[0153] <3-2> The control unit 303 may be provided with a function to set a reference position, such as the position of the horizon or the vanishing point of the road, in the captured image. Furthermore, the control unit 303 may be provided with a function to set a reference position within the image of one camera by user operation, and to set a reference position within the image of the other camera without user operation (i.e., automatically) based on the installation position relationship (arrangement) of the two cameras.

[0154] <3-3> The control unit 303 should have a function to set a reference position, such as the position of the horizon or the vanishing point of the road in the captured image. Furthermore, the control unit 303 should have a function to display the images of both cameras simultaneously and to display the respective reference positions within each image, and should automatically and simultaneously display the reference position in the other camera image that corresponds to the reference position set by user operation for the image of one camera. For example, when the reference position of the telephoto side is set, the control unit 303 should display that position with a mark or line in the telephoto side image. The control unit 303 should calculate the reference position of the wide-angle side that corresponds to that position, and automatically display that position with a mark or line in the wide-angle image. In this way, by observing the difference between the two, the reference positions of the two cameras can be easily aligned by adjusting the installation of the device to be horizontal or by adjusting the manual setting of the telephoto side.

[0155] <3-4>The configurations of each modified example of this embodiment may be applied to a device equipped with multiple cameras having different angles of view and overlapping shooting ranges, as exemplified by the first camera 102C and the second camera 103C.

[0156] [Fourth Embodiment] In this embodiment, the following configuration is preferable for recognizing the vehicle number.

[0157] <4-1> The control unit 303 may not initiate the recognition of the vehicle number triggered by the distance to the target vehicle or vehicle detection, but may instead continuously (for example, at all times) perform the process of recognizing the vehicle number. Continuously recognizing the vehicle number means that while the vehicle 10 is in motion, the control unit 303 repeatedly performs the process of recognizing the vehicle number, for example, at predetermined intervals. Continuously recognizing the vehicle number means that, regardless of whether the distance between vehicles is measured or whether a vehicle is detected or not, the vehicle number recognition result (a signal indicating that recognition is not possible, or a recognized vehicle number (for example, a 4-digit number)) is output at predetermined intervals (for example, every 50ms). In this way, not only is the vehicle number of the target vehicle recorded, but also data indicating the circumstances at the time the recognition process was performed, along with data to determine whether recognition was possible, which can be used to analyze the situation.

[0158] <4-2> The control unit 303 may record the vehicle number recognition result on the SD card 306 separately from the video information (an example of image information) acquired by the camera. Furthermore, the control unit 303 may create a continuous recording folder and an event folder on the SD card 306, similar to the method for recording video information, and sequentially record video information in the continuous recording folder while overwriting it. When a predetermined event occurs, such as vehicle detection (e.g., RCW, FCW), acceleration sensor, or manual operation, the data may be moved from the continuous recording folder to the event folder, or the overwriting of video information or vehicle number recognition results recorded in the continuous recording folder may be suppressed. Suppression of overwriting can be done by prohibiting overwriting, but it is also conceivable to implement a process that makes it difficult to allow overwriting in cases where overwriting is permitted due to other reasons, thereby making it difficult for video information or vehicle number information to be erased.

[0159] <4-3> The control unit 303 may continuously perform the process of recognizing the vehicle number, regardless of whether it can recognize the vehicle number or not, and continuously record the vehicle number information to the SD card 306. The vehicle number information may include, for example, the vehicle number if it can be recognized, and may also include information such as the time, latitude and longitude, vehicle speed, and acceleration as data indicating the circumstances when the process of recognizing the vehicle number was performed. Continuous recording means repeatedly recording at predetermined intervals while the vehicle 10 is running. The control unit 303 may also continuously record still images (for example, JPEG still images; an example of image information) based on the image information captured by the camera at a predetermined interval (for example, 1 fps). The still images are, for example, images captured by the first camera 102C, but the control unit 303 may also record images (for example, JPEG still images) captured by the second camera 103C to the SD card 306 while continuously recording with the first camera 102C.

[0160] <4-4> The control unit 303 may suppress overwriting of the vehicle number information and still images continuously recorded on the SD card 306 by means of distance measurement, vehicle detection, judgment, acceleration events (G events), and manual events, by copying them to a predetermined folder (for example, an event folder) or by applying a file lock. Suppressing overwriting may be done by prohibiting overwriting, but it is also possible to implement a process that makes it difficult to allow overwriting in cases where overwriting is permitted due to other reasons, thereby making it difficult for moving image information and vehicle number information to be erased.

[0161] <4-5> The control unit 303 may repeatedly detect the approach of a rear vehicle (e.g., RCW / BSD) using the rear camera exemplified by the drive recorder 101R, recognize the vehicle number, and if a vehicle with the same vehicle number repeatedly approaches using the front camera (e.g., FCWS) exemplified by the drive recorder 101F, it may determine that it is a serious dangerous driving offense and perform processing according to the determination result. Alternatively, the control unit 303 may repeatedly detect the approach of a front vehicle (e.g., FCWS) using the front camera exemplified by the drive recorder 101F, recognize the vehicle number, and if a vehicle with the same vehicle number repeatedly approaches using the rear camera (e.g., RCW / BSD) exemplified by the drive recorder 101R, it may determine that it is a serious offense and perform processing according to the determination result. Determining it as a serious offense may be equivalent to determining it as a dangerous driving offense. The processing according to the determination result may be the same as the processing when it is determined to be a dangerous driving offense as described above, but it may also be a different processing.

[0162] [Fifth Embodiment] In this embodiment, the following configuration may be adopted for detecting people outside the vehicle 10. <5-1> The drive recorder 101 may have a function to detect people outside the vehicle 10 and perform processing according to the detection result. The drive recorder 101 may be a camera that acquires image information of a planar image, but it is even better if it is a camera that acquires image information of a celestial spherical image, as it can capture a wider area and detect people. A camera that acquires image information of a celestial spherical image is a camera that captures an area of ​​the hemisphere, or an area wider than the hemisphere, and acquires image information. The image area of ​​the celestial spherical image is circular or elliptical. In a celestial spherical image, the distortion caused by the imaging lens (image distortion) is small near the center, and the degree of distortion increases as you approach the periphery (circumferential direction) in the radial direction. Near the center of the celestial spherical image, the subject is captured at a relatively large size, and as you approach the periphery in the radial direction, the subject is captured at a relatively small size. The celestial spherical image may be, for example, a hemisphere image or a full celestial spherical image. As shown in Figure 20, the drive recorder 101 may be, for example, a drive recorder 101F for detecting people outside the vehicle 10. The drive recorder 101F (an example of a third camera) may record outside the driver's seat or passenger seat of the vehicle 10 (for example, outside the window, through the windshield, etc.) and detect the person who committed the dangerous act (for example, the person who provoked the other driver) if a dangerous act occurred (for example, if the other driver provoked the other driver). In Figure 20, the person who provoked the other driver is shown as "Person P1".

[0163] In the drive recorder 101F, the area around the driver's or passenger's side window should be designated as the area for detecting aggressive drivers (hereinafter referred to as the "detection target area"). The shape, position, and size of the detection target area should be set by the user. This is because the areas captured inside and outside the vehicle differ depending on the vehicle. For example, the control unit 303 displays the image captured by the drive recorder 101F on its own display unit (on the LCD if an LCD is included), and allows the user to set the area of ​​the image corresponding to the outside of the vehicle as the detection target area. Alternatively, the control unit 303 may transmit the image captured by the drive recorder 101F to the user terminal 107. In this case, the user terminal 107 should display the image, allow the user to set the detection target area of ​​the image, and notify the drive recorder 101F of this setting. For example, if the image shown in Figure 21(a) (in this example, a hemispherical image) is captured by the drive recorder 101F, then detection target areas Ar1 and Ar2, as shown in Figure 21(b), will be set. Detection target area Ar1 corresponds to the position of the window, and detection target area Ar2 corresponds to the position of the windshield. The shape of the detection target area can be any shape, and while allowing the user to freely determine the shape would allow for accurate identification of the position outside the vehicle 10, a quadrilateral shape such as a square or rectangle is easy for the user to set, and polygons, ellipses, etc., are also acceptable.

[0164] Furthermore, the control unit 303 may, as a predetermined process, start event recording when it detects a person in the detection target area, and may also have a function to notify the person (for example, warn) that event recording is in progress in a human-perceptible way such as light, sound, or voice. For this purpose, for example, the drive recorder 101F may have a light-emitting part that is visible from outside the vehicle 10, or the drive recorder 101F may have a function to output sound that can be heard outside the vehicle 10. Alternatively, the drive recorder 101F may be connected to the vehicle 10 in a communicative manner and provide notification in a manner coordinated with the vehicle 10, such as turning on the vehicle 10's hazard lights or sounding a buzzer. In this way, when a person is detected outside the vehicle, the image information acquired by the drive recorder 101F is recorded, the situation when the person was detected in the vehicle is confirmed using the image information, and the person outside the vehicle is notified that recording is in progress, thus providing an excellent crime prevention effect against vehicle theft and other thefts.

[0165] The control unit 303 may perform the above-mentioned predetermined processing if it detects a person in the detection area after determining that the following vehicle 40 is driving dangerously (for example, after detecting aggressive driving). The control unit 303 may also perform the above-mentioned predetermined processing if it detects the same person in the detection area as the occupant (for example, the driver) of the following vehicle 40 after determining that the following vehicle 40 is driving dangerously (for example, after detecting aggressive driving). The same can be done with the drive recorder 101R, and the control unit 303 may determine that the vehicle in front is driving dangerously.

[0166] Object detection techniques can be applied to detect people outside vehicle 10. These techniques may include detection using HoG features and SVM, or detection using Haar-like features based on image brightness differences. In particular, deep learning methods such as R-CNN, YOLO, and SSD are recommended. When using deep learning, it is advisable to prepare by collecting numerous images containing the object to be detected and tagging them with bounding boxes and object names as the correct values. These tagged images can then be used to adjust the layer weights of the trained model to recognize specific objects.

[0167] <5-2> The control unit 303 of the drive recorder 101 should perform a process to make the face of a person difficult to see when it recognizes a person's face inside the vehicle 10 and records the captured image on the SD card 306. The process to make the face difficult to see should be a process to conceal the face, such as applying a mosaic or overlaying (masking) another image, and should be a process that makes it difficult for the human eye to understand what kind of face it is. The control unit 303 should, for example, apply a mosaic to the face of a person inside the vehicle when recording an image, even without user instruction (i.e., automatically). For faces outside the vehicle 10, the control unit 303 should not perform a process to make the face difficult to see, or if it does perform such a process, it should record the face as a separate image. Such recording is an example of recording the face of a person outside the vehicle in a way that allows for identification of that face. The reason for not performing a process to make the face difficult to see for faces outside the vehicle 10 is for security purposes. This configuration allows for the protection of the privacy of legitimate users of the vehicle, such as the vehicle's owner, while simultaneously recording the faces of potentially malicious individuals outside the vehicle in a way that allows for their identification.

[0168] When the control unit 303 transmits image data of the captured image to the user terminal 107, for example, when transmitting a stream image, it is preferable to perform the above-described process to make it difficult to see the faces of people inside the vehicle 10.

[0169] <5-3> The control unit 303 may perform a first process as a predetermined process to make it difficult to see the faces of people inside the vehicle 10 in the image area indicated by the image information acquired by the drive recorder 101F, and a second process to cancel the first process and make the image information acquired by the drive recorder 101F playable, and then record the image information after the second process. When viewing the images taken by the drive recorder 101 with predetermined software (viewer) such as a PC viewer, the process to make it difficult to see the faces of people inside the vehicle 10 may remain in place, but it may be possible to cancel it. The first process may be the same as the process to make faces difficult to see as described above. The second process may be a process that disables the first process, such as removing the mosaic or removing the overlaid image (removing the mask), and may be a process that makes it easier for the human eye to understand what kind of face it is. In order to enable deactivation by the second process, the control unit 303 may perform a predetermined image processing that enables deactivation as the first process. Alternatively, the first process may be to record image information that has not been processed to make it difficult to see the faces of people in the passenger compartment of the vehicle 10 onto the SD card 306 in a way that makes it inaccessible by normal means (for example, by making it a hidden file or by encrypting it). In this way, the privacy of people who are likely to be legitimate users of the vehicle, such as the vehicle owner, is protected, while making it possible to replay the image information so that their faces can be seen when necessary.

[0170] <5-4> The control unit 303 may perform the following processing based on the result of recognizing a human face from the captured image. The face recognition function for recognizing a human face from the captured image may be provided by the control unit 303, but it may also be provided by an external device that can communicate with the drive recorder 101F (for example, a server device, a user's smartphone or other mobile terminal). To explain using Figure 21, for example, if driver D leans forward, driver D's face may move upward in the image and into the detection target area Ar1. In this case, the control unit 303 may refrain from processing to make the human face difficult to see if it recognizes a human face within the detection target area Ar1, and may perform processing to make the human face difficult to see if it does not recognize a face. Recognizing a human face within the detection target area Ar1 may mean when it matches the face of driver D that has been registered in advance (i.e., when face recognition is successful), or it may mean that any face has been recognized. As in this embodiment, face recognition may always be performed on the entire shooting area instead of setting a detection target area, but by setting a detection target area and performing face recognition, the processing load can be reduced. In this way, even if the face of a person who appears to be a legitimate user of the vehicle, such as the vehicle's owner, enters the detection area (which is considered to be outside the vehicle) due to a change in their posture or other factors, their privacy can be protected.

[0171] <5-5>When the control unit 303 records images captured by the camera using the parking surveillance function, it is preferable to avoid processing that makes it difficult to see people's faces (i.e., to stop processing that makes it difficult to see people's faces). The parking surveillance function is a function that records images captured while the vehicle 10 is parked. The parking surveillance function is a function for monitoring the inside of the parked vehicle 10 or the outside of the area surrounding the vehicle 10. When the engine of the vehicle 10 is off, the control unit 303 receives power from an external battery and records images to the SD card 306. The control unit 303 may determine whether or not the vehicle 10 is parked based on one or more of the following: for example, that the accessory power is turned off, that the engine is turned off, that power supply from the external battery has started, that the vehicle speed is 0 km / h or below a predetermined speed, and that the location information acquired by the location information acquisition unit is predetermined location information (for example, location information of home, workplace, or parking lot). This method allows a person's face to be visible even if they are sitting in the driver's seat, such as a thief, and their face is in an area that is not covered by the window or windshield.

[0172] <5-6> The inventors discovered that when software generates an image by performing a predetermined coordinate transformation so that a celestial image can be unfolded into an image such as a rectangle that is easily recognized by AI (hereinafter referred to as "dewarp"), it can only be processed at a low frame rate of, for example, around 3 fps. It is conceivable that the processing speed will become even more critical if a process for recognizing people is added in the later stages. Also, when detecting people outside the vehicle 10 using object detection technology, applying the object detection technology to the dewarped image may improve detection accuracy rather than applying it to the celestial image. Therefore, the control unit 303 generates an image by performing a predetermined coordinate transformation on the detection target area (for example, only the window area of ​​the vehicle 10) as the dewarp target area. Then, the control unit 303 applies object detection technology to the image after the coordinate transformation to detect people outside the vehicle 10. In this way, the accuracy of detecting people outside the vehicle 10 can be improved, as well as the processing speed. Furthermore, at least two of the following areas of the celestial image may be approximately the same size: the dewarp area to be dewarped, the exposure calculation area, and the image recording area, which is the area where the image is recorded on the SD card 306. For example, the control unit 303 may determine at least two of the dewarp area, the exposure calculation area, and the image recording area based on the set detection target area.

[0173] <5-7> The configuration of this embodiment can also be applied to imaging devices other than cameras mounted on vehicles, such as indoor cameras. The imaging device may be, for example, a security camera. The imaging device may, for example, photograph an indoor space such as a residence or office, and if it recognizes a human face within a detection target area set at the location of a window or door, it is preferable that it does not perform processing that makes the human face difficult to see.

[0174] [Sixth Embodiment] Regarding the image enhancement process for clarifying license plates (or only the portion showing the vehicle number) included in images captured by the drive recorder 101, the following processing may be employed.

[0175] <6-1> The control unit 303 may sharpen the license plate in the image using, for example, deep learning. For example, a pre-trained model that has been trained on images of license plates may be prepared in advance, and the control unit 303 may provide image information to this pre-trained model to make corrections. If the drive recorder 101 takes a spherical image, the effect of sharpening can be further enhanced by using a pre-trained model that has been trained on images of the same or similar type of image (for example, the degree of geometric deformation, the type of lens used for shooting), such as using a pre-trained model that has been trained on images of license plates included in the spherical image.

[0176] The function to enhance the clarity of <6-2> and <6-1> may be installed in the drive recorder 101, but it is preferable to install it on the viewer side so that the drive recorder 101 does not have this function. In this way, the images recorded by the drive recorder 101 will not be altered (processed), and will be recorded while maintaining the original image.

[0177] <6-3> The control unit 303 may generate new image information that enhances the image shown by the image information captured by the drive recorder 101 without altering it, and record both of these image information. This is done so that the original image can be preserved as is, or because altering the original image would constitute tampering, the original image can be left as is, while an image with a clearer license plate can be obtained.

[0178] <6-4> The control unit 303 may sharpen the license plate area automatically (i.e., without user intervention) when playing back a video, but it is preferable to sharpen the license plate area only when the video is paused while the license plate is displayed. This is based on the idea that the image from the drive recorder 101 will be sharpened when it is actually played back, and the image will remain as is until then.

[0179] <6-5> The control unit 303 may automatically identify the license plate area within the image without user specification, but it is preferable for the user to manually specify it by drawing a box around it. This eliminates the need for processing related to identifying the license plate area, or allows for accurate identification of the license plate area. In addition, the user can view an image that is as close to the original as possible.

[0180] <6-6> The control unit 303 may add information to the enhanced image, such as "License plate enhanced," to indicate that the image has been enhanced, and then display or record at least one of these. In this way, the user can understand that it is not the original image but an image after license plate enhancement.

[0181] [Seventh Embodiment] The following processing may be performed by recognizing traffic signals from the images captured by the drive recorder 101F.

[0182] <7-1> The control unit 303 may detect from the captured image that the traffic signal is red and, triggered by the fact that a vehicle traveling in the same direction or in the opposite direction (opposite lane) is attempting to enter the intersection, determine that it is dangerous driving. The processing according to the determination result may be the same as the processing for when dangerous driving is determined as described above, but different processing may also be used.

[0183] <7-2> The control unit 303 may, when it detects from the captured image that the traffic signal is green, use the captured image as a trigger to determine that the vehicle is driving dangerously if it sees a vehicle entering the intersection from the side where the traffic signal is red (for example, in the intersecting direction) (for example, crashing into the intersection). The processing according to the determination result may be the same as the processing when dangerous driving is determined as described above, but different processing may also be used.

[0184] <7-3> The control unit 303 should detect the flashing of the pedestrian traffic signal from the captured image, detect that the signal may change soon, and notify the user. Such flashing signals are information about the signal changing in front of the user. For example, there are pedestrian traffic signals that count down the time until they switch to green or red, and this is the idea of ​​displaying that information.

[0185] <7-4> The control unit 303 may store the detected speed limit sign data on the map and provide notification even if the speed limit sign cannot be recognized for reasons such as inability to perform image recognition. The speed limit sign data may include, for example, the speed limit and location information of the speed limit sign's installation point (e.g., latitude and longitude information). Notification may be given, for example, when the vehicle 10 is driving near or at the location of the speed limit sign (e.g., within a predetermined distance range, while driving on the same road). Notification may be given by a human-perceptible method such as light, sound, or voice. Notification may include, for example, the speed limit displayed on the speed limit sign in km / h and whether the vehicle's current speed exceeds the speed limit. In this way, for example, the stored speed can be used to provide notification while driving in the opposite lane. The control unit 303 may also transmit the detected speed limit sign data to a server device or other devices such as other drive recorders via communication, and use it as shared information, map it, and transmit the speed limit to other vehicles. The control unit 303 should store data on speed limit signs received from the server device or other devices such as drive recorders on a map, and based on this data, it should notify the driver even if the speed limit sign cannot be recognized.

[0186] <7-5> The control unit 303 may also store information about nearby traffic signals during driving in the recordings of the drive recorder 101. This way, even if the traffic signals are not recorded in the video, their color can be determined.

[0187] [Eighth Embodiment] As a function to check the area behind the vehicle 10 when it is reversing, the drive recorder 101 may have the following functions. The drive recorder 101 (for example, drive recorder 101F) may have a function to detect an object being recorded by the drive recorder 101R and notify (for example, alert) when it detects the reverse gear (in other words, when it is determined that the gear is in reverse). The object may be, for example, a person or an obstacle, which may be an object that could collide with the vehicle 10. The control unit 303 may detect the reverse gear (determine whether the gear is in reverse) based on a signal input from the vehicle 10 via a connection terminal (not shown) (specifically, a signal that identifies the gear position). The notification may be made by sound, light, display or other human-perceptible method.

[0188] Furthermore, the control unit 303 should vary the notification (how it alerts) depending on the distance from the vehicle 10 to the object. For example, if the distance to the object is close, such as the first distance or less, the control unit 303 should warn the user not to back up the vehicle 10 (for example, by outputting an audio message such as "No backing up!!"). If the distance from the vehicle 10 to the object is a little far, for example, greater than the first distance but less than or equal to the second distance, the control unit 303 should give the user a warning (for example, by outputting an audio message). If the distance from the vehicle 10 to the object is far, for example, greater than the second distance, the control unit 303 should display the image captured by the drive recorder 101R on the display unit (for example, the display unit of the drive recorder 101), and further display the object included in the captured image with a shape such as a rectangle, and there should be no sound for the notification.

[0189] Furthermore, the control unit 303 may trigger an event (danger event) and perform event recording when the distance to the object is close, for example, when it falls below the first distance. This event recording may record images from the drive recorder 101R for a predetermined period before and after the distance to the object becomes close, but it may also record only images from the point in time when the distance to the object becomes close.

[0190] If the drive recorder 101 (for example, drive recorder 101F) is a communication type that can communicate with an external server device or the like via a network, the control unit 303 may notify the system of the occurrence of the event via the network. This notification may be a notification to register the location where the event occurred in map data such as a facade map registered in the server device or the like. If the distance from the vehicle 10 to the object is a little far, the control unit 303 may count the cumulative time spent at this distance in the same location, and when it reaches the upper limit of a certain threshold, it may notify the system to register the location where the event occurred in map data such as a facade map. This map data may be map data stored in the drive recorder 101, or map data that can be obtained from an external device such as a server device. When the data is displayed based on the acquired map data, the user can visually understand where there are objects that pose a collision risk, which can contribute to safe driving. In this embodiment, the drive recorder 101R does not have to be a device equipped with two or more cameras.

[0191] [Ninth Embodiment] The following are various other possible variations. For example, the rear camera for detecting aggressive driving can also be used as a front camera for recognizing traffic signs. This rear camera for detecting aggressive driving corresponds to the license plate detection camera 103 of the drive recorder 101R in the above-described embodiment. The rear camera for detecting aggressive driving and the front camera for recognizing traffic signs have in common that they both benefit from having a narrower field of view than the recording camera. Therefore, the performance requirements for the rear camera for detecting aggressive driving and the front camera for recognizing traffic signs are similar, and by using the same components for both, the number of items in stock can be reduced, and the number of parts that can be purchased can be increased, thus reducing manufacturing and storage costs. In this way, by mounting the drive recorder 101R in front of the vehicle 10, similar to the drive recorder 101F, the drive recorder 101R can be used for other purposes.

[0192] The rear camera used for tailgating detection can also be used as a front camera for ADAS (Advanced Driver-Assistance Systems). Furthermore, ADAS functionality may be added to the front recording camera footage, allowing it to operate in coordination with the rear camera used for tailgating detection. Furthermore, the rear camera used for detecting aggressive driving can also be used as a driver monitoring camera. Furthermore, an infrared illuminator can be attached to the rear camera used for detecting tilting. The rear camera for detecting aggressive driving can also be attached to the front camera and integrated into a single unit.

[0193] When the rear camera for detecting aggressive driving is integrated with the rear camera for recording, the two cameras are integrated in a way that prevents angle adjustment. However, when it is integrated with the front camera as a camera for recognizing traffic signs or monitoring the driver, the two cameras can be integrated in a way that allows for angle adjustment.

[0194] The drive recorder 101 can be installed both in front of and behind the vehicle 10, and it can be configured to have both a mounting section for installation in front of the vehicle 10 and a mounting section for installation in rear of the vehicle 10. Alternatively, it can be configured to allow switching between the mounting section for installation in front of the vehicle 10 and the mounting section for installation in rear of the vehicle 10.

[0195] Furthermore, the drive recorder 101F installed in front of the vehicle 10 may also be equipped with a mounting part that allows it to be used as an ADAS camera in front of the vehicle 10. Furthermore, the drive recorder 101F, which is installed in front of the vehicle 10, may also be equipped with a mounting part that allows it to be used as a camera to record the rear of the vehicle 10 from the front of the vehicle 10.

[0196] The drive recorder 101 can also incorporate an infrared illuminator. This is particularly useful when used as a camera to record the rear of the vehicle 10 from the front of the vehicle 10. Furthermore, a cover or seal can be provided to physically conceal the infrared illuminator. In particular, a cover can be provided in place of the filter in the mechanism for inserting the visible light cut filter. Furthermore, the mechanism for inserting a visible light cut filter can be provided in both the vehicle imaging camera 102 and the vehicle number detection camera 103, or in only one of them.

[0197] Alternatively, the configuration may include a mounting device that allows the cylindrical housing to be installed so that its axial direction is vertical. In this case, for example, the license plate detection camera 103 can be installed so that it faces downwards. In this case, it becomes easier to recognize license plates located on the underside of vehicles in front or behind. Conversely, the license plate detection camera 103 can be installed so that it faces upwards. In this case, signs and signals become easier to recognize.

[0198] In a configuration where vehicle 10 is equipped with both a front drive recorder 101F and a rear drive recorder 101R, the video can be stored in either the front or rear recorder, or it can be transmitted to one and recorded in the other. If recording is performed by either the drive recorder 101F or the drive recorder 101R, the other device only needs to have the function of transmitting video to the other device. Therefore, the other device does not need to be recognized as a drive recorder; it just needs to be a device with a recording function.

[0199] Even when recording on both devices, it is possible to duplicate the recording by transmitting the video from one drive recorder 101 to the other. In this case, the time of each drive recorder 101 can also be synchronized and recorded using the in-vehicle network. Furthermore, it is possible to distribute the load of processing that consumes processor power, such as aggressive driving detection, between the front and rear drive recorders 101.

[0200] Furthermore, although the drive recorder 101 of this embodiment has been described as having two cameras, a license plate detection camera 103 and a vehicle image camera 102, for example, the front drive recorder 101F may have only the vehicle image camera 102, and the rear drive recorder 101R may have two cameras. By using only a single camera for the front-facing drive recorder 101F, the camera installed at the front, which tends to obstruct the driver's view, can be miniaturized, resulting in a clearer driving view. At the rear, AI processing such as license plate recognition can be performed using the image from the narrow-angle camera.

[0201] This configuration allows for thinner cables between the front and rear devices compared to a configuration with the 101F drive recorder at the front and a separate camera at the rear. It also makes it easier to record images from both narrow-angle and wide-angle cameras. Furthermore, when transmitting power from the front standalone camera to the rear drive recorder 101R, the front standalone camera can be equipped with a power input terminal, a video output terminal for the rear drive recorder 101R, and a power output terminal.

[0202] Another way to route the power is to split the power supply from a cigarette lighter socket or similar source and supply it to both the front standalone camera and the rear dashcam 101R. The video signal is transmitted from the front standalone camera to the rear dashcam 101R via the cigarette lighter socket or similar source. With this configuration, only one cable is needed to the front standalone camera, resulting in a neater wiring setup. In particular, routing multiple cables through the A-pillar, where airbags and other components are present, can be difficult for amateurs, but this makes it easier for users to do the wiring themselves as a DIY project.

[0203] Furthermore, in a configuration where the front camera is a standalone unit and the rear camera is the Drive Recorder 101R, the manual trigger button attached to the Drive Recorder 101R is no longer close to the driver. Therefore, it would be beneficial to also provide a manual trigger button on the standalone front camera. Furthermore, it would be beneficial to also include a microphone on the front-facing camera. This would allow for clearer recording of audio from the front.

[0204] Furthermore, it is also possible to configure the system with the 101F drive recorder at the front and a separate camera at the rear, and mounting hardware and cables that allow for such a configuration can be provided. The management terminal 105, management server 106, user terminal 107, etc., can set and display where the drive recorder 101 or standalone camera is installed, as shown in Figure 1, and the processing content can be changed according to this setting. It is preferable to display the placement location on the vehicle design as shown in Figure 1, but it is also acceptable to set the installation location using text such as front right, front left, front center, rear right, rear left, rear center, etc.

[0205] Depending on the installation location of the drive recorder 101 or standalone camera, the manual trigger button on the standalone camera and the manual trigger button on the drive recorder itself can be switched on or off according to the settings, and the buttons on rear-mounted devices can be switched on or off, or the buttons on front-mounted devices can be switched on or off.

[0206] For example, with regard to the microphones mentioned above, the system could be configured to determine which microphone to prioritize during playback based on the placement settings. Alternatively, depending on the installation location, the system may switch the correspondence between the captured front and rear video signals and the recorded front and rear video data so that the front view is correctly reproduced as the front view on the PC viewer.

[0207] In addition to the method using the number of proximitys and proximity time mentioned above, aggressive driving can also be detected using the following methods, which combine image analysis and various other sensors, FCWS, and RCWS. A driver tailgating the vehicle in front wants to get as close as possible but doesn't want to hit it. Therefore, they accelerate to get closer, and then slam on the brakes when a collision seems imminent. If the vehicle in front is still moving, the sudden braking increases the distance, causing them to accelerate again to get closer. This again nearly results in a collision, leading to another sudden braking. This cycle of driving repeats itself.

[0208] Therefore, after detecting that the vehicle behind has accelerated and is approaching within a distance where a collision is possible, the system rapidly decelerates to move away from vehicle 10, and if it then accelerates again to approach within a distance where a collision is possible, the system controls the vehicle in a direction that suggests there is a high probability that the vehicle behind is tailgating. In this case, if vehicle 10 is accelerating and the vehicle behind it accelerates even more and approaches to a distance where a rear-end collision is possible, the system will control the system in a direction that suggests there is a high probability that the vehicle behind it is driving aggressively. Alternatively, if vehicle 10 is decelerating and a vehicle behind it approaches to a distance where a rear-end collision is possible, the system will control the vehicle in a direction that suggests it is unlikely the vehicle behind it is engaging in aggressive driving.

[0209] Furthermore, the system detects whether the brakes of the vehicle 10 are being applied, and if the brakes are being applied and a vehicle behind is accelerating and approaching, it controls the system in a direction that suggests there is a high probability that the vehicle behind is tailgating. In this case, if the following vehicle accelerates and approaches after the brakes are released, the system will control the vehicle in a direction that suggests there is a high probability that the following vehicle is tailgating.

[0210] In this embodiment, we have described a drive recorder 101 (F, R) having a vehicle imaging camera 102 and a license plate detection camera 103. However, instead of separating the vehicle imaging camera 102 and the license plate detection camera 103, a configuration in which each function is performed by a single camera is also possible.

[0211] In addition, in this embodiment, the configuration in which the drive recorder 101 performs the determination of aggressive driving has been described. However, the present invention can also be applied to a network camera that does not have a relatively large storage unit such as an SD card 306 for storing moving images, and transmits the captured moving images to the management server 106 via a network for storage on the management server 106 side.

[0212] In addition, the implementation of the aggressive driving determination system that executes the determination of the possibility of aggressive driving can be implemented not only in the drive recorder 101 but also, for example, in the vehicle 10 itself, the microcomputer 301 mounted on the vehicle 10, a computer, a server device, etc., or the management server 106.

[0213] When implemented in the management server 106, the processor 603 of the management server 106 analyzes the image information received via the network and determines the possibility of aggressive driving. A configuration in which the processor 603 of the management server 106 executes both or either of the measurement of the proximity count and the measurement of the proximity time instead of the drive recorder 101 may also be used. The image information received via the network may be not only from a specific vehicle but also image information uploaded by a plurality of users to the management server 106.

[0214] When the management server 106 determines the possibility of aggressive driving, the processing according to the aggressive driving determination level may also be performed on the management server 106. For example, it is possible to distribute image information etc. transmitted from a camera via a network to a high aggressive driving level folder, a medium aggressive driving level folder, a low aggressive driving level folder, etc. on the management server 106.

[0215] In addition, a configuration in which the control unit provided in the vehicle 10 itself executes the determination of the possibility of aggressive driving as described above and the processing according to the aggressive driving level which is the determination result may also be used. Further, a microcomputer, a computer, a server device, etc. mounted on the vehicle 10 may be configured to determine the possibility of the above-described aggressive driving and execute processing according to the determined aggressive driving level, and various modifications are possible for the execution entity of each process.

[0216] In the above-described embodiment, the possibility of dangerous driving is determined based on the behavior of the target vehicle in front of or behind the vehicle 10. However, the possibility of dangerous driving may be determined based on the behavior (e.g., lane encroachment) when the target vehicle is located on the side. In this case, the proximity area is also set on the side of the vehicle 10. In this case, if either the drive recorder 10F or 10R can capture the side of the vehicle 10, the possibility of dangerous driving may be determined based on the captured image, or a camera for capturing the side may be provided separately.

[0217] Note that the present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Also, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Also, for a part of the configuration of each embodiment, addition, deletion, or replacement with other configurations is possible.

[0218] Further, each of the above configurations, functions, processing units, processing means, etc. may be realized in hardware by designing a part or all of them, for example, by an integrated circuit. Also, each of the above configurations, functions, etc. may be realized in software by a processor interpreting and executing a program for realizing each function. Information such as a program, a table, a file, etc. for realizing each function can be stored in a memory, a recording device such as a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, a DVD.

[0219] Furthermore, the control lines and information lines shown are those deemed necessary for explanatory purposes, and not all control lines and information lines are necessarily shown in the actual product. In reality, it is safe to assume that almost all components are interconnected.

[0220] Furthermore, the scope of the present invention is not limited to the configurations explicitly described in the specification, but also includes combinations of various aspects of the present invention disclosed herein. While the configurations for which patent protection is sought are specified in the appended claims, we intend to include configurations disclosed herein that are not currently specified in the claims in the future.

[0221] The present invention is not limited to the configuration described in the embodiments above. The components of each embodiment and modification described above can be arbitrarily selected and combined. Furthermore, any component of each embodiment and modification can be arbitrarily combined with any component described in the means for solving the invention, or any component that embodies any component described in the means for solving the invention. The present application intends to obtain rights to these as well through amendments or divisional applications. Even if there is a description such as "in the case of..." or "when...", it is not meant to be a configuration that is limited to that case or time. Configurations that do not fall under these cases or times are also disclosed, and the present application intends to obtain rights to them. Also, even if there is a sequence of descriptions, it is not limited to that order. Configurations with some parts deleted or the order rearranged are also disclosed, and the present application intends to obtain rights to them.

[0222] Furthermore, by converting to a design registration application, we intend to acquire rights to the overall design or a partial design. The drawing depicts the entire device with solid lines, but it is a drawing that includes not only the overall design but also partial designs claimed for parts of the device. For example, it is a drawing that includes not only a partial design for a part of the device's components, but also a partial design for a part of the device regardless of its components. A part of the device may be a component of the device, or a part of a component. We intend to acquire rights not only to the overall design, but also to a partial design where any part of the solid lines in the drawing is represented by dashed lines. In addition, all modules, components, and parts inside the device's casing that are shown in the drawing are independently tradable, and similarly, we intend to acquire rights to them by converting to a design registration application.

[0223] The above-described embodiment discloses at least the following configuration. (1) It is a system, The system includes a control unit that determines the likelihood of dangerous driving by a target vehicle, based on the number of times the target vehicle, identified based on image information acquired by one or more cameras, has entered a predetermined proximity area. A system characterized by the following features. The inventors have found that the number of times a target vehicle enters a predetermined proximity area is an indicator of the likelihood of dangerous driving. By configuring the system in this way, it is possible to provide a mechanism for determining whether a photographed target vehicle is in a state where it is highly likely to be engaging in dangerous driving. (2) The control unit may determine the possibility of dangerous driving by the target vehicle based on the number of occurrences and the proximity time, which is the time the target vehicle is in the predetermined proximity area. Furthermore, the inventors discovered that by referring to the number of times a target vehicle enters a predetermined proximity zone, along with the proximity time (the duration of time spent in the proximity zone), the likelihood of dangerous driving can be determined with greater accuracy. This configuration provides a mechanism that more accurately determines whether a photographed target vehicle is likely to be engaging in dangerous driving, based on the number and duration of proximity actions. (3) The control unit may perform the count by detecting, based on the image information, that the target vehicle has entered the predetermined proximity area. This configuration eliminates the need for special sensors, allowing for detection of intrusion into the adjacent area solely through image processing by the camera. However, other sensors may be used in parallel with or in conjunction with image processing to detect intrusion into the adjacent area. (4) The possibility of dangerous driving can be represented by multiple levels of dangerous driving, and the control unit may change the processing after the determination according to the level of dangerous driving. This configuration allows for the appropriate action to be taken depending on the level of dangerous driving. (5) The control unit is characterized by storing recording information corresponding to the image information in different storage areas of the storage unit according to the level of dangerous driving. This configuration allows for the allocation of storage space for recorded information according to the level of dangerous driving, making management and display easier. (6) The control unit may store the recorded video information in a first memory area if the level of dangerous driving is high, and store the recorded video information in a second memory area with a shorter storage period than the first memory area if the level of dangerous driving is low. This configuration allows for a longer storage period for recorded images of vehicles with a high probability of dangerous driving, enabling them to be used for analysis and display over a longer period. At the same time, recorded images of vehicles with a low probability of dangerous driving can be deleted after a certain period has passed since the analysis was completed, thereby reducing the storage capacity required. (7) The control unit may store the recorded video in a first storage area that does not overwrite the recorded video if the level of dangerous driving is high, and store the recorded video in a second storage area that overwrites the recorded video if the level of dangerous driving is low. This configuration allows for a longer storage period for recorded images of vehicles with a high probability of dangerous driving, enabling them to be used for analysis and display over a longer period. At the same time, recorded images of vehicles with a low probability of dangerous driving can be overwritten and deleted after a certain period has passed since the analysis was completed, thereby reducing the amount of storage space required. (8) The control unit, If the above determination determines that the level of dangerous driving is high, the recording information is newly stored separately from the recording information already stored. If the above determination determines that the level of dangerous driving is low, it is advisable to delete the already stored video information and store the new video information. This configuration allows for the sequential saving of newly recorded images of vehicles with a high probability of dangerous driving, enabling their use for analysis and display over a longer period. At the same time, recorded images of vehicles with a low probability of dangerous driving can be deleted after a certain period has elapsed since the analysis was completed, thereby reducing the amount of storage space required. (9) The control unit should be configured not to determine that the vehicle is driving dangerously if the vehicle's acceleration meets predetermined conditions. This configuration prevents the system from mistakenly determining that a vehicle approaching from the front or rear is dangerous driving, for example, when acceleration changes due to sudden braking or acceleration. (10) The aforementioned predetermined condition is that the absolute value of the acceleration is equal to or greater than a threshold value. By configuring it in this way, it is possible to prevent the system from mistakenly determining that a vehicle in front or behind is driving dangerously when, for example, the absolute value of the acceleration exceeds a threshold and a vehicle is approaching in close proximity, such as when the brakes are applied suddenly or the car accelerates suddenly. (11) The acceleration may be corrected based on information obtained from a gyro sensor. By configuring it in this way, for example, when the acceleration is different from that during normal driving while driving on a slope, the value can be corrected and then the determination of dangerous driving can be made, improving the accuracy of the determination. (12) When the speed of the host vehicle is below a predetermined speed, the control unit may not determine it as dangerous driving. By configuring it in this way, for example, it is possible to prevent misjudging that the approach of a vehicle in front or behind during low-speed driving such as traffic congestion as dangerous driving. (13) When the number of times the target vehicle enters the predetermined proximity area within a predetermined time is less than 2, the control unit may not determine that the level of dangerous driving is high. [[ID=E]] By configuring it in this way, it is possible to prevent misjudging a single proximity behavior with a low possibility of dangerous driving as having a high level of dangerous driving. [[ID=1 The control unit may preferably have an output unit that outputs a warning when it determines that there is a high possibility of dangerous driving. By configuring it in this way, when a vehicle with a high possibility of dangerous driving approaches, a warning can be issued to prompt the driver to pay attention. (15) The control unit may extract the target vehicle from a plurality of vehicles shown in the image information and determine the possibility of dangerous driving for the extracted target vehicle. By configuring it in this way, even when a plurality of vehicles are shown in the image, the possibility of dangerous driving can be determined only for the extracted target vehicle, improving the accuracy of the determination. (16) The target vehicle may preferably be a vehicle existing on the same lane as the host vehicle. By configuring it in this way, the possibility of dangerous driving can be determined only for vehicles on the same lane as the host vehicle where dangerous driving may occur, improving the accuracy of the determination. (17) If the lane in which the vehicle is traveling is curved, it is preferable to determine that the vehicle located at the corrected position in the image information based on the information obtained from the gyro sensor is the vehicle located on the same lane. By configuring it in this way, when the lane is curved, the system can improve the accuracy of its assessment by determining the possibility of dangerous driving only for vehicles located behind or in front of the vehicle along the lane, rather than directly behind or in front of the vehicle itself. (18) The plurality of cameras may include a first camera, which is a wide-angle vehicle-shooting camera for shooting a wide area including the target vehicle, and a second camera, which is a narrow-angle vehicle number-shooting camera for shooting the vehicle number of the target vehicle. By using multiple cameras with different field-of-view angles in this configuration, the accuracy of identifying the vehicle's license plate number and other identifying details is improved compared to using only the first camera, which is a wide-angle vehicle camera, allowing for a more accurate assessment of the possibility of dangerous driving. (19) The plurality of cameras include a first camera and a second camera for photographing the target vehicle, the first camera having a larger field of view than the second camera, and the control unit preferably performs at least one of the following: measuring the distance to the target vehicle, detecting the target vehicle, and recognizing the vehicle number of the target vehicle, using the second camera when the distance to the target vehicle is greater than a first distance, and using the first camera when the distance to the target vehicle is less than or equal to the first distance. By configuring the system in this way, the camera used for measuring the distance to the target vehicle, detecting the target vehicle, and recognizing the target vehicle's license plate number is switched according to the distance to the target vehicle. This improves the accuracy of each process and enables a more accurate determination of the possibility of dangerous driving. (20) The plurality of cameras include a first camera and a second camera for photographing the target vehicle, the first camera having a larger field of view than the second camera, and the control unit preferably uses the first camera and the second camera to recognize the vehicle number of the target vehicle. This configuration reduces the possibility of the vehicle number not being recognized when switching cameras. (twenty one) The control unit may use the second camera to recognize the license plate number of the target vehicle when the distance to the target vehicle is greater than the second distance, and use the first camera and the second camera to recognize the license plate number of the target vehicle when the distance to the target vehicle approaches the second distance or less. This configuration reduces the possibility of the vehicle number not being recognized when switching cameras. (twenty two) The control unit may determine the installation height of the first camera and the second camera, and measure the distance to the target vehicle based on the determined height, a reference position set for the image information acquired by the first camera, and a reference position set for the image information acquired by the second camera. By configuring the system in this way, the effects of horizon shifts caused by differences in the images captured by the first and second cameras are compensated for, thereby improving the accuracy of distance measurement to the target vehicle using the first and second cameras. (twenty three) The control unit may continuously perform the process of recognizing the vehicle number while the vehicle is in motion, and regardless of whether the vehicle number is recognized or not, it may record the result of the recognition and data indicating the circumstances at the time the recognition process was performed. By configuring the system in this way, it is possible to record not only the license plate number of the target vehicle, but also data indicating whether or not it was recognized, as well as data showing the circumstances at the time the recognition process was performed, which can be useful for analyzing the situation. (twenty four) The control unit may use a third camera that photographs the interior of the vehicle to detect people outside the vehicle and perform processing according to the results of the detection. By configuring the vehicle in this way, it is possible to use a camera that films the interior of the vehicle to process the results of detecting people outside the vehicle. (twenty five) The control unit has a function to set the area corresponding to the area outside the vehicle from the image area indicated by the image information acquired by the third camera as the detection target area, and it is preferable to detect people who are visible within the set detection target area as people outside the vehicle. With this configuration, although the areas showing the interior and exterior of the vehicle may differ depending on the vehicle in which the camera is installed, people outside the vehicle can be detected more accurately within the image area indicated by the image information acquired by the third camera. (26) If the control unit detects a person outside the vehicle, it may perform the predetermined process of recording the image information acquired by the third camera and notifying the person outside the vehicle that recording is in progress. By configuring the system in this way, when a person is detected outside the vehicle, the image information acquired by the third camera is recorded, allowing the driver to confirm the situation when the person was detected using the image information. Furthermore, the driver is notified that recording is in progress, thus providing an excellent security measure against vehicle theft and other crimes. (27) The control unit may, as a predetermined process, perform the following: for the image region indicated by the image information acquired by the third camera, process the image to make it difficult to see the faces of people inside the vehicle and record them, and process the image to make it possible to identify the faces of people outside the vehicle and record them in a manner that allows for identification. This configuration allows for the protection of the privacy of individuals who appear to be legitimate users of the vehicle, such as the vehicle's owner, while simultaneously recording the faces of potentially malicious individuals outside the vehicle in a way that allows for their identification. (28) The control unit may record the image information after performing the following predetermined processes: a first process that makes it difficult to see the faces of people inside the vehicle in the image region indicated by the image information acquired by the third camera; and a second process that cancels the first process and makes the image information acquired by the third camera playable. In this way, it is possible to protect the privacy of individuals who appear to be legitimate users of the vehicle, such as the vehicle's owner, while also making it possible to reproduce their image information so that their faces can be identified if necessary. (29) The control unit records image information acquired by the third camera in a manner that allows for the identification of a person's face within the detection target area, which is an area outside the vehicle. If a face is recognized in the detection target area, and it is recognized that the face is a specific face, the control unit may perform processing to make the face difficult to see. In this way, even if the face of a person who appears to be a legitimate user of the vehicle (as exemplified by the vehicle's owner) enters the detection area (which is considered to be outside the vehicle) due to changes in their posture, their privacy can be protected. By setting a detection area and performing facial recognition, the processing load can also be reduced. (30) The control unit may, as part of the predetermined processing, perform a process to make it difficult to see the faces of people inside the vehicle in the image area indicated by the image information acquired by the third camera, and then record it, while refraining from performing the process to make faces difficult to see when recording while the vehicle is parked. By doing so, the privacy of those who appear to be legitimate users of the vehicle, such as the vehicle's owner, is protected, while at the same time, the faces of those inside the vehicle become more visible when it is parked and potentially stolen, thus providing an excellent crime prevention effect. (31) The image information acquired by the third camera is image information of a celestial image. The control unit performs a predetermined coordinate transformation on the image of the detection target area within the celestial image, and based on the transformed image, detects a person outside the vehicle. By doing so, the accuracy of detecting people outside the vehicle 10 can be improved, and the processing speed can be increased. (32) The second camera is preferably positioned so as not to interfere with the field of view of the first camera. This configuration prevents the inconvenience of losing part of the image from the first camera, which is a wide-angle camera used for photographing vehicles. (33) The orientation of the lens of the second camera is preferably downward compared to the orientation of the lens of the first camera. By orienting the lens of the first camera, which is used for vehicle number detection, downwards, it becomes easier to photograph and analyze the license plate located on the underside of the vehicle. (34) The aforementioned multiple cameras include a rear camera that photographs the area behind the vehicle and a front camera that photographs the area in front of the vehicle. The control unit may, when the target vehicle identified based on the first image information acquired by the front camera is the same as the target vehicle identified based on the second image information acquired by the rear camera, calculate a result of the possibility of dangerous driving by combining the result of the determination of the possibility of dangerous driving of the target vehicle determined based on the first image information acquired by the front camera and the result of the determination of the possibility of dangerous driving of the target vehicle determined based on the second image information acquired by the rear camera. By configuring the system in this way, it is possible to cumulatively count dangerous driving behaviors repeatedly performed by the same vehicle in front or behind, thereby enabling more accurate determination of dangerous driving behavior. (35) A drive recorder having one or more cameras and functioning as one of the systems described above. By configuring it in this way, it is possible to provide a drive recorder equipped with a mechanism to determine whether the captured vehicle is likely to be engaging in dangerous driving. (36) The aforementioned group of cameras may include at least a first camera with a wide field of view for capturing a wide area including at least the target vehicle, and a second camera with a narrow field of view for capturing at least the license plate number of the target vehicle, both housed in the same housing. This configuration makes it possible to provide a drive recorder that can accurately detect the license plate number of a target vehicle while capturing a wide area including that vehicle. (37) This program provides a computer that implements the functions of the control unit of any of the above systems.

[0224] The inventions described in (1) to (37) above can be combined in any way. For example, one may combine all or part of the configuration of the invention described in (1) with at least part of the configuration of at least one of the inventions described in (2) and onward. In particular, it is preferable to combine the invention described in (1) with at least part of the configuration of at least one of the inventions described in (2) and onward. Alternatively, one may extract any configuration from the inventions described in (1) to (37) and combine the extracted configurations. The applicant of this application intends to obtain rights to inventions that include these configurations. Furthermore, even if there are descriptions such as "in the case of..." or "when...", these are not meant to be descriptions that limit the configuration to that case or time. These are merely examples of better configurations, and the applicant intends to obtain rights to configurations that do not fall under these cases or times. Also, even if there is a sequence of descriptions, it is not limited to that order. Configurations with some parts deleted or the order rearranged are also disclosed, and the applicant intends to obtain rights to them as well. [Explanation of symbols]

[0225] 101F…Front drive recorder, 101R…Rear drive recorder, 102…Vehicle camera, 103…License plate detection camera, 105…Management terminal, 106…Management server, 107…User terminal, 301…Microcontroller, 302…Communication processing unit, 303…Control unit, 304…DVR controller, 305…Accelerometer, 306…SD card, 307…GNSS antenna, 308…Wireless communication antenna, 309…Speaker, 321…Image processing unit, 322…Counting unit, 324…Timekeeping unit, 326…Determination unit

Claims

1. A dashcam equipped with multiple cameras, It comprises a vehicle-photography camera that broadly photographs the area in front of or behind the vehicle, and a vehicle number detection camera that photographs the vehicle number of the target vehicle, The drive recorder is characterized by having a camera direction control unit, the camera direction control unit controlling the direction of the lens of the license plate detection camera to follow a predetermined path.

2. The drive recorder according to claim 1, wherein the camera direction control unit controls the direction of the lens of the license plate detection camera to always track a position at a predetermined distance in front of or behind the center of the vehicle.

3. The drive recorder according to claim 1 or 2, wherein the camera direction control unit controls the orientation of the lens of the license plate detection camera to always follow the direction of the center of the lane in front of or behind the vehicle.

4. The drive recorder according to any one of claims 1 to 3, wherein the camera direction control unit controls the direction of the lens of the license plate detection camera to always follow another vehicle or its license plate that is in front of or behind the vehicle.

5. The drive recorder according to any one of claims 1 to 4, further comprising a configuration that changes the orientation of the lens of the license plate detection camera by an actuator in conjunction with the installation position of the drive recorder.

6. The drive recorder according to any one of claims 1 to 5, wherein the vehicle camera is wide-angle and the license plate detection camera is a relatively telephoto camera.

7. The drive recorder according to any one of claims 1 to 7, wherein the vehicle camera is fixed to the housing, and the license plate detection camera is configured to change the direction of its lens.

Images