Terminal, imaging device, program, etc.

Abstract

To provide a terminal, a photographing device, and a program that enable image information received from the photographing device to be stored in a storage unit for the terminal.SOLUTION: A terminal includes: communication control means for receiving a plurality of pieces of image information transmitted from a photographing device located in a vehicle; and image information recording means for storing the plurality of pieces of image information received from the photographing device in storage means for the terminal when the state of the vehicle meets a predetermined recording condition.SELECTED DRAWING: Figure 30

Description

Technical Field

[0001] The present invention relates to a terminal, a photographing device, a program, and the like.

Background Art

[0002] As the background art of this technical field, there is Japanese Unexamined Patent Application Publication No. 2020-173844 (Patent Document 1). This publication describes that "the apparatus includes an observation unit that observes a situation in which a moving body operated by a user is placed, and a control unit that performs control for sending a message to the user to switch a mental state that the user will have when the observation result by the observation unit satisfies a predetermined condition to another mental state." (See the abstract).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 describes an apparatus having a function of monitoring the rear of a vehicle from an image and detecting a tailgating of a following vehicle, and having a function of sending a message to a driver when the tailgating of the following vehicle is detected. However, Patent Document 1 does not discuss a mechanism such as a terminal that stores image information transmitted from a photographing device when the state of the vehicle satisfies a specific recording condition. Therefore, the present invention provides a highly convenient function for the user, for example, provides a mechanism such as a terminal that stores image information transmitted from a photographing device when the state of the vehicle satisfies a specific recording condition.

Means for Solving the Problems

[0005] To solve the above problems, for example, the configuration described in the claims is adopted. This application includes a plurality of means for solving the above problems. For example, a terminal includes communication control means for receiving a plurality of pieces of image information transmitted from a photographing device disposed in a vehicle, and image information recording means for storing the plurality of pieces of image information received from the photographing device in storage means for the terminal when the state of the vehicle satisfies a predetermined recording condition.

Effect of the Invention

[0006] According to the present invention, a highly convenient function can be provided to a user. For example, a mechanism such as a terminal that stores a plurality of pieces of image information transmitted from a photographing device when the state of a vehicle satisfies a specific recording condition can be provided. Problems, configurations, and effects other than those described above will be clarified by the description of the following embodiments.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. The embodiments shown below are one of the embodiments providing the present invention, and the content of the present invention of the present application should not be construed as being limited based on the following description.

[0009] FIG. 1 is an example of an explanatory diagram for explaining the installation positions of the imaging devices 101 (101F, 101R) and the user terminal (terminal) 102 in the vehicle 10 (hereinafter, sometimes referred to as the host vehicle 10) of the present embodiment. FIG. 1(a) is a side view of the vehicle 10, and FIG. 1(b) is a top view of the vehicle 10.

[0010] As shown in FIGS. 1(a) and 1(b), the vehicle 10 has an imaging device 101F in front of the passenger compartment and an imaging device 101R behind the passenger compartment. The imaging device 101 is an electronic device having an imaging function, and for example, it may be a drive recorder, a front camera, a rear camera, or a side camera that cooperates with the drive recorder, or it may be a camera provided for security, monitoring, or other sensing purposes. Note that the vehicle 10 may be configured to include only the imaging device 101F for the front of the vehicle 10 or the imaging device 101R for the rear. Also, it may be provided with three or more imaging devices. The imaging devices 101F and 101R may simply be referred to as the imaging device 101 regardless of their installation positions.

[0011] The imaging device 101 (101F) shown in FIG. 1 is, for example, a drive recorder and is an imaging device disposed on the front side of the vehicle 10. The imaging device (drive recorder) 101F is, in this example, disposed on the front side inside the vehicle 10 (that is, in the passenger compartment) and is attached to a predetermined portion such as the windshield. The imaging device (drive recorder) 101F images the front outside the vehicle 10. The imaging device 101 (101R) is, for example, a rear camera and is an imaging device disposed on the rear side of the vehicle 10. The imaging device (rear camera) 101R is, in this example, disposed on the rear side inside the vehicle 10 and is attached to a predetermined portion such as the rear windshield. The imaging device (rear camera) 101R images the rear outside the vehicle 10.

[0012] The imaging device 101F and the imaging device 101R are connected via a cable 11. The cable 11 is a cable for electrically connecting the imaging device 101F and the imaging device 101R. The cable 11 includes, for example, a power line for supplying power from the imaging device 101F to the imaging device 101R and a signal line for transmitting various signals. The imaging device 101R operates by receiving power supply via the cable 11. The imaging device 101R has a function of taking images and a function of outputting image data indicating the taken images to the imaging device 101F via the cable 11. The imaging device 101F has a function of taking images, a function of recording image data indicating the taken images, and a function of recording the image data received from the imaging device 101R. As will be described later, the functions, configurations, etc. of the imaging device 101F and the imaging device 101R according to this embodiment are not limited to the above examples, and furthermore, the total number of imaging devices 101 may be one or three or more.

[0013] The user terminal 102 may be, for example, a mobile terminal such as a smartphone, a tablet, a mobile phone, a personal digital assistant (PDA), or a wearable terminal such as a glasses type, a wristwatch type, or a clothing type. It may also be a stationary or portable computer, or a server arranged on a cloud or a network. Further, as a function, it may be a VR (Virtual Reality) terminal, an AR (Augmented Reality) terminal, or an MR (Mixed Reality) terminal. Alternatively, it may be a combination of a plurality of these terminals. For example, a combination of one smartphone and one wearable terminal can function logically as one terminal. It may also be an information processing terminal other than these.

[0014] The user terminal 102 includes a processor 303 that executes an operating system, application programs, etc. respectively, a main memory device 301 such as a RAM (Random Access Memory), an auxiliary storage device 302 such as an IC card, a hard disk drive, an SSD (Solid State Drive), a flash memory (such as an SD card), a communication control unit 307 such as a network card, a wireless communication module, a mobile communication module, etc., an input device 304 such as a touch panel, a keyboard, a mouse, an input by voice input, motion detection by imaging of a camera unit, etc., and an output device 305 such as a monitor, a display, a speaker, etc. Note that the output device 305 may be a device or a terminal that transmits information for output to an external monitor, display, speaker, printer, device, etc.

[0015] Various programs and applications (modules) are stored in the main memory device 301, and each functional element of the entire system is realized by the processor executing these programs and applications. Note that these modules may be implemented in hardware by integration or the like. Also, each module may be an independent program or application, or may be implemented in the form of some sub-programs or functions in one integrated program or application.

[0016] In this specification, each module is described as the subject that performs the processing, but actually, the processor that processes various programs and applications (modules) executes the processing. Various databases (DBs) are stored in the auxiliary storage device. A "database" is a functional element (storage unit) that stores a data set so as to be able to respond to any data operation (for example, extraction, addition, deletion, overwrite, etc.) from a processor or an external computer. The implementation method of the database is not limited, and for example, it may be a database management system, a spreadsheet software, or a text file such as XML or JSON.

[0017] FIG. 2 is an example of a connection diagram of the imaging device 101 (101F, 101R), the user terminal 102, and the external server 103. The imaging device 101, the user terminal 102, and the external server 103 are each connected via the network 104. As will be described later, in the present embodiment, among the imaging devices 101F and 101R, the imaging device 101F is connected to the network 104 to perform communication. When data transmission and reception are required between the imaging device 101R and the network 104, or when data transmission and reception are required between the imaging device 101R and the user terminal 102, the imaging device 101F performs communication to relay data transmission and reception between the imaging device 101R and the network 104. However, a configuration in which the imaging device 101R directly transmits and receives data to and from the user terminal 102 may also be used. Note that the network 104 may be wired or wireless, and each user terminal 102 can transmit and receive information via the network 104. The network 104 is not limited to the Internet, and may be a combination of networks with different protocols.

[0018] As described above, the imaging device 101 is a device mounted on, for example, the vehicle 10 and records images during driving. However, although the vehicle 10 to be mounted is a four-wheel automobile in this embodiment, it is not limited to four-wheel automobiles, and any vehicle on which the imaging device 101 can be installed may be used. For example, large transport vehicles with four or more wheels such as buses and trucks, two-wheel vehicles such as motorcycles and bicycles, and other vehicles may be targeted. Also, for example, vehicles of transportation means such as trains, monorails, and linear motor cars may also be targeted for mounting. The imaging device 101 can transmit various information obtained and information regarding the result of event determination described later to the user terminal 102 via the network 104.

[0019] The user terminal 102 is a terminal used by, for example, a user who drives an automobile. The user terminal 102 is connected to the imaging device 101 via an in-vehicle network 104 such as a mobile data communication network or the Wi-Fi (registered trademark) of the automobile, and can operate the imaging device 101. In addition, the user terminal 102 is connected to the external server 103 via these networks 104 and can transmit various information to the external server 103.

[0020] FIG. 3 is an example of the hardware configuration of the user terminal 102. The user terminal 102 is composed of terminals such as smartphones, tablets, notebook PCs, desktop PCs, etc. In the main storage device 301, programs and applications such as an image information recording module 310, a warning module 311, a display control module 312, a photographing device cooperation module 313, and a photographing device control module 314 are stored. By the processor 303 executing these programs and applications, each functional element of the user terminal 102 is realized.

[0021] The image information recording module 310 stores the image information transmitted from the photographing device 101 via the network 104 in the auxiliary storage device 302 for the terminal. The auxiliary storage device 302 is an example of the storage means for the terminal. The storage means for the terminal means the storage means secured for the user terminal 102 and can be used by the user terminal 102 for data recording. Whether the storage means for the terminal is an internal storage means of the user terminal 102 or an external storage means of the user terminal 102 does not matter. The external storage means of the user terminal 102 may be, for example, a storage means accessed by the user terminal 102 using communication means, such as a storage means secured on a network.

[0022] The warning module 311 uses an output device 305 such as a display or a speaker to give warnings to the user regarding various events determined by the photographing device 101. The display control module 312 controls the display content and display layout on the output device 305 such as the display of the user terminal 102. The photographing device cooperation module 313 executes the connection and functional cooperation with one or more photographing devices 101. The camera control module 314 executes control from the user terminal 102 for the imaging device 101 that is connected and functionally coordinated by the imaging device cooperation module 313. The specific functions of these individual modules will be described in detail with reference to other drawings.

[0023] As described above, the main storage device 301 stores applications, programs, etc. (modules) for displaying and recording the image information transmitted from the imaging device 101. Such applications may be denoted as APP in the figure.

[0024] The display control module 312 displays an operation area and a display area on the output device 305 of the user terminal 102. The operation area is an area for displaying operation elements (operation parts) for receiving operations and settings for the user terminal 102 and the imaging device 101, for example. The display area is an area for displaying the image information transmitted from the imaging device 101, for example. The operation area and the display area will be described in more detail in relation to FIG. 14. Also, various examples of the operation area and the display area are shown in other drawings.

[0025] The auxiliary storage device 302 stores the setting information for each module stored in the main storage device 301 and the image information received and stored from the imaging device. In this specification, the image information means, for example, still image information, moving image information, etc. The image information recording module 310 stores still images in still image file formats such as JPEG, PNG, GIF, etc., and also stores moving images in moving image file formats such as MOV, MP4, AVI, WMV, WemM, etc.

[0026] FIG. 4 is an example of the hardware configuration of the imaging device 101. The imaging device controller 404 is connected to a camera 402, an acceleration sensor 405, an SD card 406, a GNSS sensor 407, and other sensors (not shown), and controls them. Also, based on the information received from these elements, the imaging device controller 404 generates status information indicating the status of the imaging device controller 404. Further, the imaging device controller 404 transmits the generated status information to the microcomputer 401.

[0027] The microcomputer 401 performs various processes such as the following, for example. · Controls the communication processing unit 414. · Notifies the imaging device controller 404 of the position information acquired by the communication processing unit 414. · Converts the status information acquired from the imaging device controller 404 and the position information acquired from the communication processing unit 414 into data of, for example, 16 bytes. · Performs sleep control during secondary battery operation. · Executes a test mode during manufacturing and testing.

[0028] The microcomputer 401 is programmable, and a program for executing various processes as described above is stored in the microcomputer 401. By the processing unit of the microcomputer 401 executing this program, the various processes described above are realized. In the present embodiment, the processing unit of this microcomputer 401 or the microcomputer 401 itself may also be referred to as a control unit. Note that the front imaging device 101F receives image information from the rear imaging device 101R via a network 104 such as a cable 11, and the imaging device controller 404 and the microcomputer 401 of the front imaging device 101F execute various processes on the received image information. However, the rear imaging device 101R may also be configured to include an imaging device controller 404 and a microcomputer 401, and the rear imaging device 101R itself may perform the above-described processes on the image information.

[0029] Note that the imaging device controller 404, the microcomputer 401, and the communication processing unit 414 can each be implemented by an integrated circuit (chip) such as a SoC (System on a chip), or a configuration in which a plurality of these are integrated into one chip may also be used. In particular, in this embodiment, since the imaging device controller 404 and the microcomputer 401 operate in cooperation, they may be implemented together on one chip.

[0030] Also, a configuration may be adopted in which the microcomputer 401 executes some functions of the imaging device controller 404, or a configuration may be adopted in which the imaging device controller 404 executes some functions of the microcomputer 401. In this embodiment, the imaging device controller 404 and the microcomputer 401 are collectively referred to as the control unit 403. The control unit 403 may be a single chip, or may be configured with a plurality of chips in which the functions of the imaging device controller 404 and the microcomputer 401 are separated.

[0031] Note that by mounting an AI processing function for performing pre-processing on the image sensor of the camera 402, the following outputs can be performed, for example, together with or instead of generating a normal captured image. The AI processing function has a function of performing image recognition based on the video captured by these cameras. · Output the target object from the image as metadata. · Output an image in an ISP (Image Signal Processor) output format such as YUV or RGB. · Output an image with only a specific area cut out.

[0032] By performing pre-processing on the image sensor side in this way, it becomes possible to reduce the data amount and perform real-time tracking of the target object by high-speed AI processing. For example, when a plurality of imaging devices 101 (101F, 101R) are provided inside a vehicle, and each of the imaging devices 101F, 101R is equipped with an AI processing function for performing pre-processing on the image sensor of the camera 402, each of the imaging devices 101F, 101R can execute video processing and image processing independently. Therefore, each of the imaging devices 101F, 101R can create image information with ideal image quality. In addition, it is also possible to avoid the concentration of the computational burden of video processing and image processing on one of the imaging devices 101F, 101R.

[0033] For example, when the imaging device 101F is a front camera and the imaging device 101R is a rear camera, and each performs video processing, for example, compared with a system that processes the videos of the front camera and the rear camera using only the front camera, the front camera processes only the front video, resulting in reduced load and stable operation. The rear camera processes the video with an engine mounted on itself, and can realize ideal image quality.

[0034] As the network 104 responsible for communication between the imaging device 101F and the imaging device 101R, it is preferable to use, for example, a communication standard of digital transmission (Ethernet (registered trademark) method). Connect the imaging device 101F and the imaging device 101R with a cable 11 compatible with the Ethernet standard. For example, when transmitting image information from the imaging device 101R to the imaging device 101F, even with the same FULL HD image quality, there is no degradation and it is possible to record clearly. For example, such digital transmission enables clear and sharp rear video recording compared to the conventional analog transmission model.

[0035] The acceleration sensor 405 detects acceleration and generates acceleration information. For example, when a sudden change in acceleration occurs in response to an impact, sudden steering, sudden braking, etc., the control unit 403 detects the occurrence of an event. The control unit 403 can infer the situation of an accident or the like that has occurred in an automobile or the like by analyzing the change in this acceleration information. The SD card 406 is a storage device that records various types of information such as image information and recording information corresponding to the image information. Note that it is not limited to an SD card, and any configuration capable of storing data may be used, but a flash memory with high vibration resistance is preferred for in-vehicle use. In addition, when there are multiple imaging devices 101F and 101R in the vehicle 10 and they can share the storage device, it is not necessary for all the imaging devices 101F and 101R to have a storage device such as the SD card 406.

[0036] For example, when the imaging device 101F is a drive recorder and the imaging device 101R is a rear camera without a recording function, the rear camera does not necessarily have to have a storage device such as the SD card 406. Furthermore, configurations such as determining the state of the vehicle described later, storing the image information captured by the imaging devices 101F and 101R, and transmitting the image information to the user terminal 102 can be aggregated in the front-side drive recorder, and those configurations can be omitted in the rear camera. The rear camera may be configured to at least transmit the image information to the drive recorder.

[0037] Regarding other configurations, functions, etc., when they can be shared among the multiple imaging devices 101, it is also possible to omit some or all of the configurations, functions, etc. for each imaging device. In this embodiment, basically, the imaging device 101F performs advanced functions such as image processing and transmission processing to the user terminal 102 and has a configuration with a storage device, while the rear side has a configuration with a simple imaging function. However, conversely, a configuration may be adopted in which advanced functions are aggregated on the rear side and the front side is a front camera that only performs simple imaging.

[0038] The communication processing unit 414 transmits the information of the imaging device 101 to the user terminal 102 via the wireless communication antenna 408 and via the network. The GNSS sensor 407 has a position information acquisition unit implemented by an integrated circuit or the like, and acquires position information based on signals received from a GNSS antenna corresponding to GNSS (Global Navigation Satellite System), such as GPS (Global Positioning System). Further, the position information acquisition unit can also acquire time information, moving speed information, and moving direction information.

[0039] The imaging device 101 has a power source 411 connected to the vehicle power source 410. The power source 411 supplies the electric power received (or sometimes called power reception) from the vehicle power source 410 to the imaging device controller 404, and also charges the secondary battery 412.

[0040] The secondary battery 412 is a rechargeable battery such as a lithium-ion battery, nickel-cadmium battery, nickel-metal hydride battery, or lead-acid battery. However, a non-rechargeable primary battery may also be used. When the accessory power source (ACC power source) is turned off, or when the power supply from the vehicle power source 410 to the power source 411 of the imaging device 101 is interrupted due to a failure such as disconnection or wiring disconnection caused by an accident, or when the power source 411 fails and power supply becomes impossible, the secondary battery transmits (or sometimes called power transmission or power supply) power to the microcomputer 401, communication processing unit 414, etc., and maintains at least some functions of the imaging device 101.

[0041] The control unit 403 acquires image information from the camera 402 of the imaging device 101 and stores it in a storage device such as an SD card 406. In this specification, "storage" is sometimes called "recording". The control unit 403 of the imaging device 101 further detects the state of the host vehicle 10 based on the image information from the camera 402, such as the distance or positional relationship with at least one of the vehicle in front of the host vehicle 10 or the vehicle behind the host vehicle 10, the time length that one of the vehicles in front or behind enters a determination area, other reference areas or proximity areas described later, the speed of the host vehicle, and the speed of one of the vehicles in front or behind.

[0042] The control unit 403 determines whether the detected state of the host vehicle 10 satisfies specific event recording conditions described below. The control unit 403 can also set individual event recording conditions for each of a plurality of different events described below. The control unit 403 can also use the functions of a forward vehicle collision warning system (FCWS) or a rear vehicle collision warning system (RCWS) to measure, for example, the length of time that a vehicle either ahead or behind has entered a specific reference area (proximity area).

[0043] FIG. 5 is an example of the hardware configuration of the external server 103. In FIG. 5, the external server 103 is shown as an example of a server operated to provide an SNS service, but it may also be an external server of other private companies, public institutions, etc. The external server 103 includes a main storage device 501, an auxiliary storage device 502, a processor 503, an input device 504, an output device 505, and a communication control unit 506, and a bus that interconnects these components.

[0044] The main storage device 501 stores a command reception module 510 and a content posting module 511. By the processor 503 executing these programs and applications, each functional element of the external server 103 is realized.

[0045] The command reception module 510 receives and processes commands from users who use the SNS service. The command reception module 510 stores data provided by the user, such as image data or video data, etc., in the auxiliary storage device 502 as content information. The content posting module 511 outputs the processing results of commands from users by the command reception module 510 in a state accessible by other users who use the SNS service.

[0046] According to the result of processing a command from the user, the content posting module 511 can also output the content information stored in the auxiliary storage device 502 in a state accessible by other users who use the SNS service. The auxiliary storage device 502 stores various program information 520 and content information 530 related to the SNS service.

[0047] Subsequently, the operation of this embodiment will be described. Among the operations described below, the operation of the user terminal 102 may be realized based on an application (a smartphone application for a photographing device) installed in the user terminal 102. FIG. 6 is an example of a first flow 60 of image recording, event notification information output, and index information addition by the photographing device 101.

[0048] The control unit 403 of the photographing device 101 executes the following processing according to the first flow 60. Before explaining the first flow 60, the processing in the previous stage thereof will be explained. As a setting of the photographing device 101, there are a setting (for example, Wi-Fi ON mode) in which the photographing device 101 is connected to the user terminal 102 via an in-vehicle network 104 such as Wi-Fi (registered trademark) from when the power of the photographing device 101 is turned on, and a setting (for example, Wi-Fi OFF mode) in which the photographing device 101 is not connected to the user terminal 102 when the power is turned on. The default setting is the latter. In the latter case, it is possible to connect the photographing device 101 by operating the user terminal 102 of the user. In the following description, the setting of the photographing device 101 may be either, but it is assumed that the photographing device 101 and the user terminal 102 are connected.

[0049] With these settings, the photographing device 101 performs data transmission automatically / manually via Wi-Fi (registered trademark) connection, and the user of the user terminal 102 can check the image information photographed by the photographing device 101 at hand and directly post the confirmed image information to an SNS or the like in an appropriate manner. Furthermore, through the following process, the user terminal 102, such as a smartphone, performs video recording by automatic transmission from the imaging device 101. The automatic transmission performs video recording for, for example, 30 to 60 seconds after sensor detection, such as video recording of drifting driving. Details regarding these points will be described later.

[0050] Also, when a plurality of imaging devices 101 are present inside the vehicle as described above, only one imaging device 101, for example, the imaging device 101F, executes the first flow 60, and the other imaging devices 101, for example, the imaging device 101R, may be configured to only send image information to the imaging device 101F that executes the first flow 60 via, for example, the cable 11 in FIG. 1. In that case, the imaging device 101F that executes the first flow 60 can record the image information of the imaging device 101R and also communicate with the user terminal 102 to send the image information. In the following embodiments, the imaging device 101 as the imaging device 101F is connected to the user terminal 102 through communication to perform communication. When data transmission and reception are required between the imaging device 101R and the user terminal 102, the imaging device 101F performs communication to relay data transmission and reception between the imaging device 101R and the user terminal 102.

[0051] Subsequently, the first flow 60 will be described. The control unit 403 acquires (S600) and stores (S610) image information (having year / month / day, time, camera identification number, etc.) from the camera 402. The steps described below are applicable to various different image recording processes and video recording processes by the imaging device 101, and specific descriptions of the image recording process and the video recording process are omitted here.

[0052] The communication processing unit 414 sequentially transmits a plurality of pieces of image information of the imaging device 101 to the user terminal 102 that has already established a connection via, for example, an in-vehicle network 104 such as the Wi-Fi (registered trademark) of the vehicle, based on, for example, the event recording conditions described later (S620). Note that the event recording conditions will be described in detail in FIGS. 6, 25, etc. When there are two imaging devices 101F and 101R in the vehicle, among the multiple pieces of image information transmitted from the imaging device 101 to the user terminal 102, for example, one of the imaging devices 101F may transmit only the image information to be displayed on the preview screen described later among the image information captured by the imaging devices 101F and 101R, or may always transmit both pieces of image information captured by the imaging devices 101F and 101R.

[0053] The control unit 403 measures driving information such as the speed and acceleration of the host vehicle 10 by the acceleration sensor 405 (S630), and stores this speed and acceleration information (S640). Step S620 may be performed after steps S630 and S640. Also, in another more concise embodiment, it is possible to omit S620. When S620 is omitted, the imaging device 101 transmits the image information along with the start of the following S640.

[0054] Note that the acceleration information in steps S630 and S640 is the component in the longitudinal direction of the traveling direction of the vehicle 10 acquired from the acceleration sensor 405, but it is also possible to use the value synthesized from three axes other than the gravitational acceleration. More specifically, the control unit 403 generates recording information by summarizing the acquired speed and acceleration and the acquired information from various sensors of the imaging device 101. The control unit 403 associates the recording information with the image information acquired from the camera 402 and stores it in the SD card 406 (storage unit) (S640). For example, the control unit 403 stores the year, month, day, time, camera identification number, front or rear type, driving information (speed, acceleration, position coordinates, etc.) in association with each other in the storage means.

[0055] The control unit 403 analyzes the acquired image information and executes a vehicle determination process for determining whether the vehicle 10 satisfies the event recording condition (S650). The event recording condition and the vehicle determination process will be described later with reference to another figure. The event recording condition is a condition determined based on the state of the vehicle 10, for example, a condition such as whether there is an approaching vehicle in either the front or the rear. However, as will be described later, the event recording condition may be other conditions.

[0056] Based on the acquired image information, the control unit 403 determines whether the state of the vehicle 10 satisfies the event recording condition (S660). If the state of the vehicle 10 does not satisfy the event recording condition (No in S660), the control unit 403 repeats the steps from S630 to S650. If the state of the vehicle 10 satisfies the event recording condition (Yes in S660), the control unit 403 transmits event notification information (notification information for notifying that the recording condition has been satisfied, or information for causing the terminal to store the plurality of transmitted image information) to the user terminal 102. The event notification information is notification information for notifying that the own vehicle 10 has satisfied the recording condition, and is data that serves as a trigger for starting the recording of the image information transmitted from the imaging device 101 by the image information recording module 310 of the user terminal 102 when the own vehicle 10 satisfies the event recording condition.

[0057] Particularly advantageously, in this embodiment, the vehicle determination process (S650) may perform an event determination on the image recorded by the imaging device 101, particularly a determination of dangerous driving such as tailgating from behind. In the vehicle determination process (S650), the control unit 403 executes, for example, a vehicle determination process of analyzing the acquired image information and determining whether there is a vehicle in front of or behind the vehicle 10.

[0058] The determination of whether there is a nearby vehicle follows the detection of the forward vehicle collision warning system (FCWS) or the rear vehicle collision warning system (RCWS), but is not limited thereto. For example, when the size of the vehicle in the image captured by the camera 402 is equal to or greater than a predetermined size, for example, when approaching within 30 m, proximity determination can be made. Alternatively, for example, proximity determination can also be made by measuring the distance with a distance measuring sensor using radio waves, sound waves, etc. by a method other than image processing.

[0059] By the above-described vehicle determination process (S650), it becomes possible to start event recording on the terminal after an event is detected by a forward vehicle collision warning system (FCWS) or a rear vehicle collision warning system (RCWS). Thereby, for example, when the rear vehicle collision warning system (RCWS) detects that a following vehicle is approaching abnormally, the terminal starts recording, and when the forward vehicle collision warning system (FCWS) detects that a preceding vehicle has suddenly braked, the terminal starts recording. According to this embodiment, it is possible to reliably store image information of an event in which the image information after the occurrence of the event is more important.

[0060] Note that the proximity range for performing the stalking determination can also be made variable. For example, the proximity range setting is changed according to the speed of the host vehicle 10. For example, when the traveling speed of the host vehicle 10 is high, the inter-vehicle distance increases, so the proximity range is set wide, and when the traveling speed is low, the inter-vehicle distance decreases, so the proximity range can be set narrow. A more specific example of the vehicle determination process will be described in more detail later in association with FIG. 13 in particular.

[0061] As a result of the vehicle determination, when the state of the vehicle 10 does not satisfy the event recording condition (No in S660), the control unit 403 returns the first flow 60 to S630. As a result of the vehicle determination, when the state of the vehicle 10 satisfies the event recording condition (Yes in S660), the control unit 403 adds index information related to the event to the image information from the camera 402 to be stored in the storage unit of the imaging device 101 such as the SD card 406 (S670). The index information can be created based on information such as the type of the occurred event, numerical values characteristic of each event, for example, a sudden change in acceleration, the distance to the preceding and following vehicles, the GPS coordinates of the location where the event occurred, and the event occurrence date and time.

[0062] Index information related to an event facilitates calling, from the imaging device image information stored in the storage unit of the imaging device 101 such as the SD card 406, the imaging device image information related to individual events stored in the storage unit of the user terminal 102. After adding (S670) index information related to the event to the image information to be stored in the storage unit of the imaging device 101, the control unit 403 transmits event notification information to the user terminal 102 (S680).

[0063] The control unit 403 of the imaging device 101 can use the index information attached to the imaging device image information already recorded on the SD card 406 of the imaging device 101 as event notification information to be transmitted to the user terminal 102 or as part of the event notification information. In that case, since the control unit 403 of the imaging device 101 does not need to create the event notification information to be transmitted to the user terminal 102 or such a load is reduced, compared with the case of providing a dedicated configuration for creating the notification information for starting the recording of the event video on the auxiliary storage device 302 for the user terminal 102, it is significantly advantageous from the viewpoints of the arithmetic load and the manufacturing cost.

[0064] Also, when using the index information as event notification information, for example, the imaging device control module 314 of the user terminal 102 can call, from the imaging device 101, the imaging device image information recorded on the SD card 406 of the imaging device 101 that has the index information corresponding to the event notification information sent to the user terminal 102.

[0065] In the first flow 60, the index information is configured to be added to the captured image information after it is determined that the index information satisfies the event recording conditions (S670), but it may also be configured to be added when storing the image in the imaging device in S610. Also, a configuration that does not use the index information can be adopted.

[0066] Based on this comparison result, the imaging device control module 314 can instruct the imaging device 101 to search for and extract the imaging device image information (second image information) recorded on the SD card 406, etc. of the imaging device 101 corresponding to the event video recorded in the auxiliary storage device 302 of the user terminal 102, and list them if there is a plurality of corresponding imaging device image information. Then, the imaging device control module 314 can instruct the imaging device 101 to transmit the searched imaging device image information to the user terminal 102. Also, when the image information captured by the imaging device 101 is stored on the cloud, it can also be searched and displayed based on the index information.

[0067] The imaging device image information is, for example, image information including the event of the event video recorded in the auxiliary storage device 302 of the user terminal 102, and is image information with a longer time, or higher image quality, or both longer time and higher image quality. Also, the imaging device image information may include audio information, which is particularly preferable when the image information recording module 310 stores image information without audio information in the auxiliary storage device 302.

[0068] In particular, the imaging device image information may be, for example, image information from a period before the time when the state of the vehicle 10 satisfied the event recording condition, that is, image information from before the event recorded in the auxiliary storage device 302 of the user terminal 102. The imaging device image information may further be image information captured during a period including before and after the time when the state of the vehicle 10 satisfied the event recording condition.

[0069] Also, the event notification information may be different data depending on the event to be recorded on the user terminal 102, the settings of the user terminal 102, and the settings of the imaging device 101. For example, when the rear abnormal approach is an event to be recorded on the user terminal 102, the control unit 403 can create event notification information based on, for example, data related to the distance from the vehicle behind. Also, for example, when a forward abnormal approach is an event to be recorded by the user terminal 102, the control unit 403 can create event notification information based on, for example, data related to the distance from the vehicle ahead.

[0070] By transmitting event notification information related to a rear abnormal approach or event notification information related to a forward abnormal approach, the control unit 403 can notify the user terminal 102 of the occurrence of events such as a rear sway or a forward sway. The control unit 403 can also select whether to transmit event notification information to the user terminal 102 for each event. The event notification information does not necessarily have to be common to all events.

[0071] If the control unit 403 has not received the reception confirmation information of the event notification information from the user terminal 102 after transmitting the event notification information (No in S690), the control unit 403 returns the first flow 60 to S680, and the control unit 403 transmits the event notification information to the user terminal 102 again. When the control unit 403 receives the reception confirmation information from the user terminal 102 in S690 (Yes in S690), the control unit 403 determines whether to end the first flow 60 (S695).

[0072] If the control unit 403 does not end the first flow 60 (No in S695), the control unit 403 returns to S630 of the first flow 60. If the control unit 403 ends the first flow 60 (Yes in S695), the control unit 403 ends the first flow 60. Examples of the case where the control unit 403 ends the first flow 60 (Yes in S695) include, for example, when the imaging device cooperation module 313 releases the connection with the user terminal 102 or when the imaging device control module 314 outputs an end command for the first flow 60.

[0073] Note that in the example of FIG. 6, regardless of the occurrence or non-occurrence of an event and the notification or non-notification of notification information to the user terminal 102, the control unit 403 continues to execute image recording until the end of the first flow 60 (S610). In particular, by S630 - S660, the control unit 403 can determine whether at least one of the event recording conditions based on at least the impact generated in the vehicle 10 and the event recording conditions based on the approach of other vehicles to the host vehicle 10 is satisfied. Then, the control unit 403 can transmit an event recording notification that triggers the recording of the event video to the auxiliary storage device 302 for the user terminal 102 to the user terminal 102 without requiring an operation by the user.

[0074] According to the present embodiment, for example, even if a driver detects tailgating of a following vehicle by a sensor (VOICE: Rear Abnormal Approach Warning) while driving the host vehicle 10 and the driver or a passenger becomes anxious or shaken and exclaims "Whoa! Being tailgated!", since the rear video after detection is automatically transmitted to and stored in the smartphone (the drive recorder main body also records the front and rear videos), it is possible to give the driver and the passenger a sense of security by checking the smartphone and saying "It's properly recorded and can be used as evidence in case of need".

[0075] FIG. 7 is an example of a second flow 70 of image recording by the imaging device 101, event notification information output, and index information addition. Since each step of S700, S720 to S750, S756, S760 to S795 of the second flow 70 corresponds to each step of S600, S620 to S650, S670, S660 to S695 of the above-described first flow 60, the description thereof is omitted. Different from the example of FIG. 6, in the example of FIG. 7, the control unit 403 acquires image information from the camera 402 (S600), and transmits the acquired image information to the user terminal 102 without storing it in the storage unit of the imaging device 101 until the imaging device recording conditions are satisfied (S720).

[0076] Thereafter, similar to the example of FIG. 6, the control unit 403 measures driving information such as the speed and acceleration of the host vehicle 10 (S730), stores this speed and acceleration information (S740), and performs vehicle determination processing (S750) for determining whether the vehicle 10 satisfies the camera recording conditions and the event recording conditions.

[0077] After the vehicle determination processing (S750), when the vehicle 10 satisfies the camera recording conditions (the first recording conditions) (Yes in S752), the control unit 403 records the image information acquired from the camera 402 in the storage unit of the camera device 101 (S754), further adds index information to the image information (S756), and proceeds to step S760. When the vehicle 10 does not satisfy the camera recording conditions (No in S752), the control unit 403 directly proceeds to step S760.

[0078] Thereafter, the control unit 403 executes steps S760, S780, S790, and S795 in the same manner as the first flow 60 of FIG. 6. When the host vehicle 10 satisfies the event recording conditions (the second recording conditions) (Yes in S760), the control unit 403 surely transmits event notification information for storing a plurality of image information in the auxiliary storage device 302 of the user terminal 102 to the user terminal 102, for example, after the time point when the event recording conditions are satisfied (S780, S790).

[0079] In another embodiment, for example, when there is an approach of a following vehicle to the vehicle 10 as an event recording condition, the control unit 403 issues an alarm using an effect sound such as "beep" or voice such as "Please pay attention to the rear" or "Dangerous driving to the rear has been detected" as will be described later with reference to FIG. 27, and can also transmit information for storing a plurality of image information in the auxiliary storage device 302 of the user terminal 102.

[0080] In the example of FIG. 7, when the vehicle 10 satisfies the imaging device recording conditions, the control unit 403 records the image information in the storage unit of the imaging device 101. Therefore, compared with the example of FIG. 6, the imaging device 101 can use the capacity of the storage unit efficiently. Accordingly, for example, the image information recorded in the storage unit of the imaging device 101 can be recorded with a higher image quality than the image information recorded in the storage unit of the imaging device 101 in the example of FIG. 6.

[0081] In the example of FIG. 7, the control unit 403 can separately set the imaging device recording conditions for recording the image information in the storage unit of the imaging device 101 and the event recording conditions for recording the image information in the auxiliary storage device 302 of the user terminal 102. Further, preferably, each setting can be changed according to the content of each event.

[0082] Thereby, for example, the control unit 403 can record in the storage unit of the imaging device 101 image information related to various dangerous driving such as "abnormal approach from behind", "abnormal approach from the front", "dangerous lane change", "sudden approach", "dangerous cut-in" by other vehicles from the left and right lanes, "excessive honking" or "abnormal passing / high beam" by other vehicles traveling around the host vehicle 10, and "overtaking obstruction" by the vehicle ahead. Also, the user terminal 102 can record only the image information related to, for example, "abnormal approach from behind" among those dangerous driving situations.

[0083] In yet another embodiment, the control unit 403 records the imaging device image after acquiring the imaging device image as in the example of FIG. 6 (S600, S610). Further, when an event corresponding to the imaging device recording conditions of S752 in the example of FIG. 7 occurs, for example, the recording image quality of the image information can be changed according to the priority for each event, and it is also possible to record higher-quality image information than the image information recorded in S610.

[0084] By appropriately combining the embodiments of FIGS. 6 and 7, the following operations become possible. · The imaging device 101 performs event recording and transmits information serving as a trigger for executing the event recording to cause the user terminal 102 to also execute event recording. · While the imaging device 101 continues to sequentially transmit video information to the user terminal 102, if it detects the occurrence of an event, it transmits information serving as a trigger for executing event recording to the user terminal 102. Note that sequentially transmitting video information means transmitting the frames (pictures) of the image information captured by the imaging device 101 in chronological order in this embodiment. The image information transmitted thereby is displayed as an operation on the preview screen.

[0085] · For each event type, it is possible to set whether to transmit trigger information for causing the user terminal 102 to execute event recording. Trigger information is transmitted only for the set types of events, and not for others. Depending on the type of event, information such as video is recorded only by the imaging device 101, or recorded by both the imaging device 101 and the user terminal 102. In this case, it is more desirable to transmit trigger information for a rear-end collision. · Since it is possible to cause the user terminal 102 to issue a warning as well, switch between transmitting trigger information for performing event recording and transmitting trigger information for outputting a warning. Note that in this embodiment, the warning may be used to mean a notification with a lower level than an alarm.

[0086] · Regarding the transmission of image information from the imaging device 101 to the user terminal 102, switch between a setting to always continue transmitting when connected by WiFi or the like, and a setting to start transmitting when detecting the occurrence of a specific event. For example, only in the case of a rear-end collision does the imaging device 101 transmit image information, and the user terminal 102 executes event recording (and may further display) of the transmitted image information.

[0087] FIG. 8 is an example of an image information recording flow 80 of the imaging device 101 by the user terminal 102. In the example of FIG. 8, the imaging device cooperation module 313 of the user terminal 102 searches for and connects to the corresponding imaging device 101 (S800). The connection to the corresponding imaging device 101 may be executed when the power of the corresponding imaging device 101 is turned on, or may be executed when the image information recording module 310 of the user terminal 102 receives an input of a connection command to the imaging device 101. For details on this point, refer to the description already detailed in relation to FIG. 6.

[0088] In this embodiment, at least one of the imaging devices 101 is described as being, for example, a drive recorder. However, the imaging device 101 is not limited to a drive recorder, and the imaging device cooperation module 313 of the user terminal 102 can also establish connections with three or more imaging devices 101. For example, the imaging device cooperation module 313 of the user terminal 102 can also establish a connection with an imaging device provided at a position where it can determine the centering from the side of the own vehicle 10.

[0089] If the connection between the user terminal 102 and the imaging device 101 is not established (NO in S805), the imaging device cooperation module 313 of the user terminal 102 repeats S800. When the connection between the user terminal 102 and the imaging device 101 is established (Yes in S805), the user terminal 102 receives image information from the imaging device 101 using the communication control unit 307 and the display control module 312 (S810). At this point, the user terminal 102 does not store the image information from the imaging device 101. In another embodiment, it is also possible to omit S810. When S810 is omitted, the reception of image information from the imaging device is performed, for example, after S820 described later.

[0090] The display control module 312 of the user terminal 102 displays the image information from the imaging device 101 on an output device 305 such as the display of the user terminal 102 (S815). The output device 305 such as the display of the user terminal 102 is hereinafter sometimes simply referred to as the "screen". In another embodiment, the display control module 312 may omit the execution of S815, or may selectively execute it for each event.

[0091] The imaging device cooperation module 313 of the user terminal 102 receives event notification information from the imaging device 101 via the communication control unit 307 (S820). In another embodiment, S820 may be omitted. When S820 is omitted, for example, an image analysis module (not shown) of the user terminal 102 may analyze an image received from the imaging device, and a vehicle determination module (not shown) of the user terminal 102 may determine the state of the host vehicle 10.

[0092] When an event occurs for the host vehicle 10 and the state of the host vehicle 10 satisfies the event recording condition (Yes in S825), the user terminal 102 checks whether there is a warning setting for the target event (S830). In another embodiment, the reception of the event notification information in S820 may be used as a condition for the state of the host vehicle in S825 to satisfy the event recording condition (Yes in S825).

[0093] The event is related to the event determined in the vehicle determination process S650 of the first flow 60, which has already been described with reference to FIG. 6. This event can be various dangerous driving situations such as "abnormal approach from behind", "abnormal approach from the front", "dangerous lane change", "sudden approach", "dangerous cut-in" by other vehicles from the left and right lanes, "excessive honking" or "abnormal passing / high beam" by other vehicles traveling around the host vehicle 10, and "overtaking obstruction" by the vehicle in front. The control unit 403 of the imaging device 101 can detect a rear sway, a front sway, a side sway, a sudden acceleration of the rear vehicle, or a sudden brake of the front vehicle by determining "abnormal approach from behind", "abnormal approach from the front", "dangerous lane change", and "sudden approach". Furthermore, the vehicle determination process S650 can also determine "tire puncture" and "wheel drop / deviation".

[0094] The image information recording module 310 of the user terminal 102 may determine that the event recording condition is satisfied by receiving event notification information from the imaging device 101. In another embodiment, the image information recording module 310 may perform a process similar to the vehicle determination process S650 of the first flow 60 in FIG. 6 on the image information received from the imaging device 101 to determine whether the state of the host vehicle 10 satisfies the event recording condition.

[0095] When there is a setting to issue a warning for the occurred event (Yes in S805), the warning module 311 of the user terminal 102 issues a warning according to each event. For example, when receiving event notification information about the approach of a vehicle behind, the user terminal 102 may output a voice warning such as "Please pay attention to the rear" from an output device 305 such as a speaker (S835).

[0096] After the warning is output or when there is no setting to issue a warning for the occurred event (No in S805), the image information recording module 310 of the user terminal 102 checks the recording setting of the event (S840).

[0097] When there is a setting to record image information for the occurred event (Yes in S840), the user terminal 102 uses the image information recording module 310 to store the image information of the occurred event in the auxiliary storage device 302 for a set length of time (the length of time of a plurality of pieces of image information to be stored) (S845, S850).

[0098] After the image information is recorded or when there is no setting to record image information for the occurred event (No in S840), the image information recording module 310 of the user terminal 102 checks whether to end the image information recording flow 80 (S860). When continuing the image information recording flow 80 (No in S860), the image information recording module 310 of the user terminal 102 executes S815. When ending the image information recording flow 80 (Yes in S860), examples where the image information recording module 310 ends the image information recording flow 80 (Yes in S860) include, for example, when the shooting device connection module 313 disconnects the connection with the user terminal 102 or when the image information recording module 310 receives an end command for the image information recording flow 80.

[0099] As a result of the above steps, in the example of FIG. 8, the shooting device 101 transmits image information to a user terminal 102 such as a smartphone, and the user terminal 102 can store the image information transmitted from the shooting device 101 in the auxiliary storage device 302. In particular, it becomes possible to automatically transfer and store the video to the smartphone in relation to weaving driving, strong impact, sudden braking, sudden steering, etc.

[0100] In particular, immediately after an event actually occurs, the user terminal 102 can store the video of the occurred event. Such video recording after the occurrence of an event is particularly important for dangerous driving events such as reckless driving. According to this embodiment, the user terminal 102 and the shooting device 101 can provide important information after such an event occurs to the user without the user operating the user terminal 102 himself / herself. Also, since the image information is stored in the user terminal 102, the user is not forced to take out the SD card from the shooting device 101, attach it to a computer, and view it on a viewer, and the user can easily view the video taken by the shooting device 101 only with the user terminal 102.

[0101] The user terminal 102 can provide the user with an image of the periphery of the vehicle 10 photographed by the shooting device 101 with the user terminal 102 by S815, and can record the image information of the image displayed when the state of the vehicle 10 satisfies the event recording condition for each event in the storage unit for the user terminal 102 by S845. In addition, the user terminal 102 can also accept, for each event, a setting as to whether to record the image information in the storage unit for the user terminal 102.

[0102] Based on the event notification information (S820) received from the imaging device 101, when the vehicle 10 satisfies the event recording condition (S825), the image information recording module 310 of the user terminal 102 can record the image information in the state where the host vehicle 10 satisfies the event recording condition in the auxiliary storage device 302 for the terminal (S845).

[0103] Furthermore, the imaging device 101 can record image information corresponding to the event video stored in the user terminal 102 and stored for a longer time and with higher image quality. For example, when any one of the plurality of imaging devices 101 is a drive recorder, a plurality of pieces of image information can be recorded from the start to the end of driving.

[0104] As will be described later, the imaging device control module 314 of the user terminal 102 can also command the imaging device 101 to transmit the image information stored in the imaging device 101 corresponding to the event video in the user terminal 102.

[0105] FIG. 9 is an example of a setting flow 90 of the imaging device 101 by the user terminal 102. In the example of FIG. 9, the imaging device cooperation module 313 of the user terminal 102 accepts (S900) a setting input by the user using the input device 304 and the output device 305 of the user terminal 102 for the imaging device 101 connected to the user terminal 102.

[0106] The imaging device control module 314 of the user terminal 102 transmits (S910) the setting input accepted in S900 to the imaging device 101 via the communication control unit 307 and the network 104. The imaging device control module 314 of the user terminal 102 receives (S920) a confirmation notification regarding the setting input from the imaging device 101 and ends the setting flow 90.

[0107] An example of the setting of the imaging device 101 by the user terminal 102 will be described in more detail with reference to FIGS. 11, 12, 16, 17, 19 - 22, 25, 28, 29, and 31. In particular, according to the setting flow 90 in FIG. 9, the imaging device control module 314 of the user terminal 102 can perform various settings for the imaging device 101 for individual events. Thereby, the user terminal 102 can provide the user with the possibility of easily changing the settings of the imaging device 101 according to various driving states and surrounding completion.

[0108] FIG. 10 is an example of the image information display stop flow 100 by the user terminal 102. In the example of FIG. 10, the display control module 312 of the user terminal 102 displays the image information received from the imaging device 101 in the display area of the screen (hereinafter, may be simply referred to as the "screen"), which is the output device 305 of the user terminal 102, and displays operation elements in the operation area of the screen of the user terminal 102 (S1000). Examples of the display area and the operation area will be described more specifically with reference to FIG. 14 and the like.

[0109] When the first set time has not elapsed (No in S1010), the display control module 312 continues to display the image information on the output device 305. When the non - operation time has elapsed the first set time (Yes in S1010), the display control module 312 stops the display in the display area while continuing to display the operation area (S1020). The display control module 312 may determine, for example, whether the first set time has elapsed since the start of the screen display, or may determine whether the non - operation time, which is the time of no operation on the user terminal 102 (input device 304), has elapsed the first set time.

[0110] When the second set time has not elapsed (No in S1030), the display control module 312 continues to display only the operation area on the output device 305. When the second set time has elapsed (Yes in S1030), the display control module 312 stops displaying the operation area and the display area (S1040). The display control module 312 may determine, for example, whether the second set time has elapsed since the start of the screen display, or may determine whether the non-operation time, which is the time of no operation on the user terminal 102 (input device 304), has elapsed the second set time.

[0111] The display stop flow 100 shown in FIG. 10 will be described in more detail in relation to FIGS. 20 to 24. According to this embodiment, the user terminal 102 can record an event that is particularly difficult for the driver to directly visually confirm, such as a rear draft, without requiring an active operation or action by the user.

[0112] Also, when the host vehicle 10 is exposed to dangerous driving such as a rear draft, the driver is forced to pay the utmost attention to ensuring the safe driving of the host vehicle. Therefore, it is extremely difficult to record event information corresponding to the storage unit for the user terminal 102 for backup or immediate submission of evidence such as image information related to dangerous driving. Also, even in such a case, it is not legally recognized for the driver to operate the user terminal 102 during the driving operation.

[0113] However, according to this embodiment, even when exposed to such dangerous driving, the user terminal 102 can surely record an event video without requiring an active operation or action by the driver.

[0114] FIG. 11 is an example of a screen showing a part of an operation for setting reference lines 1101 and 1102 for event determination. FIG. 12 is an example of a screen showing a part of an operation for setting reference lines 1201 and 1202 for event determination following the example of the operation shown in FIG. 11.

[0115] The display control module 312 can receive a user's selection of "front reference position setting" or "rear reference position setting" in the "Active Safety settings" menu (see FIG. 26). In "front reference position setting" and "rear reference position setting", the imaging device cooperation module 313 can receive an input for setting reference lines 1101, 1102, 1201, and 1202 for event determination by the imaging device 101. Then, the imaging device control module 314 sets the received setting input in the imaging device 101.

[0116] Hereinafter, using the examples of FIGS. 11 and 12, the procedure for "front reference position setting" will be described. However, the imaging device control module 314 can also perform "rear reference position setting" in a similar procedure.

[0117] When the display control module 312 receives a user's selection of "front reference position setting" in the "Active Safety settings" menu 1100, the display control module 312 displays the image information received from the imaging device 101 connected by the imaging device cooperation module 313 in the display area 1110 on the output device 305 of the user terminal 102. In the examples of FIGS. 11 and 12, the display control module 312 displays, within the display area 1110, in addition to the video transmitted from the imaging device 101, the red lines 1101 and 1201 in the screen, which are reference lines for event determination, and the white lines 1102 and 1202 (the two broken lines above the "C" shape) in the screen, which are also reference lines.

[0118] The display control module 312 further displays buttons 1104 to 1109, which are operation elements, and a determination button and a cancel button in the operation area 1120 on the output device 305 of the user terminal 102.

[0119] The camera device control module 314 of the user terminal 102 can move the two white (reference) lines 1102 in a chevron shape and the one red (reference) line 1101 extending horizontally, which are displayed superimposed on the video from the camera device 101 within the display area 1110, up and down and left and right, or narrow or widen the widths of the two white (reference) lines 1102, in response to user input via the buttons 1104 to 1109. For appropriate event determination, it is preferable that the two white (reference) lines 1102 overlap the white lines 1103 of the road, and it is preferable that the red (reference) line 1101 overlaps the horizon (horizontal line).

[0120] When the camera device control module 314 receives an input to the "OK" button in the operation area 1120, it finalizes the input setting of the reference lines to the camera device 101 and ends the input setting. When the input setting of the reference lines to the camera device 101 by the camera device control module 314 is completed, the display control module 312 displays the "Active Safety Settings" menu 1100 again (see FIG. 26).

[0121] For example, when the camera device 101 is a drive recorder with an LCD (an example of a display unit), the camera device control module 314 of the user terminal 102 can also adjust the video displayed on the LCD and the reference lines for event determination according to the user input results by the buttons 1104 to 1106 displayed in the operation area 1120 of the user terminal 102, in the same procedure as illustrated in FIGS. 11 and 12.

[0122] FIG. 13 shows an example in which the reference lines 1301, 1302, D are appropriately set together with the virtual determination lines A, B, C. The example in FIG. 13 shows the actual video after setting the reference lines for the video transmitted from the rear camera device 101R, and below it, illustrates "good samples" and "bad samples" for the setting of the reference lines in an illustration.

[0123] After the reference line is appropriately set by the user terminal 102, the imaging device 101 determines dangerous driving such as tailgating from behind using the set reference line and the virtual determination lines A, B, and C set therebetween. Note that the virtual determination lines A, B, and C are not actually displayed on the display of the user terminal 102, nor are they included in the recorded image information. However, depending on the design and settings, these virtual determination lines can also be displayed and set. Also, a line D shown in the example of FIG. 13 is a reference line D corresponding to the red lines (reference lines) 1101 and 1102 in the screens shown in FIGS. 11 and 12, and is preferably adjusted to overlap with a horizontal line.

[0124] When the vehicle behind is present in the area defined by the two reference lines 1301 and 1302, the reference line D, and the determination line C, the imaging device 101 determines that an appropriate inter-vehicle distance is maintained between the host vehicle 10 and the vehicle behind.

[0125] The control unit 403 of the imaging device 101 can use a first determination time and a second determination time in addition to the reference line to determine dangerous driving. The control unit 403 of the imaging device 101 can set the first determination time and the second determination time to fixed values, for example, 5 seconds. FIG. 27 described later shows an example in which both the first determination time and the second determination time are set to 5 seconds. For example, the control unit 403 of the imaging device 101 can further strengthen the countermeasure against weaving driving by using two-stage identification using the first determination time and the second determination time.

[0126] In another embodiment, the first determination time and the second determination time may be variable values. These variable values can be changed using a reference table or the like according to, for example, the speed of the host vehicle 10, the speed difference between the host vehicle 10 and the vehicle behind, the distance and positional relationship between the host vehicle 10 and the vehicle behind, and the driving environment such as whether it is an ordinary road or a highway.

[0127] For example, when a following vehicle exists within a first determination area defined by a reference line and determination lines B and C for a first determination time, the imaging device 101 determines that a following approaching vehicle exists. The control unit 403 can also change the interval between the determination lines B and C, the area of the first determination area, and the first determination time according to, for example, the speed of the host vehicle 10, the speed difference between the host vehicle 10 and the following vehicle, the distance and positional relationship between the host vehicle 10 and the following vehicle, and the driving environment such as whether it is an ordinary road or a highway, using, for example, a reference table or the like.

[0128] Similarly, when a following vehicle exists within a second determination area defined by a reference line and determination lines A and B for a preset second determination time, the imaging device 101 determines that a following abnormally approaching vehicle exists. The control unit 403 of the imaging device 101 can also change the interval between the determination lines A and B, the area of the second determination area, and the second determination time according to, for example, the speed of the host vehicle 10, the speed difference between the host vehicle 10 and the following vehicle, the distance and positional relationship between the host vehicle 10 and the following vehicle, and the driving environment such as whether it is an ordinary road or a highway, using, for example, a reference table or the like.

[0129] As an example of the approach determination of a following vehicle using the first determination area and the second determination area, when the host vehicle 10 is traveling at a speed between 50 km / h and 80 km / h, if a following vehicle is detected within the second determination area for 5 seconds, a following approach warning is issued, and if it is detected for another 5 seconds, that is, a total of 10 seconds, a following abnormal approach warning and event recording are performed. Also, when the host vehicle 10 is traveling at a speed of 81 km / h or more, if a following vehicle is detected within the first determination area and the second determination area for 5 seconds, a following approach warning is issued, and if it is detected for another 5 seconds, that is, a total of 10 seconds, a following abnormal approach warning and event recording may be performed.

[0130] In this way, it is possible to issue a warning when a following vehicle exists in the area according to the speed and the approach distance, and by the event recording setting described later, image information related to the following approach warning and the following abnormal approach warning can be stored in the storage unit for the user terminal 102. The control unit 403 of the front imaging device 101F executes proximity determination of a rear approaching vehicle by performing image processing and determination on the video information received from the rear imaging device 101R. However, in the case where the rear imaging device 101R has the control unit 403, the rear imaging device 101R may perform proximity determination of the rear approaching vehicle.

[0131] FIG. 14 shows an example of switching between display layouts 1420 and 1430 according to the orientation of the screen of the user terminal 102. In the example of FIG. 14, switching between the display layout of the display area 1401 and the operation area 1402 when the screen of the user terminal 102 is in the vertical orientation and the display layout of the display area 1401 and a plurality of operation elements (operation parts) 1403 to 1410 of the operation area when the screen of the user terminal 102 is in the horizontal orientation is shown.

[0132] The display control module 312 of the user terminal 102 displays the display area 1401 and the operation area 1402 (1403 to 1410) on an output device 305 such as a screen (display) of the user terminal 102. The operation area 1402 is an area composed of operation elements 1403 to 1410 for inputting an input operation to the image information recording module 310 and settings for the imaging device 101, for example. The display area 1401 is an area for displaying, for example, the image information transmitted from the imaging device 101. Note that various examples of the operation area and the display area are also shown in other drawings.

[0133] The display control module 312 of the user terminal 102 switches whether to arrange the display area 1401 and the operation area 1402 of the image information recording module 310 in separate areas within the screen of the user terminal 102 or to arrange the operation area overlapping the display area according to the orientation of the user terminal 102. More specifically, when the screen of the user terminal 102 is in the vertical orientation, the display control module 312 displays in a display layout in which the display area 1401 and the operation area 1402 are arranged in separate areas within the screen. When the screen of the user terminal 102 is in the horizontal orientation, the display control module 312 displays in a display layout in which the operation area 1402 is arranged overlapping the display area 1401.

[0134] Furthermore, when the operation area is composed of a plurality of operation elements (operation parts) 1403 to 1410, the display control module 312 can also arrange each of the plurality of operation elements 1403 to 1410 spaced apart from each other on the display area 1401 and overlapping the display area 1401.

[0135] In particular, in the image display at the time of event occurrence, since parts that are important or highly interesting to the user often exist in the center, when the screen is in the horizontal orientation, as in the example of FIG. 14, the display control module 312 arranges and displays the individual operation elements 1403 to 1410 along the four sides of the screen, which can maximize the visual recognition area and maximize the visibility of important or highly interesting parts.

[0136] FIG. 15 is an example of the preview screen 1500 of the user terminal 102. The example of FIG. 15 shows the preview screen of the image information recording module 310 of the user terminal 102 when the output device 305 such as a display is in the vertical orientation. Also in this preview screen, the display control module 312 can change the display layout of the display area 1510 and the operation areas 1520 and 1530 according to the orientation of the screen of the user terminal 102, similar to FIG. 14.

[0137] The display control module 312 can call and display the setting screen etc. of the image information recording module 310 from the preview according to the example of FIG. 15. Also, when the user terminal 102 is connected to a plurality of imaging devices 101, the display control module 312 can accept settings regarding which video from which imaging device 101 to display on the preview screen.

[0138] In addition, the display control module 312 can also display the images from the plurality of imaging devices 101 (101F, 101R) on the output device 305 in a picture-in-picture manner. For example, when the imaging device 101F is a front camera and the imaging device 101R is a rear camera, the display control module 312 can display the image information from the rear camera across the entire display area, and display the image information from the front camera by overlaying it on a part of the image display from the rear camera.

[0139] The switching of the image information displayed on the output device 305 can be implemented in various ways as follows, for example. · The front imaging device 101F aggregates the image information of the front imaging device 101F and the rear imaging device 101R, and transmits it to the user terminal 102. The display control module 312 of the user terminal 102 selects one of the two pieces of image information captured by the front imaging device 101F and the rear imaging device 101R transmitted from the front imaging device 101F to switch the display image. · The front imaging device 101F and the rear imaging device 101R each transmit the image information to the user terminal 102. The display control module 312 of the user terminal 102 selects one of the image information captured by the front imaging device 101F transmitted from the front imaging device 101F and the image information captured by the rear imaging device 101R transmitted from the rear imaging device 101R to switch the display image. · The front imaging device 101F aggregates the image information of the front imaging device 101F and the rear imaging device 101R, and the front imaging device 101F executes the image switching process. That is, the front imaging device 101F selects one of the two pieces of image information captured by the front imaging device 101F and the rear imaging device 101R, transmits the selected piece of image information to the user terminal 102, and the user terminal 102 displays the transmitted image information. Furthermore, the display control module 312 may use the above configurations appropriately according to the event recording settings described later.

[0140] The display control module 312 displays, for example, the camera image (front, rear, picture-in-picture) selected by the user on the preview screen, but can also forcibly switch to the rear camera video when a rear approach is detected.

[0141] The three buttons 1501 to 1503 arranged at the bottom of FIG. 15 will be described. Data confirmation button 1501: Displays a confirmation screen of the files in the auxiliary storage device 302 of the user terminal 102. One-touch recording button 1502: Saves the video transmitted from the imaging device 101 and currently displayed on the preview screen. Settings button 1503: Calls up a screen for performing various settings. Examples of the setting screens that can be called directly or indirectly from the settings button 1503 will be described in more detail in relation to FIGS. 11, 12, 16, 17, 19 to 22, 25, 28, 29, 31.

[0142] The four buttons 1504 to 1507 arranged at the top of FIG. 15 will be described. Screen switching button 1504: Switches between a plurality of imaging devices 101. For example, the screen is switched in the order of front camera → rear camera → picture-in-picture. Snapshot button 1505: Saves the video transmitted from the imaging device 101 and currently displayed on the preview screen as a still image. Power saving mode button 1506: Lowers the brightness of the screen of the user terminal 102. In particular, it darkens the screen of the user terminal 102 while it is being displayed. Microphone on / off button 1507: Switches whether to attach audio to the data stored by the imaging device 101.

[0143] The display control module 312 has a display area and an operation area on the output device 305 of the user terminal 102. The camera cooperation module 313 can display the video from the camera device 101 in the display area of the output device 305 of the user terminal 102. Also, the camera device control module 314 can receive the setting input from the user for the camera device 101 via a plurality of operation elements displayed in the operation area. In addition, the input from the user using the plurality of operation elements displayed in the operation area may be a setting input related only to the user terminal 102.

[0144] FIG. 16 is an example of an operation for displaying the setting screens 1601 and 1602 of the camera device 101 from the preview screen. In the example of FIG. 16, the user terminal 102 can provide the user with the possibility of setting the camera device 101 that can be operated simply and intuitively. In particular, through the setting screens 1601 and 1602, not only the setting for various modules of the user terminal 102 but also the possibility of setting the camera device 101 can be provided.

[0145] FIG. 17 is an example of the setting menus 1702 to 1704 displayed when the "Smartphone Event Recording Setting" menu shown in FIG. 16 is selected. In the example of FIG. 17, when "Smartphone Event Recording Setting" is selected from the "System Setting" menu 1701, the display control module 312 displays the setting menu 1702 of "Smartphone Event Recording Selection" and "Smartphone Event Recording Setting". The "Smartphone Event Recording" menu shows the options of events to be recorded on the user terminal 102. In the setting menu 1703 of "Smartphone Event Recording Setting", the image information recording module 310 receives the selection input of the events to be saved. In the setting menu 1704 of "Smartphone Event Recording Setting", "Rear Abnormal Approach Recording" is set to ON.

[0146] The communication control unit 307 of the user terminal 102 can receive image information from the imaging device 101 whose connection has been established by the imaging device cooperation module 313 via the network 104, either before the event occurs or immediately after the event occurs. The image information recording module 310 records the image information of the selected event based on "Smartphone Event Recording Selection".

[0147] In the example of FIG. 17, examples of the options in the "Smartphone Event Recording Selection" menu include "G-sensor Recording", "One-touch Recording", and "Rear Abnormal Approach Recording". Examples of events related to the "G-sensor" include events that detect an impact of a certain level or more of the G-sensor, such as a collision of the vehicle 10. Examples of events related to "One-touch Recording" include events caused by a button operation by the user. Examples of events related to "Rear Abnormal Approach Recording" include events that detect the approach of a vehicle behind for 10 seconds while driving. More specifically, events in which a vehicle behind exists for a certain period of time, for example, 10 seconds, within the first or second determination region already described in relation to FIG. 13.

[0148] It is also possible to add other events. For example, the "Smartphone Event Recording Selection" menu may also include various dangerous driving such as "Front Abnormal Approach", "Sudden Approach (sudden acceleration of the vehicle behind, sudden braking of the vehicle in front)", "Dangerous Weaving", "Dangerous Cutting In", "Excessive Horn", "Abnormal Passing / High Beam", "Sudden Braking of the Vehicle in Front", and "Obstruction of Overtaking by the Vehicle in Front" as candidates for events to be automatically saved.

[0149] For example, for each individual event set as a dangerous driving in the imaging device 101, the image information recording module 310 can accept an ON / OFF setting for automatic saving individually. Note that the "automatic saving" mentioned in this specification means saving image information that does not require an operation of the user terminal 102 by the user during or after the occurrence of an event.

[0150] FIG. 18 is an example of a screen 1800 in a state where an event is displayed. In the example of FIG. 18, the image information recording module 310 for the user terminal 102 to record the video from the imaging device 101 is activated, and also shows an example of the image information displayed on the screen of the user terminal 102 when the imaging device 101 detects the occurrence of an event that has been set or selected.

[0151] On the video screen with the text "Fanned video" in the figure, for example, a video showing a state of being fanned by a vehicle behind transmitted from the imaging device 101R that actually captured the event is displayed. For example, when other image information from another imaging device (another imaging device, a front camera) 101F is displayed in the display area of the preview screen, the display control module 312 switches the video to be displayed in the display area of the preview screen to the video transmitted from the imaging device (rear camera) 101R that captured the event. To clarify that event recording is being executed, the user terminal 102 may display a screen such as FIG. 30, for example.

[0152] When an event for which the recording setting is on occurs, the display control module 312 of the user terminal 102 can display the corresponding event on the output device 305 of the user terminal 102 without requiring an operation by the user. When the user terminal 102 is present at an appropriate position, for example, on the driver's line of sight during driving, the user terminal 102 can provide the driver with an opportunity to view the video related to the event without requiring an operation by the driver himself / herself.

[0153] When an event such as a rear draft occurs, it is very useful and important from the perspective of driving safety that the user, especially the driver, can check information about the event without performing additional operations by himself / herself.

[0154] FIG. 19 is an example of a setting screen 1900 of the user terminal 102. In the example of FIG. 19, the setting screen 1900 displays a selection menu of the image information recording module 310 regarding "Video" and "System". The image information recording module 310 can execute "Smartphone Event Recording Settings" and "Smartphone APP Screen Settings" regarding the system for recording the video of the photographing device 101.

[0155] When the display control module 312 accepts the selection of "Smartphone Event Recording Settings", the setting screen 1900 shifts to, for example, the setting screen 3100 of "Event Memory Selection" and "Event Recording Settings" in FIG. 31. Also, when the display control module 312 accepts the selection of "Smartphone APP Screen Settings", the setting screen shifts to, for example, the setting screen 2000 of "Smartphone APP Screen Settings" in FIG. 20.

[0156] The display control module 312 can present the user with the possibility of setting the recording target, recording time, video display time, etc. of the image information from the photographing device 101 on these setting screens 2000 and 3100. The setting screens 2000 and 3100 will be described later in relation to FIGS. 20 and 31.

[0157] FIG. 20 is an example of the setting screen 2000 of two types of screen settings. In the example of FIG. 20, the display control module 312 of the user terminal 102 displays the "Smartphone APP Screen Settings" menu. The "Smartphone APP Screen Settings" menu has "Camera Video Display Time" and "Power Saving Screen Transition Time".

[0158] The "camera video display time" is the length of time for which the video received from the imaging device 101 is displayed on the preview screen of the user terminal 102. Also, the "power-saving screen transition time" is the length of time from the activation of the image information recording module 310 until the entire screen of the user terminal 102 is switched to a state with lower power consumption, for example, the length of time until the entire screen is turned off. In FIG. 20, the camera video display time is set to 1 minute and the power-saving screen transition time is set to 30 seconds.

[0159] FIG. 21 is an example of a selection screen 2100 for the camera video display time. In the example of FIG. 21, the display control module 312 can accept the user's selection of a desired camera video display time from among 30 seconds, 1 minute, 2 minutes, 5 minutes, 10 minutes, 30 minutes, and continuous display. In another embodiment, it is also possible to accept direct input of the camera video display time by the user himself / herself.

[0160] This camera video display time can correspond to the first set time of the display stop flow 100 in FIG. 10. When the selected camera video display time has elapsed, the display control module 312 stops the display of the image information from the imaging device 101 on the screen of the user terminal 102. When the user selects continuous display, the video from the imaging device 101 is continuously displayed on the preview screen of the user terminal 102.

[0161] FIG. 22 is an example of a selection screen 2200 for the power-saving screen transition time. In the example of FIG. 22, the display control module 312 can accept the user's selection of a desired power-saving screen transition time from among 30 seconds, 1 minute, 2 minutes, 5 minutes, 10 minutes, 30 minutes, and continuous display. In another embodiment, it is also possible to accept direct input of the camera video display time by the user himself / herself.

[0162] This power-saving screen transition time can correspond to the second set time of the display stop flow 100 in FIG. 10. When the selected power-saving screen transition time elapses, the display control module 312 stops the display of the entire screen of the user terminal 102 or reduces the brightness of the entire screen (darkens while keeping the display). In this case, the process of "reducing the brightness" also includes the process of making the screen completely black. When the user selects continuous display, these transitions are not performed, and the preview screen of the user terminal 102 continuously displays the video from the imaging device 101.

[0163] The options shown in FIGS. 21 and 22 may increase or decrease the displayed options according to the selection result of one option. For example, if the user selects 5 minutes in the example of FIG. 22, the options shown in the example of FIG. 21 may be only 30 seconds, 1 minute, 2 minutes, and continuous display.

[0164] When the image information recording module 310 receives the settings regarding the "camera video display time" and the "power-saving screen transition time", it stores each setting as setting information 320 in the auxiliary storage device 302. The display control module can call the stored setting information regarding the "camera video display time" and the "power-saving screen transition time" to execute the display stop flow 100 in FIG. 10.

[0165] FIG. 23 shows examples of screen displays 2301 and 2302 before and after the elapse of the camera video display time. In the example of FIG. 23, after the elapse of the camera video display time, the user terminal 102 displays a screen display 2302 of image information with lower power consumption regarding the screen display than the image information from the imaging device 101 instead of the screen display 2301 of the image information from the imaging device 101. For example, after the elapse of the camera video display time, the display control module 312 stops the display of the image information from the imaging device 101 in the display area and makes the display stop black. The display control module 312 can also add and display text information such as "video display" to the image information with lower power consumption as shown in FIG. 23. The user terminal 102 can also display a more intuitive notification screen using marks, symbols, images, etc. other than text.

[0166] The display control module 312 can switch the screen display 2302 of the image information with lower power consumption to the screen display 2301 of the image information from the imaging device 101 by a trigger from the imaging device 101 or a tap on the screen by the user. In the example of FIG. 23, the display control module 312 can reduce the power consumption associated with the display of the video from the imaging device 101 in the display area 2304 while providing the user with the possibility of input using the operation area 2303. Note that the user terminal 102 can record the corresponding event when the event occurs in any screen state of FIG. 23.

[0167] FIG. 24 shows examples of screen displays 2401 and 2402 before and after the passage of the power-saving screen transition time. In the example of FIG. 24, after the passage of the power-saving screen transition time, the user terminal 102 changes not only the image information from the imaging device 101 but also the screen display 2401 of the user terminal 102 to a screen display 2402 in a state with lower power consumption, for example, a state where the brightness of the entire screen is reduced. Particularly preferably, the user terminal 102 turns off the display of the entire screen of the user terminal 102 and reduces the power consumption associated with the screen display of the user terminal 102 as much as possible.

[0168] The display control module 312 can also return the display of the entire screen in a state with lower power consumption to the preview screen of FIG. 15 or the screen immediately before transitioning to the screen in a state with lower power consumption by a trigger from the imaging device 101 or a tap on the screen by the user. In the example of FIG. 24, the user terminal 102 can reduce the power consumption related to the screen display as much as possible. Note that the user terminal 102 can record the corresponding event when the event occurs in any screen state of FIG. 24.

[0169] FIG. 25 shows examples of selection screens 2502 to 2504 for events to be warned about. In the example of FIG. 25, the display control module 312 of the user terminal 102 accepts the user's tap selection of "Active Safety" from the state where the "Settings" menu 2501 having "Video Resolution", "Settings", and "Active Safety" as options is displayed on the output device 305 of the user terminal 102, and transfers to the "Active Safety Settings Screen" 2502 having "Front Reference Position Setting", "Rear Reference Position Setting", "Forward Inter-vehicle Distance Warning", "Start Delay Warning", "Lane Departure Warning", "Rear Approach Warning", "Rear Abnormal Approach Warning", and "Rear Abnormal Approach Recording" as options.

[0170] On the "Active Safety Settings Screen" 2502, the imaging device control module 314 of the user terminal 102 can accept the setting of the safe driving support function (Active Safety) by the imaging device 101 whose connection is established by the imaging device cooperation module 313. The safe driving support function is a function executed by the imaging device 101 for assisting the driver's judgment and reducing the driving load.

[0171] On the "Active Safety Settings Screen", the ON / OFF settings of each safe driving support function of "Forward Inter-vehicle Distance Warning", "Start Delay Warning", "Lane Departure Warning", "Rear Approach Warning", "Rear Abnormal Approach Warning", and "Rear Abnormal Approach Recording" can be individually set. For example, the warning of the safe driving support function set to ON can be output by at least one of the imaging device 101 or the user terminal 102.

[0172] On the selection screen 2503, warnings for all events and rear abnormal approach records are set to OFF, and on the selection screen 2504, warnings for all events and rear abnormal approach records are set to ON. When the "Rear Abnormal Approach Record" shown in FIG. 25 is set to ON, the image information recording module 310 of the user terminal 102 can record, for example, image information of a cut-in driving from the rear in the auxiliary storage device 302 according to the image information recording flow 80 of FIG. 8 when the vehicle 10 meets the event recording conditions.

[0173] In addition to the "Rear Abnormal Approach Record", the "Active Safety Setting Screens" 2502 to 2504 may have options for recording different events, such as "Front Abnormal Approach Record", "Sudden Approach Record", "Lateral Abnormal Approach Record", "Abnormal Interruption Record", "Overtaking Obstruction Record", etc. In that case, the image information recording module 310 can also accept a priority setting regarding which event recording to prioritize when these events occur simultaneously for a plurality of events.

[0174] FIG. 26 shows an example of the response of the imaging device 101 and the indicator display when a rear approach is detected. In the example of FIG. 26, when the front imaging device 101F detects the approach of a vehicle from the rear based on the video or the like captured by the rear imaging device 101R, the imaging device 101F blinks the indicator attached to the imaging device 101F together with a beeping sound effect. The sound effect and the indicator display can be set based on the recording settings of the imaging device 101F or the user terminal 102, the type of event, etc. In this embodiment, the recording lamp indicating recording blinks quickly in blue.

[0175] Also, for example, when the imaging device 101F recognizes the approach of a vehicle from the rear, in addition to or instead of the imaging device 101F performing the above-mentioned response and indicator display, the rear imaging device 101R may perform the same response and indicator display.

[0176] Also, in the configuration of FIG. 26, the video of the rear imaging device 101R is transmitted to the front imaging device 101F, and the front imaging device 101F performs event determination such as proximity determination. However, the rear imaging device 101R may execute this event determination. Regarding proximity from the front, if the front imaging device 101F outputs a warning, for example, by voice, and regarding proximity from the rear, if the rear imaging device 101R outputs a warning, for example, by voice, the driver and passengers can easily recognize that a neighboring vehicle is approaching from the direction where the warning sound is made.

[0177] By these warnings, the imaging device 101 can notify the user, by outputting sound or light from the imaging device 101, that, for example, recording of an event corresponding to the user terminal 102 or the imaging device 101 has started.

[0178] FIG. 27 shows an example of the response of the imaging device 101 and the indicator display when a rear approach and a rear abnormal approach are detected. In the example of FIG. 27, when the imaging device 101 continues to detect the approach of a vehicle from the rear for 5 seconds, the imaging device 101 issues a voice warning saying "Please pay attention to the rear". At this point, the user terminal 102 has not yet started event recording.

[0179] After that, when the imaging device 101 continues to detect the approach of a vehicle from the rear for another 5 seconds, the imaging device 101F issues a voice warning saying "Rear dangerous driving has been detected" and changes the display of the indicator to the warning state. Also, the imaging device 101 changes the display of the indicator to the recording state, and the user terminal 102 starts recording the event. Similar to the embodiment of FIG. 26, at the time of event detection, in addition to the imaging device 101, the user terminal 102 can also give a similar notification by sound, light, and screen display, etc. For example, instead of a voice warning, the user terminal 102 can also output an event sound such as "beep".

[0180] In the example of FIG. 27, the user terminal 102 starts event recording after the second warning (rear abnormal approach warning). However, in another embodiment, the user terminal 102 may perform event recording at both the timing after the first warning (rear approach warning) and the timing after the second warning (rear abnormal approach warning). In yet another embodiment, the first warning (rear approach warning) may be omitted, and warning and event recording may be performed at the timing of the rear abnormal approach warning.

[0181] In yet another embodiment, when the warning module 311 of the user terminal 102 detects an event to be warned about in FIGS. 26 and 27, it may give a notification similar to the warning by the imaging device 101, such as a screen display using sound, light, text, etc. Also, the imaging device cooperation module 313 of the user terminal 102 can set for each event and for each imaging device 101 which event these notifications are to be given for.

[0182] FIG. 28 is an example of a selection screen 2800 for the terminal storage unit. In the example of FIG. 28, the image information recording module 310 can accept a selection by the user from the options of "Folder in Smartphone Body" and "Folder in Micro SD" shown in the "Smartphone Event Storage Destination" menu.

[0183] However, the "Smartphone Event Storage Destination" menu may have another auxiliary storage device 302 as an option. The other auxiliary storage device 302 may be, for example, a storage device on the cloud connected to the user terminal 102 and the network 104. Further, in another embodiment, the image information recording module 310 can also accept the addition of a storage destination for an event by the user himself / herself.

[0184] FIG. 29 is an example of a selection screen 2900 for the storage time of the image information received from the imaging device 101. In the example of FIG. 29, "Smartphone Event Recording Time" can be selected from any of 10 seconds, 20 seconds, 30 seconds, and 60 seconds.

[0185] In another embodiment, it is also possible for the user himself / herself to input and set the recording time. For example, when the SNS service or the like registered by the user sets the longest video time for the uploaded video, the image information recording module 310 may perform event recording in advance at that set time. Thereby, the user terminal 102 can record image information that conforms to the requirements of the upload destination without the user processing the image information during or after the event occurs.

[0186] Furthermore, the selection screen 2900 in FIG. 29 may present, as options, recording times suitable for various SNS services, or recording times specified by public institutions that are the submission destinations of evidentiary images, etc. By accepting such options from the user in advance, even when an event such as dangerous driving is encountered, the user's desired data can be recorded in the auxiliary storage device 302 of the user terminal 102 without imposing the psychological burden of the user having to process or confirm the image of themselves engaged in dangerous driving.

[0187] The communication control unit 307 of the user terminal 102 can also upload the image information stored in an appropriate format for content posting to an external server 103 that provides an SNS service or the like, reporting and submitting evidence to an external server such as a public institution, providing information to an external server of a private company or the like, backing up data, etc. In any of the embodiments, the user terminal 102 does not require the user to perform an image information processing operation after or during the event for uploading the image information. Also, the image information recording module 310 of the user terminal 102 can accept settings regarding which event to send to which external server on the setting screen of the image information recording module 310.

[0188] Furthermore, when the user terminal 102 uploads image information to the external server 103, for example, it may upload the index information described with reference to FIG. 7 added thereto. Particularly advantageously, the user terminal 102 can also upload the image information after performing processing such that at least a part of the index information can be confirmed on the image information.

[0189] Thereby, even if the user terminal 102 or the imaging device 101 is damaged and it becomes difficult to transfer data after the event, the user terminal 102 can surely save, submit, publish, etc. the image information of the important event and the occurrence status of the event without any operation of the user terminal 102 by the user himself / herself during or after the event.

[0190] In another embodiment, when the user terminal 102 uploads image information to an external server, the user terminal 102 may add information stored as setting information 320 in the auxiliary storage device 302 of the user terminal 102 to the image information to be uploaded. The added setting information 320 may be information about physical characteristics such as height, weight, blood type, etc. of the user who owns the user terminal 102 and his / her family, friends, or passengers, medical institution visit history, medications, allergies, etc., or may be emergency contact information, etc. Thereby, the user terminal 102 can add information pre-stored in the user terminal 102 to the image information captured by the imaging device 101 and upload it to the external server.

[0191] For example, when the vehicle 10 gets into a state of having a flat tire, it is also possible to transmit the image information related to the flat tire together with index information composed of GPS coordinates indicating the current position of the vehicle 10 to the server of the organization providing the road service to which the driver subscribes.

[0192] The image information recorded in the user terminal 102 is the image information of event recording triggered by the occurrence of an event. Therefore, even if the user does not process the image information, impactful image information can be easily created, and this can be sent from the smartphone to an external server and posted. For example, it is possible to post a video suitable for public display on an SNS (Social Networking Service). The video of event recording has an impact and a short duration, so it is suitable for SNS.

[0193] FIG. 30 shows examples of the display screens 3001 and 3002 in memory that have received the image information from the imaging device 101. During event recording, the corresponding event video is displayed on the display screen. In the example of FIG. 30, both pieces of image information received from two imaging devices 101 (for example, the front imaging device 101F is called the front camera and the rear imaging device 101R is called the rear camera) are displayed in a picture-in-picture manner. As already described with respect to FIG. 15, the display of the picture-in-picture video can be switched by tapping the screen switching button, for example, front camera video → rear camera video → picture-in-picture video with the rear camera video displayed in the front camera video → picture-in-picture video with the front camera video displayed in the rear camera video. The switching patterns can be various combinations according to the number and type of the imaging devices 101, etc., and it is also possible for the user himself / herself to set a desired pattern.

[0194] Also, for example, when the user terminal 102 receives image information from four imaging devices, it is also possible to display the screen in four parts. Alternatively, the user terminal 102 can also display only the image information related to the event. To notify that event recording is actually in progress, the user terminal 102 can display notification information such as "Event Recording" and "Steam recording" along with the countdown of the set event recording time. For example, in Figure 30, the display screen 3001 with the text information "Event Recording 18" indicates that the remaining time for event recording is 18 seconds. Also, the display screen 3002 with the text information "Steam recording 28" indicates that the remaining time for event recording is 28 seconds.

[0195] The screen of the user terminal 102 can also display a more intuitive notification screen using marks, symbols, or images other than text. In addition to displaying the notification screen on the screen, the user terminal 102 can also output corresponding sounds or sound effects.

[0196] Figure 31 is an example of a setting screen 3100 for event memory selection and event recording settings. In the example of Figure 31, the user terminal 102 displays two items on the screen: "Smartphone Event Recording Selection" and "Smartphone Event Recording Selection". Since the "Smartphone Event Recording Selection" menu is equivalent to the one described in relation to Figure 17, the description is omitted.

[0197] The "Smartphone Event Recording Selection" menu has options such as "Smartphone Event Recording Time", "Smartphone Event Overwrite", and "Smartphone Event Save Destination". The image information recording module 310 accepts the user's selections for these options.

[0198] In the example of Figure 31, the image information recording module 310 has accepted the selections that the "Smartphone Event Recording Time" is 30 seconds, the "Smartphone Event Overwrite" is ON, and the "Smartphone Event Save Destination" is the smartphone body. The image information recording module 310 can provide the user with very concise options regarding the recording settings of individual events transmitted from the imaging device 101 and the recording method into the auxiliary storage device 302 for the user terminal 102 via the "Smartphone Event Recording Selection" menu and the "Smartphone Event Recording Selection" menu.

[0199] FIG. 32 is an example of a notification screen 3200 regarding insufficient capacity of the auxiliary storage device 302. In FIG. 32, the screen of the user terminal 102 is displaying a notification screen having text information "The capacity of the smartphone is insufficient, so events cannot be recorded", but the content of the text information can be set as appropriate.

[0200] Also, the screen of the user terminal 102 can display a more intuitively recognizable notification screen using marks, symbols, images, etc. other than text. In addition to displaying the notification screen on the screen, the user terminal 102 can also output corresponding sounds, sound effects, etc. By displaying such a notification screen, the user terminal 102 can quickly and surely notify the user of the operation status of the imaging device 101.

[0201] FIG. 33 is an example of a notification screen 3300 for notifying that the imaging device 101 is receiving an operation by an operation element provided in the imaging device 101. The operation elements are, for example, buttons, switches, slide knobs, rotary knobs, etc. provided in the imaging devices 101F and 101R, a touch panel type operation screen, and buttons, icons, etc. displayed thereon. In FIG. 33, the screen of the user terminal 102 is displaying a notification screen 3300 having text information "During operation of the imaging device" (predetermined display information, information related to the operation state of the imaging device), but the content of the text information can be set as appropriate. The text information may be for notifying a process being executed in the imaging device 101, such as "Formatting SD card", for example.

[0202] While the notification screen 3300 is being displayed, the display control module 312 restricts the input of operations by the operation area of the user terminal 102 on the imaging device 101. In another embodiment, while the notification screen 3300 is being displayed, the display control module 312 may always accept some operations, such as those that do not conflict with the operations of the imaging device 101 among the operation elements in the operation area, for example, those that can only be operated on the user terminal 102. Further, the display control module 312 may also be configured not to accept operations for only a partial period during the operation of the imaging device 101.

[0203] Also, the screen of the user terminal 102 can display a more intuitively recognizable notification screen using marks, symbols, or images other than text. In addition to displaying the notification screen on the screen, the user terminal 102 can also output corresponding sounds, sound effects, etc. By displaying such a notification screen, the user terminal 102 can notify the user of the operation status of the imaging device 101 quickly and surely.

[0204] FIG. 34 is an example of a diagram showing two appearances (A) and (B) of the imaging device 101F. In the example of the appearance (A) of FIG. 34, the imaging device 101F is designed as a drive recorder 3401 for imaging the front of the vehicle 10. The drive recorder 3401 is shown as seen through the windshield from the front side of the vehicle 10. The drive recorder 3401 is connected, for example, via a cable 3402 for electrically connecting to the rear imaging device 101R and a connection portion 3402a. This cable 3402 corresponds to the cable 11 in FIG. 1 and is connected to the rear imaging device 101R on the side opposite to the drive recorder 3401.

[0205] The housing of the imaging device 101F generally has a cylindrical shape (or a circular tube shape). On the vehicle direction side of the imaging device 101, the imaging lens 3404 of the imaging device 101F (drive recorder 3401) is arranged. On the opposite side of the position where the imaging lens 3404 is arranged in the housing of the imaging device 101F, LEDs for notifying various information, operation buttons, and microphone holes are arranged. The drive recorder 3401 is further connected to a cable 3403 for connecting to a multi-battery or a power supply unit with a voltage monitoring function etc. via a connection part 3403a, but it can also function without the cable 3403.

[0206] Even in the example of the external appearance (B) of FIG. 34, the imaging device 101F is designed as a drive recorder 3411 for performing imaging in front of the vehicle 10. Since the drive recorder 3411, the cable 3412, and the connection part 3412a have a configuration similar to that of the one in the external appearance (A), the description is omitted. As is clear from the comparison with the example of the external appearance (A) of FIG. 34, various variations are conceivable for the external appearance design of the imaging device 101.

[0207] [Configuration of the image recording device] FIG. 35 is another example of a diagram showing the external appearance of the imaging device 101 which is a drive recorder with an LCD as the above-described display unit. The operation unit 3515 is one of the user interfaces for inputting various instructions to the imaging device 101. As an example of the operation unit 3515, as shown in FIG. 35, there are operation buttons arranged at various positions on a predetermined surface of the housing 3524. Details will be described later.

[0208] The display unit 3516 is constituted by, for example, a liquid crystal display (LCD). The control unit 403 performs control to display, in real time, the image data captured by, for example, the imaging unit (camera) 402 on the display surface of the display unit 3516, or to read out and display the image data etc. recorded in a storage device such as the SD card 406.

[0209] The power supply 411 receives external power supply and supplies a predetermined power supply voltage to devices, equipment, etc. within the imaging device 101. The power supply 411 includes, for example, a DC jack 3531 for attaching a power cable for connecting to an external power supply.

[0210] As shown in FIG. 35, the external configuration of the imaging device 101 includes a flat substantially rectangular housing 3524. The housing 3524 stands upright in the correct posture when attached to a predetermined position on a vehicle, for example. On the front surface 3541 located on the front side in the moving direction of the vehicle in the correct posture, a lens 3542 of the built-in imaging unit (camera) 402 is arranged. The imaging unit (camera) 402 images the area in front of the advancing direction of the vehicle. Above the lens 3542 on the front surface 3541 of the housing 3524, a through hole 3581 for a speaker 409 built in the housing 3524 is provided. Also, below the lens 3542 on the front surface 3541 of the housing 3524, a through hole 3571 for a microphone built in the housing 3524 is provided.

[0211] On the upper surface 3544 of the housing 3524, a joint rail 3523 and a camera jack 3521 are provided. The joint rail 3523 detachably attaches one end of a bracket, which is an example of an attachment member not shown in the figure. The other end of the bracket serves as an attachment portion for attaching to, for example, a predetermined position above the front glass of a vehicle, a vehicle's rearview mirror, the ceiling in the vehicle interior, etc. This attachment portion is attached and fixed to, for example, the front glass of a vehicle, which is an example of an attachment site. Thereby, the imaging device 101 is arranged at a predetermined position in the vehicle interior. Also, when the imaging device 101 is arranged at the predetermined position and set in the correct posture as described above, the front surface 3541 faces the front glass of the vehicle, and the imaging unit (camera) 402 images the area in front of the vehicle.

[0212] The camera jack 3521 connects to one end of a camera cable (not shown). The other end of this camera cable is connected to another external imaging device. The external imaging device is preferably one that captures an area different from the imaging area of the imaging unit (camera) 402. In this embodiment, a cable for communicating with the imaging device 102R is connected. The camera jack 3521 may comply with, for example, the USB Type-C standard and may also serve as a terminal for communicating with the imaging device 101R in accordance with the Ethernet (registered trademark) standard.

[0213] On the back surface of the housing 3524, there are provided the display surface of the display unit 3516, the operation unit 3515, and a power lamp. In the example shown in FIG. 35(b), the operation unit 3515 is provided with a plurality of operation buttons vertically in the area on the right side of the display surface of the display unit 3516. In this embodiment, four operation buttons are arranged in order from the top: the first operation button 3501, the second operation button 3502, the third operation button 3503, and the fourth operation button 3504.

[0214] The first operation button 3501 is a button for switching the video displayed on the display surface of the display unit 3516 and a button for instructing the formatting of a storage device such as the SD card 406. As described above, the imaging device 101 has a function of connecting a separate external imaging device (not shown) using a camera cable connected to the camera jack 3521 and displaying the image data captured by the external imaging device on the display surface of the display unit 3516. When the first operation button 3501 is briefly pressed, the control unit 403 switches the display between the image captured by the imaging unit (camera) 402 built in the imaging device 101 and the image captured by the external imaging device. When the first operation button 3501 is long-pressed, the control unit 403 executes the formatting of the storage device such as the SD card 406.

[0215] The second operation button 3502 is a button for displaying a selection screen for selecting the image data to be played back. When the second operation button 3502 is pressed, the control unit 403 displays a selection screen for selecting and playing back the video data recorded on the SD card 406.

[0216] The third operation button 3503 is a button for displaying the setting menu. This setting menu is hierarchical, and by appropriately operating each operation button, the menu screen of the lower layer can be displayed, and various settings can be made on the target setting screen.

[0217] The fourth operation button 3504 is a button for starting / stopping recording. For example, in the continuous recording function, when the fourth operation button 3504 is short-pressed during recording, the control unit 403 that detects this temporarily stops the recording, and when the fourth operation button 3504 is short-pressed during the temporary stop, the control unit 403 resumes the recording. Also, when this fourth operation button 3504 is long-pressed, the control unit 403 changes the frame rate during recording.

[0218] On the right side surface 3543 of the housing 3524, a DC jack 3531 and a recording medium insertion port 3525 are provided. The DC jack 3531 is used to supply power to the imaging device 101. For example, a connector of a cigarette plug cord can be connected to the DC jack 3531, and the cigarette plug cord can be connected to the cigarette socket of the vehicle to receive power supply. Instead of the cigarette plug cord, an OBD adapter that can be connected to the OBDII connector of the vehicle may be used.

[0219] The OBDII connector is also called a fault diagnosis connector, is connected to the ECU of the vehicle, and various vehicle information is output regularly. Therefore, by connecting this OBDII adapter and the OBDII connector on the vehicle body side, and connecting the connector attached to the tip of the connection cord of the OBDII adapter to the DC jack 201, power supply to the imaging device 101 is performed, and at the same time, the control unit 403 regularly acquires various vehicle information. The recording medium insertion port 3525 is an insertion port for inserting an SD card, which is an example of a recording medium. A reader / writer is mounted inside the recording medium insertion port 3525, and a recording medium such as an SD card 406 can be detachably attached to the reader / writer.

[0220] On the left side surface 3545 of the housing 3524, a fifth operation button 3505, which is an example of the operation unit 3515, is provided. This fifth operation button 3505 is an input button for manually instructing the start of recording. When the fifth operation button 3505 is pressed, the control unit 403 may perform event recording for starting the recording of the image data captured by the imaging unit (camera) 402 and the like and the sound collected by the microphone and the like.

[0221] Furthermore, the program according to the present invention causes the user terminal 102 to realize the above-described functions of each module (each means) of the user terminal 102. Also, another program according to the present invention causes the imaging device 101 to realize the above-described functions of each module (each means) of the imaging device 101.

[0222] [Other Embodiments] Furthermore, the embodiments of the present invention may be as follows. The respective embodiments may be combined as appropriate. (A) In one embodiment of the present invention, as a problem in the case of providing both a warning by the imaging device 101 (in this embodiment, the imaging device 101F) and a warning by the user terminal 102 (more specifically, the warning module 311), when sounds (for example, warning voices) are emitted by both the imaging device 101 and the user terminal 102, the sound for the same event may be emitted twice with a time difference between the imaging device 101 and the user terminal 102.

[0223] Therefore, sounds are emitted from both the imaging device 101 and the user terminal 102 in response to the same event, but the two sounds (that is, the notification contents) may be made different. Note that event notification information is usually sent from the imaging device 101 to the user terminal 102, but it may be sent from the user terminal 102 to the imaging device 101. By doing so, sensors for event detection in the imaging device 101 (for example, a G sensor exemplified by an acceleration sensor) can be reduced, and the configuration can be simplified and the cost can be reduced.

[0224] Both the imaging device 101 and the user terminal 102 may be configured to detect an event, send event notification information to each other, and record image information based on the event notification information received from another device (here, the other party's device). Also, both the imaging device 101 and the user terminal 102 may emit sounds respectively in response to the same event, exchange event notification information with each other, and perform recording of image information, etc. In that case, it is preferable that the device that detected the event emits a sound, and the device that has not detected the event but has received event notification information from another device does not emit a sound.

[0225] It is preferable to provide a function that allows the user to set which device, the imaging device 101 or the user terminal 102, emits a sound for which event. This setting function may be provided, for example, in the user terminal 102, and the user terminal 102 may notify the imaging device 101 of this setting. At this time, when selecting to have only one of the devices emit a sound, it is preferable to provide a setting item so that the other devices are automatically set not to emit a sound.

[0226] For the event detected by the imaging device 101, a notification threshold (e.g., for warning) set higher than the detection threshold (e.g., for acceleration) for detecting the occurrence of the event is provided on the user terminal 102 side. For an event that exceeds the detection threshold for the occurrence of the event but does not exceed the notification threshold, it is preferable that the user terminal 102 side does not perform notification. Any threshold can be set from the application program installed on the user terminal 102, and the user terminal 102 preferably has a function of sending threshold information to the imaging device 101 side.

[0227] The imaging device 101 may be provided with a function of automatically matching the notification setting language to the setting language of the user terminal 102. The setting language of the user terminal 102 may be settable, for example, from an application program installed in the user terminal 102. By doing so, it is possible to output sounds such as warnings in a language that the user can understand from each device without individually setting the language for each device. By doing as described above, a highly convenient warning function can be provided to the user.

[0228] (B) In one embodiment of the present invention, the following functions may be installed as functions related to voice recognition of the user terminal 102. The imaging device 101 may not have a voice recognition function, or even if it has a voice recognition function, it has a function of recognizing voices different from those of the user terminal 102. And the imaging device 101 may be provided with a function of performing processing on the imaging device 101 based on the result of voice recognition on the application program side of the user terminal 102.

[0229] The user terminal 102 may generate a command for causing the imaging device 101 to execute processing based on the result of voice recognition and transmit it to the imaging device 101. Examples of the processing include event recording. In this case, when a predetermined phrase such as "Record!" is recognized by the application program of the user terminal 102, event recording may be executed. In particular, the imaging device 101 or the user terminal 102 may attach the recognized phrase as a tag and record the image information. For example, if the imaging device 101 or the user terminal 102 can recognize "drifting", it may attach the tag of "drifting" to the image information. By doing so, for example, the image information can be searched using the tag as a search key.

[0230] Recently, in smartphones and the like, price competition for packet communication fees has become fierce, and it has become possible to tether constantly without any problems. If a smartphone or the like is brought into a vehicle, the smartphone or the like can be used as a router, so that, for example, it is easy to use the function of voice recognition in a cloud computing environment. Therefore, by performing voice recognition on the user terminal 102 side, the voice is converted into a character string, and the character string is sent to the imaging device 101. Then, on the imaging device 101 side, a tag received at that time or including the range immediately before that is added to the video, and the image information is recorded, and it is preferable that the video with this tag added can be searched and displayed by a viewer or the like.

[0231] Note that the present invention is not limited to the above-described embodiments, and various modifications are included. For example, the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Also, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Also, for a part of the configuration of each embodiment, it is possible to add, delete, or replace other configurations.

[0232] Also, each of the above configurations, functions, processing units, processing means, etc. may be realized in hardware, for example, by designing a part or all of them with an integrated circuit. Also, each of the above configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as a program, table, and file that realizes each function can be stored in a memory, a storage device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0233] Also, control lines and information lines show those considered necessary for explanation, and not necessarily all control lines and information lines are shown on the product. In practice, it may be considered that almost all configurations are interconnected. Note that the above-described embodiments disclose at least the configurations described in the claims.

[0234] Note that the above-described embodiments disclose at least the following configurations. (1) A terminal, communication control means for receiving a plurality of pieces of image information transmitted from imaging equipment arranged on a vehicle, image information recording means for storing the plurality of pieces of image information received from the imaging equipment in storage means for the terminal when the state of the vehicle satisfies a predetermined recording condition; A terminal having the above. With such a configuration, when the imaging equipment arranged on the vehicle satisfies the recording condition, the image information captured can be recorded in the terminal without the user instructing the recording through the operation of the terminal. Therefore, for example, without taking out a recording medium such as an SD card on which the imaging equipment is mounted and mounting it on a PC and displaying the video on a viewer installed on the PC, the user can view the video when the recording condition is satisfied using the terminal, which is convenient for the user. The terminal can record image information related to an event, even an event such as a rear draft that is difficult for the driver to directly visually confirm without using a mirror or monitor, in the storage device for the terminal without the driver operating the terminal or the imaging equipment after or during the event. Also, for example, since the driver does not need to operate the terminal himself / herself to record the image information, he / she can concentrate on driving and can respond to dangerous driving such as a rear draft in a safer state and with a more reassuring or calm mood. (2) The terminal has display means and display control means for displaying a screen, The communication control means sequentially receives the plurality of pieces of image information from the imaging equipment, The display control means displays the received plurality of pieces of image information in a display area within the screen, A terminal that stores the plurality of pieces of image information in the storage means for the terminal when the recording conditions are satisfied while the plurality of pieces of image information are being displayed. With this configuration, the user can view the images around the vehicle captured by the imaging device on a terminal at hand, such as a smartphone, without operating the imaging device. Also, the user can leave the image information of the images displayed when the recording conditions are satisfied in the storage unit for the terminal without checking the imaging device or the like again by himself / herself. For example, in the case where a serious event such as tailgating or an accident occurs, the situation can be easily confirmed and displayed on a terminal such as a smartphone. (3) An image information recording means is a terminal that stores the plurality of pieces of image information when it receives notification information notifying that the recording conditions are satisfied from the imaging device. With this configuration, when the terminal receives the notification information, it records the image information received from the imaging device as the image information when the recording conditions are satisfied, using the reception as a trigger, so that the user can easily check it later. (4) A display control means is a terminal that displays an operation area for operating the imaging device or the terminal in the screen. With this configuration, the terminal can provide the user with operations not only on the terminal itself but also on the imaging device via the screen. Thereby, the user can operate and set the terminal at hand and the imaging device as if they were an integrated system. (5) A display control means is a terminal that stops the display of the display area while keeping the display of the operation area after the elapse of a first set time. By configuring in this way, after the elapse of the first set time, the terminal can reduce the power consumption related to the display of the display area. Since the main power consumption related to the screen display may be caused by the display of image information in the display area, the terminal can reduce the main power consumption related to the screen display while maintaining the state of being able to receive operations and settings of the terminal by the user via the operation area during display. As a result, it is possible to suppress the occurrence of problems such as image information not being stored in the storage unit for the terminal due to the battery running out of the terminal. (6) The display control means is a terminal that stops the display of the display area and the operation area after the elapse of the second set time. By configuring in this way, after the elapse of the second set time, the terminal can reduce the power consumption related to the display of the display area and the operation area. When the time during which the user does not look at the terminal screen is long during driving or the like, the display of not only the operation area but also the display area can be stopped, and power consumption can be saved more effectively. (7) The display control means accepts changes to the first set time and the second set time Terminal. By configuring in this way, the user can set the desired power consumption savings each time with a simple operation according to their driving style, driving situation, remaining battery capacity of the terminal, etc. (8) When the imaging device is being operated by the operation element on the imaging device side, the display control means is a terminal that displays predetermined display information in the display area and restricts the input of operations by the operation area of the terminal with respect to the imaging device. By configuring in this way, it is possible to prevent interference in the input of operations on the imaging device by the terminal and the imaging device. The imaging device is arranged at a position where the user's line of sight is not obstructed, such as near the upper end of the front glass or the rear glass, and is often arranged at a position where it is difficult for the user to reach with their hand. Therefore, the user can easily check the operation status and processing status of the imaging device at hand, and it is convenient because it is not necessary to directly operate the imaging device itself. For example, the user can easily and surely know the status of the rear camera without moving to check the status of the imaging device by themselves. (9) The display control means is a terminal that displays information related to the operation state of the imaging device as the predetermined display information. By configuring in this way, the user can know on the terminal side whether the imaging device is being operated by the operation elements on the imaging device side. For example, when a passenger is operating the rear camera, the driver can check whether the operation is being performed as intended while sitting in the driver's seat. (10) The display control means switches whether to arrange the operation area and the display area in separate areas within the screen or to arrange the operation area overlapping the display area according to the orientation of the terminal. By configuring in this way, the terminal can easily switch the display so that the display area and the operation area are in a state with higher visibility and operability for the user according to the orientation of the screen. The user can easily obtain an event image from the imaging device displayed on the entire terminal screen just by changing the orientation of the terminal, for example, when taking a screenshot. This is much more advantageous for the user in terms of labor, image quality, safety, etc. compared to a case where a stirring image or the like shown on the liquid crystal screen of a conventional drive recorder with a liquid crystal display (LCD) is directly photographed by the camera of the terminal. (11) The operation area has a plurality of operation parts, When the display control means overlays and displays the operation area on the display area, the terminal arranges each of the plurality of operation portions in the display area so as to be separated from each other and overlays them on the display area. By configuring in this way, when the operation area is overlaid and displayed on the display area, the visibility of the display area can be displayed with higher visibility. In addition, since the interval between the operation portions of the operation area becomes wider, the user feels less cramped in individual operations, so the terminal can provide comfortable operability and good visibility to the user at the same time. Alternatively, the display in which the operation area (operation portion) is overlaid on the display area can significantly improve the degree of freedom in layout design and the degree of freedom in customization by the user himself / herself. (12) The terminal has a photographing device cooperation means, and performs setting of the photographing device based on an input to the display area and the operation area. By configuring in this way, the terminal can provide the user with not only the possibility of setting the terminal but also the possibility of individual setting of the photographing device. For example, the photographing device may not have higher user interface performance than the terminal, and the photographing device itself may not have an output device such as a display. However, the user can easily perform various settings of the photographing device via the display of the terminal at hand. (13) The photographing device cooperation means sets, in the photographing device, a reference line for outputting that the state of the vehicle satisfies a predetermined recording condition based on an input to the operation area. By configuring in this way, even in a design in which the photographing device itself does not have an output device such as a display, the user can easily perform fine setting of the reference line of the photographing device via the display of the terminal at hand. The user can perform the setting in a state that is easier to see or confirm (for example, less affected by ambient light) or in a more comfortable posture than when using the screen of a photographing device fixed to a vehicle. In addition, since the terminal UI such as a smartphone is often more sophisticated than the photographing device, the operability of the user is improved. (14) The imaging device is a terminal that notifies that the state of the vehicle satisfies a predetermined recording condition by determining at least one of the distance or the positional relationship with respect to at least one of the vehicle in front of the vehicle or the vehicle behind the vehicle based on the set reference line. Approach with other vehicles is often a cause of trouble or accidents, and thus it is particularly necessary to record it. By configuring in this way, the terminal can issue a notification according to the determination based on the performance of the imaging device. In addition, not only can the user obtain a notification from the terminal, but the user can also easily recognize that the image information of the event related to such a notification is recorded. (15) When the state of the vehicle satisfies a predetermined recording condition while another piece of image information from another imaging device is being displayed in the display area, the display control means switches the image information to be displayed in the display area to the image information from the imaging device that has detected that the state of the vehicle satisfies the predetermined recording condition. By configuring in this way, the terminal can display, to the user, image information that is problematic or more interesting on the screen. For example, even in the case of an event such as a rear sway that is difficult for the driver to directly visually confirm without using a mirror or a monitor, the user can check the image information related to the event on the terminal at hand without individually operating the terminal each time. This can provide good driving safety and a sense of security to the user. (16) The display control means can display the image of the front camera of the vehicle and the image of the rear camera of the vehicle. When detecting the approach of another vehicle from behind the vehicle during the display of the image of the front camera of the vehicle, the display control means switches to and displays the image of the rear camera. With such a configuration, even in the case of a rear-end collision that is difficult for a user such as a driver or a passenger to directly visually confirm without using a mirror or a monitor, the image information regarding the rear-end collision can be recorded in the storage device for the terminal without any operation of the terminal or the imaging device by the user after or during the occurrence of the rear-end collision. The user can confirm, on the screen display at hand, the rear-end collision detected with high accuracy without continuously checking the behavior of a following vehicle suspected of rear-end driving while paying attention to the front during driving, and can further leave the image information in the terminal. In a conventional terminal, especially when there is only one driver in the vehicle, even if the driver can turn their attention to the rear and recognize the rear-end collision early, it has been difficult to leave a record of the image information in the terminal at hand on the spot. By using this terminal, the user does not need to turn their attention to the rear more than necessary or perform an operation to start recording the image information themselves. (17) The control means of the imaging device determines at least whether the recording conditions based on the impact generated in the vehicle are satisfied and whether the recording conditions based on the approach of another vehicle to the vehicle are satisfied. The image information recording means is a terminal that records the plurality of pieces of image information in the storage means for the terminal when the recording conditions based on the approach of another vehicle to the vehicle are satisfied. With such a configuration, it is possible to properly use the recording conditions based on an impact such as a collision accident and the recording conditions based on the proximity before a collision accident, which is a situation where an accident is likely to occur. For example, the image information of an event that becomes more important after the occurrence of an event such as tailgating or dangerous driving can be preferentially stored in the storage unit for the terminal. (18) The terminal has warning means for outputting a warning regarding the fact that the state of the vehicle satisfies a predetermined recording condition using at least one of sound, light, and text. By configuring in this way, the terminal can issue a warning regarding the state of the vehicle determined by the imaging device to the user from the terminal. For example, when the imaging device is the rear camera, the user can more reliably receive a warning related to the rear camera from the terminal at hand. Further, when image information of an event related to a warning about weaving driving, dangerous driving, etc. is recorded in the terminal, the driver who is the user can be liberated from focusing on securing evidence, etc., and can concentrate on safe driving. (19) The image information recording means is a terminal that stores the image information not including audio information in the storage means. By configuring in this way, the terminal can store lower-capacity image information and efficiently use the capacity of the storage unit for the terminal. (20) The image information recording means stores the image information provided with index information in the storage means, and based on the index information, acquires second image information corresponding to the plurality of pieces of image information stored by the imaging device. By configuring in this way, the terminal can easily search for and acquire, based on the index information, image information related to the image information stored in the storage unit for the terminal from the image information stored in the storage unit for the imaging device. Also, the user can easily recognize the content of the image information recorded in the terminal from the index information even from the list on the save screen. (21) The second image information is a terminal including audio information. By configuring in this way, even when the storage unit for the terminal stores lower-capacity image information not including audio information, for example, when it becomes necessary to grasp a more detailed situation from the video later, the terminal can obtain audio information from the second image information. Since the user has secured both image information including audio information and image information not including audio information for the image information related to the same event, there is no need to perform processing of the image information regarding the presence or absence of audio information. (22) The second image information is a terminal having image information of a period before the time when the state of the vehicle satisfies a predetermined recording condition. With such a configuration, the terminal can obtain image information of a period before the image information stored in the storage unit for the terminal. Therefore, the user can check the behavior of the target vehicle before it is determined to be, for example, a weaving drive, and can also submit it as evidence if necessary. For example, even when it is difficult to determine whether the target event is intentional or accidental based only on the image information after the event occurs, the behavior before that can be added as a judgment material. The user can grasp the situation when the recording condition is satisfied in more detail. (23) The image information recording means is a terminal that sets the length of time of the plurality of pieces of image information to be stored and controls the time for storing the plurality of pieces of image information based on the set length of time. With such a configuration, after the terminal stores the image information in the storage unit for the terminal, the terminal can provide the image information of the length of time desired by the user without any processing by the user regarding the length of time of the image information. Also, thereby, the user can upload the image information with an appropriate playback time to SNS or the like without any processing. (24) The communication control means is a terminal that transfers the stored image information to an external server. With such a configuration, the terminal can also leave the image information stored in the storage unit for the terminal in an external server. Therefore, for example, even if the terminal or the imaging device is damaged after an event occurs and the image information serving as a record or evidence is lost, the image information of an important scene can be surely left. For example, when the external server that can be set in the terminal is operated by a public investigation agency or a private road service, a prompt report and contact can be made together with the specific situation. (25) A program for causing the terminal to realize the functions of each of the means described in any one of the above. With such a configuration, the above control can be performed by providing a program to an existing terminal. (26) A photographing device arranged on a vehicle, a transmitting means for transmitting a plurality of photographed image information to an external terminal, a detecting means for detecting the state of the vehicle, and a control means for transmitting information for causing the terminal to store the plurality of transmitted image information when the state of the vehicle satisfies a predetermined recording condition. The photographing device having these components. With such a configuration, when the recording condition is satisfied by the photographing device arranged on the vehicle, the image information photographed can be recorded on the terminal without instructing the recording by operating the user's terminal. Therefore, for example, without taking out a recording medium such as an SD card on which the photographing device is mounted and mounting it on a PC and displaying the video on a viewer installed on the PC, the user can use the terminal to view the video when the recording condition is satisfied. It is convenient for the user to confirm. The photographing device can record image information related to an event on the terminal without any operation of the photographing device or the terminal by the driver after or during the event, even in an event such as a rear draft that is difficult for the driver to directly visually confirm without using a mirror or a monitor. As a result, the driver does not need to operate the photographing device himself / herself to record the image information on the terminal, so that he / she can concentrate on driving and respond to dangerous driving such as rear drafts in a safer state and with a more reassuring or calm feeling. (27) The information for storing the plurality of image information in the terminal is notification information for notifying that the recording condition has been satisfied, the transmitting means sequentially transmits the plurality of photographed image information, and the control means is a photographing device that transmits the notification information when the recording condition is satisfied. By configuring in this way, the imaging device notifies the terminal, based on the notification information, that the recording conditions have already been satisfied, and uses this as a trigger for starting the storage of image information on the terminal. Thus, it is possible to cause the terminal to store image information without the need to determine the recording conditions on the terminal. (28) The control means is an imaging device that causes the storage means for the imaging device to store the plurality of pieces of image information captured during a period including the time before and after the time when the state of the vehicle satisfies the predetermined recording conditions. By configuring in this way, the imaging device can store image information for a longer period, including the periods before and after the image information stored in the terminal. As a result, the user can obtain image information for the periods before and after the image information stored in the storage unit for the terminal. The user can, for example, check the behavior of the target vehicle and the speech and actions of the user of the target vehicle before and after it is determined to be, for example, weaving driving, and can also submit it as evidence if necessary. For example, even when it is difficult to determine whether the target event is intentional or due to negligence based on only the image information for a short period after or before the event occurs, the previous behavior can be added as a factor for judgment. Further, when it later becomes an issue what speech and actions the driver of the target vehicle took, it is possible to provide the user with the necessary image information at that stage. (29) The control means causes the storage means for the imaging device to store the plurality of pieces of image information when a first recording condition is satisfied, and is an imaging device that transmits information for causing the terminal to store the plurality of pieces of image information after the time when the second recording condition is satisfied, when the second recording condition is satisfied. By configuring in this way, the imaging device can record image information for the imaging device and the terminal under different recording conditions, so that the user can set more appropriate conditions according to the subject to be memorized (the imaging device or the terminal). For example, by setting the second recording condition to relatively extreme conditions, such as very dangerous conditions, the user can preferentially save the image information that can be expected to have a great response on SNS or the like in the terminal. (30) An imaging device that issues a predetermined warning when a following vehicle approaches the vehicle as the second recording condition, and transmits information for storing the plurality of pieces of image information in the terminal. By configuring in this way, the imaging device can issue a warning about the approach of the following vehicle and prompt the storage of image information related to the approach of the following vehicle in the terminal. Therefore, the imaging device not only issues a warning about the detected event, but also even if the terminal does not have a configuration or function for determining the distance and positional relationship with the following vehicle by itself, it can record the image information of an event such as a following draft detected according to the highly accurate determination based on the performance of the imaging device. As a result, the user can quickly recognize events such as following drafts, which tend to be delayed in self-recognition due to warnings, and can secure image information of high interest in the hand-held terminal without individually operating the terminal by himself / herself. Therefore, the user can concentrate on safe driving with a sense of security that the evidentiary image information is secured. (31) A program for causing the imaging device to realize the functions of each of the means described in any one of the above. By configuring in this way, the above control can be performed by providing a program to the existing imaging device.

Explanation of Signs

[0235] 10…Vehicle, 101, 101F, 101R…Imaging device, 102…User terminal, 103…External server, 104…Network, 301…Main memory device, 302…Auxiliary storage device, 310…Image information recording module, 311…Warning module, 312…Display control module, 313…Imaging device cooperation module, 314…Imaging device control module, 320…Setting information, 321…Image information, 303…Processor, 304…Input device, 305…Output device, 306…Camera unit, 307…Communication control unit, 401…Microcomputer, 402…Camera, 403…Control unit, 404…Imaging device controller

Claims

**Claim 1** A portable terminal, comprising: communication control means having a function of receiving a plurality of pieces of image information constituting a moving image transmitted from an imaging device disposed in a vehicle; image information recording means having a function of storing the plurality of pieces of image information received from the imaging device in storage means for the portable terminal when the state of the vehicle satisfies a predetermined recording condition; and the portable terminal having display means and display control means for displaying a screen, the communication control means having a function of sequentially receiving the plurality of pieces of image information from the imaging device, the display control means having a function of displaying the received plurality of pieces of image information in a display area within the screen, the display control means having a function of storing the plurality of pieces of image information being displayed in the storage means for the portable terminal when the recording condition is satisfied while the plurality of pieces of image information are being displayed, the display control means having a function of displaying, within the screen, an operation area for operating the imaging device or the portable terminal, when the imaging device is being operated by an operation element on the imaging device side, the display control means having a function of displaying predetermined display information in the display area and restricting input of an operation by the operation area of the portable terminal on the imaging device; a portable terminal. **Claim 2** The display control means displays, as the predetermined display information, information related to an operation state of the imaging device. The portable terminal according to claim 1. **Claim 3** A portable terminal, comprising: communication control means having a function of receiving a plurality of pieces of image information constituting a moving image transmitted from an imaging device disposed in a vehicle; image information recording means having a function of storing the plurality of pieces of image information received from the imaging device in storage means for the portable terminal when the state of the vehicle satisfies a predetermined recording condition; and the portable terminal having display means and display control means for displaying a screen, the communication control means having a function of sequentially receiving the plurality of pieces of image information from the imaging device, the display control means having a function of displaying the received plurality of pieces of image information in a display area within the screen, the display control means having a function of storing the plurality of pieces of image information being displayed in the storage means for the portable terminal when the recording condition is satisfied while the plurality of pieces of image information are being displayed, the display control means having a function of displaying, within the screen, an operation area for operating the imaging device or the portable terminal, The portable terminal has camera device cooperation means, and has a function of setting the settings of the camera device based on inputs to the display area and the operation area. The camera device cooperation means has a function of setting, in the camera device, a reference line for notifying that the state of the vehicle satisfies the predetermined recording conditions based on an input to the operation area. Portable terminal. **Claim 4**: A portable terminal, comprising communication control means having a function of receiving a plurality of pieces of image information constituting a moving image transmitted from a camera device disposed in a vehicle; image information recording means having a function of storing, in storage means for the portable terminal, the plurality of pieces of image information received from the camera device when the state of the vehicle satisfies predetermined recording conditions; and the image information recording means stores the image information to which index information is attached in the storage means for the portable terminal, and has a function of acquiring second image information, which is information stored in storage means for the camera device by the camera device and corresponds to the plurality of pieces of image information, based on the index information. Portable terminal. **Claim 5**: The image information recording means has a function of storing, in the storage means, the image information that does not include audio information, the image information to which the index information is attached does not include audio, and the second image information includes audio information. The portable terminal according to claim 4. **Claim 6** The portable terminal according to claim 4 or 5, wherein the second image information includes image information for a period prior to the time when the state of the vehicle satisfies the predetermined recording conditions. **Claim 7** A program for causing the portable terminal to realize the functions of the respective means according to any one of claims 1 to 6. **Claim 8** A camera device disposed in a vehicle, comprising transmission means having a function of transmitting a plurality of pieces of image information constituting a moving image that has been photographed to a portable terminal; detection means having a function of detecting the state of the vehicle; control means having a function of transmitting, when the state of the vehicle satisfies predetermined recording conditions, information for causing the portable terminal to store the plurality of pieces of image information that has been transmitted; and an operation element having a function of operating the camera device; and the control means has a function of executing an operation of the camera device based on an operation input to the camera device from the portable terminal. When the imaging device is being operated by the operation element, the control means has a function of transmitting predetermined information for restricting the input of the operation of the imaging device based on the operation input to the imaging device from the portable terminal to the portable terminal. Imaging device.

9. The predetermined information is information related to the operation state of the imaging device. The imaging device according to claim 8.

10. An imaging device disposed in a vehicle, a transmission means having a function of transmitting a plurality of pieces of image information constituting a video taken to a portable terminal; a detection means having a function of detecting the state of the vehicle; a control means having a function of transmitting information for storing the plurality of pieces of transmitted image information in the portable terminal when the state of the vehicle satisfies a predetermined recording condition; having the control means has a function of receiving an input for setting the imaging device from the portable terminal; the control means has a function of setting a reference line for notifying that the state of the vehicle satisfies the predetermined recording condition in response to an input from the portable terminal. Imaging device.

11. Based on the set reference line, the imaging device determines at least one of the distance or the positional relationship with respect to at least one of the vehicle in front of the vehicle or the vehicle behind the vehicle, and notifies the portable terminal that the state of the vehicle satisfies the predetermined recording condition. The imaging device according to claim 10.

12. An imaging device disposed in a vehicle, a transmission means having a function of transmitting a plurality of pieces of image information constituting a video taken to a portable terminal; a detection means having a function of detecting the state of the vehicle; a control means having a function of transmitting information for storing the plurality of pieces of transmitted image information in the portable terminal when the state of the vehicle satisfies a predetermined recording condition; having the transmission means has a function of transmitting the image information provided with index information to the portable terminal; the control means receives transmission command information of second image information based on the index information from the portable terminal, and transmits the second image information stored in the storage means for the imaging device, which corresponds to the plurality of pieces of image information provided with the index information, to the portable terminal. Imaging device.

13. The transmission means has a function of transmitting the image information not including voice information to the portable terminal, the image information with the index information added does not include voice information, the second image information includes voice information The imaging device according to claim 12.

14. The imaging device according to claim 12 or 13, wherein the second image information has image information of a period before a time point when the state of the vehicle satisfies the predetermined recording condition.

15. A program for causing the imaging device to realize the functions of the respective means according to any one of claims 8 to 14.

Images