Terminal, photographing device, program and the like

Abstract

To provide a terminal, a photographing device, and a program that enable image information received from the photographing device to be stored in a storage unit for the terminal.SOLUTION: A terminal includes: communication control means for receiving a plurality of pieces of image information transmitted from a photographing device located in a vehicle; and image information recording means for storing the plurality of pieces of image information received from the photographing device in storage means for the terminal when the state of the vehicle meets a predetermined recording condition.SELECTED DRAWING: Figure 30

Description

[Technical Field]

[0001] The present invention relates to a terminal, a photographing device, a program, and the like. [Background technology]

[0002] JP 2020-173844 A (Patent Document 1) is a background technology in this technical field. This publication states that "the device comprises an observation means for observing the situation of a moving object operated by a user, and a control means for performing control to send to the user a message for switching the psychological state that the user would have to another psychological state when the observation result by the observation means satisfies a predetermined condition" (see abstract). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-173844 Summary of the Invention [Problem to be solved by the invention]

[0004] The aforementioned Patent Document 1 describes a device that has a function of monitoring the rear of a vehicle from video and detecting tailgating by a following vehicle, and that has a function of sending a message to the driver when tailgating by a following vehicle is detected. However, the aforementioned Patent Document 1 does not consider a mechanism for a terminal or the like that stores image information transmitted from the imaging device when the vehicle state satisfies specific recording conditions. Therefore, the present invention provides a function that is highly convenient for users, for example, a mechanism such as a terminal that stores image information transmitted from a photographing device when the vehicle condition satisfies specific recording conditions. [Means for solving the problem]

[0005] In order to solve the above problems, for example, the configurations described in the claims are adopted. The present application includes multiple means for solving the above problem, and one example is characterized in that the terminal has a communication control means for receiving multiple pieces of image information transmitted from a photographing device installed in a vehicle, and an image information recording means for storing the multiple pieces of image information received from the photographing device in a storage means for the terminal when the state of the vehicle satisfies predetermined recording conditions. [Effects of the Invention]

[0006] According to the present invention, it is possible to provide a function that is highly convenient for users. For example, it is possible to provide a mechanism for a terminal or the like that stores multiple pieces of image information transmitted from a photographing device when the state of a vehicle satisfies a specific recording condition. Problems, configurations, and effects other than those described above will become clear from the description of the following embodiments. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of the installation position of an image capturing device 101 and a user terminal 102 in a vehicle 10 according to this embodiment. [Figure 2] FIG. 2 is an example of a connection diagram of a photographing device 101, a user terminal 102, and an external server 103. [Figure 3] FIG. 3 shows an example of the hardware configuration of the user terminal 102. [Figure 4] FIG. 4 shows an example of the hardware configuration of the image capturing device 101. As shown in FIG. [Figure 5] FIG. 5 shows an example of the hardware configuration of the external server 103. [Figure 6] FIG. 6 shows an example of a first flow 60 of image recording by the photographing device 101, output of event notification information, and addition of index information. [Figure 7] FIG. 7 shows an example of a second flow 70 of image recording by the photographing device 101, output of event notification information, and addition of index information. [Figure 8]FIG. 8 shows an example of a flow 80 of recording image information of the photographing device 101 by the user terminal 102. [Figure 9] FIG. 9 shows an example of a setting flow 90 for the photographing device 101 by the user terminal 102 . [Figure 10] FIG. 10 shows an example of a flow 100 for stopping the display of image information by the user terminal 102 . [Figure 11] FIG. 11 is an example of a screen showing some of the operations for setting reference lines 1101 and 1102 for event determination. [Figure 12] FIG. 12 is an example of a screen showing a part of the operation for setting reference lines 1201 and 1202 for event determination, following the example of the operation shown in FIG. [Figure 13] FIG. 13 is an example showing properly established reference lines 1301, 1302, and D along with imaginary decision lines A, B, and C. [Figure 14] FIG. 14 shows an example of switching between display layouts 1420 and 1430 according to the orientation of the screen of the user terminal 102. [Figure 15] FIG. 15 shows an example of a preview screen 1500 of the user terminal 102. [Figure 16] FIG. 16 shows an example of an operation for displaying setting screens 1601 and 1602 for the image capturing device 101 from the preview screen. [Figure 17] FIG. 17 shows examples of setting menus 1702 to 1704 that are displayed when the "smartphone event record setting" menu shown in FIG. 16 is selected. [Figure 18] FIG. 18 is an example of a screen 1800 in which an event is displayed. [Figure 19] FIG. 19 shows an example of a setting screen 1900 of the user terminal 102. [Figure 20] FIG. 20 shows an example of a setting screen 2000 for two types of screen settings. [Figure 21] FIG. 21 shows an example of a selection screen 2100 for camera image display time. [Figure 22] FIG. 22 shows an example of a power-saving screen transition time selection screen 2200. [Figure 23] FIG. 23 shows examples of screen displays 2301 and 2302 before and after the camera image display time has elapsed. [Figure 24] FIG. 24 shows examples of screen displays 2401 and 2402 before and after the power-saving screen transition time has elapsed. [Figure 25] FIG. 25 shows examples of screens 2502 to 2504 for selecting events to be warned about. [Figure 26] FIG. 26 shows an example of the response of the image capturing device 101 and the indicator display when rear-end approach is detected. [Figure 27] FIG. 27 shows an example of the response of the image capturing device 101 and the indicator display when rear approach and abnormal rear approach are detected. [Figure 28] FIG. 28 shows an example of a selection screen 2800 for a terminal storage unit. [Figure 29] FIG. 29 shows an example of a selection screen 2900 for selecting the storage time of the image information received from the photographing device 101. In FIG. [Figure 30] FIG. 30 shows examples of display screens 3001 and 3002 when image information received from the photographing device 101 is being stored. [Figure 31] FIG. 31 shows an example of a setting screen 3100 for selecting an event storage and setting an event recording. [Figure 32] FIG. 32 shows an example of a notification screen 3200 regarding insufficient capacity of the auxiliary storage device 302 . [Figure 33] FIG. 33 shows an example of a notification screen 3300 that notifies the user that the photographing device 101 is accepting an operation via an operation element provided on the photographing device 101. [Figure 34] FIG. 34 is a diagram showing two examples of the appearance of the image capturing device 101, (A) and (B). [Figure 35] FIG. 35 is a diagram showing another example of the appearance of the photographing device 101. In FIG. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment shown below is one embodiment of the present invention, and the content of the present invention should not be interpreted as being limited based on the following description.

[0009] FIG. 1 is an example of an explanatory diagram illustrating the installation positions of photographing devices 101 (101F, 101R) and a user terminal (terminal) 102 in a vehicle 10 (hereinafter, also referred to as the vehicle 10) according to this embodiment. FIG. 1(a) is a side view of the vehicle 10, and FIG. 1(b) is a top view of the vehicle 10. As shown in FIG.

[0010] 1(a) and 1(b), a vehicle 10 has an image capturing device 101F at the front of the vehicle interior and an image capturing device 101R at the rear of the vehicle interior. The image capturing device 101 is an electronic device with an image capturing function, and may be, for example, a drive recorder, a front camera, a rear camera, or a side camera that cooperates with the drive recorder, or a camera provided for security, surveillance, or other sensing purposes. The vehicle 10 may be configured to include only the front-facing photographing device 101F or the rear-facing photographing device 101R of the vehicle 10. Alternatively, the vehicle 10 may include three or more photographing devices. The photographing devices 101F and 101R may be simply referred to as the photographing devices 101, regardless of their installation locations.

[0011] 1 is, for example, a drive recorder, and is an imaging device arranged on the front side of the vehicle 10. In this example, the imaging device (drive recorder) 101F is arranged on the front side inside the vehicle 10 (i.e., the passenger compartment) and attached to a predetermined location such as the windshield. The imaging device (drive recorder) 101F captures images of the area outside the vehicle 10 in front of it. The photographing device 101 (101R) is, for example, a rear camera, and is a photographing device arranged on the rear side of the vehicle 10. In this example, the photographing device (rear camera) 101R is arranged on the rear side inside the vehicle 10 and attached to a predetermined location such as the rear window. The photographing device (rear camera) 101R photographs the outside rear of the vehicle 10.

[0012] The photographic device 101F and the photographic device 101R are connected via a cable 11. The cable 11 is a cable for electrically connecting the photographic device 101F and the photographic device 101R. The cable 11 includes, for example, a power line for supplying power from the photographic device 101F to the photographic device 101R and a signal line for transmitting various signals. The photographic device 101R operates by receiving power supply via the cable 11. The photographing device 101R has a function of taking a photograph and a function of outputting image data representing the photographed image to the photographing device 101F via the cable 11. The photographing device 101F has a function of taking a photograph, a function of recording image data representing the photographed image, and a function of recording image data received from the photographing device 101R. As will be described later, the functions, configurations, etc. of the photographing devices 101F and 101R according to this embodiment are not limited to the above example, and the total number of photographing devices 101 may be one or three or more.

[0013] The user terminal 102 may be, for example, a portable terminal (mobile terminal) such as a smartphone, tablet, mobile phone, or personal digital assistant (PDA), or may be a wearable terminal such as glasses, a wristwatch, or clothing. It may also be a stationary or portable computer, or a server located on the cloud or a network. It may also function as a VR (Virtual Reality) terminal, an AR (Augmented Reality) terminal, or an MR (Mixed Reality) terminal. Alternatively, it may be a combination of multiple of these terminals. For example, a combination of one smartphone and one wearable terminal can logically function as a single terminal. It may also be any other information processing terminal.

[0014] The user terminal 102 includes a processor 303 that executes an operating system, application programs, etc., a main memory device 301 such as RAM (Random Access Memory), an auxiliary memory device 302 such as an IC card, hard disk drive, SSD (Solid State Drive), or flash memory (SD card, etc.), a communication control unit 307 such as a network card, wireless communication module, or mobile communication module, an input device 304 such as a touch panel, keyboard, mouse, voice input, or input based on motion detection captured by a camera unit, and an output device 305 such as a monitor, display, or speaker. The output device 305 may also be a device or terminal that transmits information to be output to an external monitor, display, speaker, printer, device, etc.

[0015] The main memory device 301 stores various programs, applications, etc. (modules), and the processor executes these programs and applications to realize the various functional elements of the overall system. These modules may be implemented in hardware, such as by integration. Each module may be an independent program or application, or may be implemented as a subprogram or function within a single integrated program or application.

[0016] In this specification, each module is described as the entity (subject) that performs the processing, but in reality, the processing is carried out by a processor that processes various programs, applications, etc. (modules). Various databases (DBs) are stored in the auxiliary storage device. A "database" is a functional element (storage unit) that stores a set of data so that it can accommodate any data manipulation (e.g., extraction, addition, deletion, overwriting, etc.) from a processor or an external computer. There are no limitations on how the database is implemented; for example, it can be a database management system, spreadsheet software, or a text file such as XML or JSON.

[0017] FIG. 2 is an example of a connection diagram of the photographing devices 101 (101F, 101R), the user terminal 102, and the external server 103. The photographic device 101, the user terminal 102, and the external server 103 are each connected via a network 104. As will be described later, in this embodiment, of the photographic devices 101F and 101R, the photographic device 101F connects to the network 104 to perform communication. When data transmission / reception is required between the photographic device 101R and the network 104, or when data transmission / reception is required between the photographic device 101R and the user terminal 102, the photographic device 101F performs communication to relay the data transmission / reception between the photographic device 101R and the network 104. However, a configuration in which the photographic device 101R transmits and receives data directly to the user terminal 102 is also acceptable. Note that the network 104 may be wired or wireless, and each user terminal 102 can transmit and receive information via the network 104. The network 104 is not limited to the Internet and may be a combination of networks using different protocols.

[0018] As described above, the image capturing device 101 is a device that is mounted on, for example, the vehicle 10 and records images while driving. However, although the vehicle 10 to be installed is a four-wheeled automobile in this embodiment, it is not limited to a four-wheeled automobile, and may be any vehicle in which the imaging device 101 can be installed. For example, large transport vehicles with four or more wheels, such as buses and trucks, two-wheeled vehicles such as motorcycles and bicycles, and other vehicles may be suitable for installation. Furthermore, for example, transportation vehicles such as trains, monorails, and linear motor cars may also be suitable for installation. The photographing device 101 can transmit various acquired information, information relating to the results of event determination (described later), and the like, to the user terminal 102 via the network 104 .

[0019] The user terminal 102 is a terminal used by, for example, a user driving a car. The user terminal 102 is connected to the photographing device 101 via a mobile data communication network or an in-vehicle network 104 such as Wi-Fi (registered trademark) in the car, and can operate the photographing device 101. Furthermore, the user terminal 102 is connected to an external server 103 via these networks 104 and can transmit various types of information to the external server 103 .

[0020] FIG. 3 shows an example of the hardware configuration of the user terminal 102. The user terminal 102 is configured as a terminal such as a smartphone, a tablet, a notebook PC, or a desktop PC. The main memory device 301 stores programs and applications such as an image information recording module 310, a warning module 311, a display control module 312, a photographing device linkage module 313, and a photographing device control module 314, and the processor 303 executes these programs and applications to realize each functional element of the user terminal 102.

[0021] The image information recording module 310 stores image information transmitted from the photographing device 101 via the network 104 in the auxiliary storage device 302 for the terminal. The auxiliary storage device 302 is an example of a storage means for the terminal. The storage means for the terminal is a storage means reserved for the user terminal 102 and can be used by the user terminal 102 to record data. The storage means for the terminal may be a storage means internal to the user terminal 102 or a storage means external to the user terminal 102. The storage means external to the user terminal 102 may be, for example, a storage means accessed by the user terminal 102 using a communication means, such as a storage means reserved on a network.

[0022] The warning module 311 uses the output device 305 such as a display and a speaker to warn the user about various events determined by the photographing device 101 . The display control module 312 controls the display content and display layout on the output device 305 such as a display of the user terminal 102 . The image capturing device cooperation module 313 executes connection and functional cooperation with one or more image capturing devices 101 . The photographing device control module 314 executes control from the user terminal 102 over the photographing devices 101 that are connected and functionally linked via the photographing device linking module 313 . The specific functions of these individual modules will be described in detail using other drawings.

[0023] As described above, the main memory device 301 stores applications, programs, etc. (modules) for displaying and recording image information transmitted from the image capturing device 101. Such applications may be referred to as APP in the drawings.

[0024] The display control module 312 displays an operation area and a display area on the output device 305 of the user terminal 102. The operation area is an area for displaying operation elements (operation parts) for accepting operations and settings for the user terminal 102 and the photographing device 101, for example. The display area is an area for displaying image information transmitted from the photographing device 101, for example. The operation area and the display area will be described in more detail in relation to FIG. 14. Various examples of the operation area and the display area are also shown in other drawings.

[0025] The auxiliary storage device 302 stores setting information for each module stored in the main storage device 301 and image information received from and stored in the image capture device. In this specification, image information refers to, for example, still image information, video information, etc. The image information recording module 310 stores still images in still image file formats such as JPEG, PNG, and GIF, and stores moving images in moving image file formats such as MOV, MP4, AVI, WMV, and WemM.

[0026] FIG. 4 shows an example of the hardware configuration of the image capturing device 101. As shown in FIG. The photographing device controller 404 is connected to and controls the camera 402, acceleration sensor 405, SD card 406, GNSS sensor 407, and other sensors (not shown). Based on information received from these elements, the photographing device controller 404 generates status information indicating the state of the photographing device controller 404. The photographing device controller 404 also transmits the generated status information to the microcomputer 401.

[0027] The microcomputer 401 performs the following various processes, for example. Controls the communication processing unit 414. The communication processing unit 414 notifies the photographing device controller 404 of the acquired location information. The status information acquired from the photographing device controller 404 and the position information acquired from the communication processing unit 414 are converted into, for example, 16-byte data. - Performs sleep control when powered by secondary battery. - Execute test mode during manufacturing and testing.

[0028] The microcomputer 401 is programmable, and stores programs for executing the various processes described above, which are executed by a processing unit of the microcomputer 401 to achieve the various processes described above. In this embodiment, the processing unit of the microcomputer 401 or the microcomputer 401 itself may also be referred to as a control unit. The front photographing device 101F receives image information from the rear photographing device 101R via a network 104 such as a cable 11, and the photographing device controller 404 and microcomputer 401 of the front photographing device 101F perform various processes on the received image information. However, the rear photographing device 101R may also be configured to include the photographing device controller 404 and the microcomputer 401, and the rear photographing device 101R itself may perform the above-described processing on the image information.

[0029] The photographing device controller 404, the microcomputer 401, and the communication processing unit 414 can each be implemented as an integrated circuit (chip) such as a SoC (System on a Chip), or a plurality of these may be implemented together on a single chip. In particular, in this embodiment, the photographing device controller 404 and the microcomputer 401 operate in cooperation with each other, so they may be implemented together on a single chip.

[0030] Also, the microcomputer 401 may be configured to execute some of the functions of the photographing device controller 404, or the photographing device controller 404 may be configured to execute some of the functions of the microcomputer 401. In this embodiment, the photographing device controller 404 and the microcomputer 401 are collectively referred to as the control unit 403. The control unit 403 may be a single chip, or may be configured to include multiple chips in which the functions of the photographing device controller 404 and the microcomputer 401 are separated.

[0031] By equipping the image sensor of the camera 402 with an AI processing function that performs preprocessing, it is possible to generate the following output, for example, in addition to or instead of generating a normal captured image. The AI ​​processing function has the function of performing image recognition based on the video captured by these cameras. -Export objects from images as metadata. - Outputs images in ISP (Image Signal Processor) output format, such as YUV or RGB. - Output an image cropped from a specific area.

[0032] By performing preprocessing on the image sensor side in this way, it is possible to reduce the amount of data and enable real-time tracking of objects using high-speed AI processing. For example, if a vehicle is equipped with multiple image capturing devices 101 (101F, 101R) and each of the image capturing devices 101F and 101R is equipped with an AI processing function for preprocessing the image sensor of the camera 402, each of the image capturing devices 101F and 101R can independently perform video processing and image processing, allowing each of the image capturing devices 101F and 101R to create image information with ideal image quality. This also prevents the computational load of video processing and image processing from being concentrated on one of the image capturing devices 101F and 101R.

[0033] For example, if photographing device 101F is a front camera and photographing device 101R is a rear camera, and each performs image processing, then compared to a system in which only the front camera processes images from the front and rear cameras, the front camera processes only the front image, resulting in reduced load and stable operation, and the rear camera processes the image using its own image processing engine, thereby achieving ideal image quality.

[0034] It is preferable to use a network 104 that supports communication between the image capturing devices 101F and 101R, such as one that complies with a digital transmission (Ethernet (registered trademark) system) communication standard. When the image capturing devices 101F and 101R are connected with a cable 11 that complies with the Ethernet standard and image information is transmitted from the image capturing device 101R to the image capturing device 101F, the image can be recorded clearly without degradation, even with the same FULL HD image quality. For example, such digital transmission enables clearer and sharper rearward video recording than conventional analog transmission models.

[0035] The acceleration sensor 405 detects acceleration and generates acceleration information. For example, when a sudden change in acceleration occurs due to an impact, sudden steering, sudden stopping, etc., the control unit 403 detects the occurrence of an event. The control unit 403 can estimate the circumstances of an accident, etc. that has occurred to a car, by analyzing this change in acceleration information. The SD card 406 is a storage device that records various information such as image information and recording information corresponding to the image information. Note that the SD card is not limited to an SD card and any configuration that can store data will do, but a flash memory with high vibration resistance is preferable for in-vehicle use. In addition, if there are multiple photographing devices 101F, 101R in the vehicle 10 and the multiple photographing devices 101F, 101R can share a storage device, it is not necessary for all of the photographing devices 101F, 101R to have a storage device such as an SD card 406.

[0036] For example, if the photographing device 101F is a drive recorder and the photographing device 101R is a rear camera without a recording function, the rear camera does not need to have a storage device such as the SD card 406. Furthermore, the configuration for determining the vehicle condition described below, the configuration for storing image information captured by the photographing devices 101F and 101R, and the configuration for transmitting image information to the user terminal 102 may be integrated into the front drive recorder, and these configurations may be omitted from the rear camera, with the rear camera being configured to at least transmit image information to the drive recorder.

[0037] Regarding other configurations, functions, etc., if the configurations, functions, etc. can be shared by multiple image capturing devices 101, it is also possible to omit some or all of the configurations, functions, etc. for each image capturing device. In this embodiment, the photographing device 101F basically performs advanced functions such as image processing and transmission processing to the user terminal 102 and has a storage device, and the rear side is described as having a simple photographing function, but conversely, the rear side may have advanced functions concentrated, and the front side may be a front camera that only performs simple photographing.

[0038] The communication processing unit 414 transmits information about the image capturing device 101 to the user terminal 102 via the wireless communication antenna 408 and the network. The GNSS sensor 407 has a location information acquisition unit implemented as an integrated circuit or the like, and acquires location information based on a signal received from a GNSS antenna compatible with a GNSS (Global Navigation Satellite System) such as a GPS (Global Positioning System). The location information acquisition unit can also acquire time information, moving speed information, and moving direction information.

[0039] The image capture device 101 has a power supply 411 that is connected to a vehicle power supply 410 . The power supply 411 supplies power received (or sometimes referred to as power reception) from the vehicle power supply 410 to the photographing device controller 404 and also charges the secondary battery 412.

[0040] The secondary battery 412 is a rechargeable battery such as a lithium ion battery, a nickel-cadmium battery, a nickel-metal hydride battery, or a lead-acid battery. However, a non-rechargeable primary battery may also be used. When the accessory power supply (ACC power supply) is turned off, or when the supply of power from the vehicle power supply 410 to the power supply 411 of the photographing device 101 is interrupted due to a fault such as a broken wire or a disconnected wire caused by an accident, or when the power supply 411 is unable to supply power due to a malfunction or the like, the secondary battery transmits (or may also be referred to as power transmission or power supply) power to the microcomputer 401, the communication processing unit 414, etc., and maintains at least some of the functions of the photographing device 101.

[0041] The control unit 403 acquires image information from the camera 402 of the photographing device 101 and stores it in a storage device such as an SD card 406. Note that in this specification, "storage" is sometimes referred to as "recording." The control unit 403 of the photographing device 101 further detects the state of the vehicle 10 based on the image information from the camera 402, such as the distance or positional relationship to at least one of the vehicles in front of the vehicle 10 or the vehicles behind the vehicle 10, the length of time that either the front or rear vehicle is in the judgment area or other reference area or proximity area described below, the speed of the vehicle, and the speed of either the front or rear vehicle.

[0042] The control unit 403 determines whether the detected state of the vehicle 10 satisfies a specific event recording condition, which will be described later. The control unit 403 can also set individual event recording conditions for each of a plurality of different events, which will be described later. The control unit 403 can also use the functions of a forward collision warning system (FCWS) or a rearward collision warning system (RCWS), for example, to measure the length of time that either the front or rear vehicle is in a specific reference area (proximity area).

[0043] FIG. 5 shows an example of the hardware configuration of the external server 103. 5, the external server 103 is shown as an example of a server operated to provide an SNS service, but it may also be an external server of another private company, public institution, etc. The external server 103 has a main memory device 501, an auxiliary memory device 502, a processor 503, an input device 504, an output device 505, a communication control unit 506, and a bus connecting these devices to one another.

[0044] The main memory device 501 stores a command receiving module 510 and a content posting module 511, and the processor 503 executes these programs and applications to realize the various functional elements of the external server 103.

[0045] The command receiving module 510 receives and processes commands from users of the SNS service. The command receiving module 510 stores data provided by the users, such as image data and video data, in the auxiliary storage device 502 as content information. The content posting module 511 outputs the processing result of the command from the user by the command receiving module 510 in a state that can be accessed by other users of the SNS service.

[0046] Depending on the result of processing a command from the user, the content posting module 511 can also output the content information stored in the auxiliary storage device 502 in a state that is accessible by other users of the SNS service. The auxiliary storage device 502 stores various program information 520 and content information 530 related to the SNS service.

[0047] Next, the operation of this embodiment will be described. Among the operations described below, the operation of the user terminal 102 may be realized based on an application (a smartphone app for a photographing device) installed in the user terminal 102. FIG. 6 shows an example of a first flow 60 of image recording by the photographing device 101, output of event notification information, and addition of index information.

[0048] The control unit 403 of the photographing device 101 executes the following process in accordance with the first flow 60. Before explaining the first flow 60, the preceding process will be explained. The photographing device 101 is configured to connect to the user terminal 102 via an in-vehicle network 104 such as Wi-Fi (registered trademark) when the photographing device 101 is powered on (for example, Wi-Fi There are two settings: a setting in which the photographing device 101 is connected to the user terminal 102 when the power is on (Wi-Fi ON mode), and a setting in which the photographing device 101 is not connected to the user terminal 102 when the power is on (for example, Wi-Fi OFF mode). The latter is the default setting. In the latter case, the user can connect to the photographing device 101 by operating the user terminal 102. In the following explanation, either setting of the photographing device 101 is acceptable, but it is assumed that the photographing device 101 and the user terminal 102 are connected.

[0049] With these settings, the photographing device 101 automatically or manually transmits data via a Wi-Fi (registered trademark) connection, and the user of the user terminal 102 can check the image information captured by the photographing device 101 at hand and post the confirmed image information directly to an SNS or the like in an appropriate manner. Furthermore, by the following process, video recording is performed on the user terminal 102, for example, a smartphone, by automatic transmission from the photographing device 101. The automatic transmission records video for 30 to 60 seconds after sensor detection, for example, video recording of aggressive driving. These points will be described in detail later.

[0050] Furthermore, when there are multiple photographic devices 101 in the vehicle as described above, only one of the photographic devices 101, for example, the photographic device 101F, may execute the first flow 60, and the other photographic devices 101, for example, the photographic device 101R, may simply send image information to the photographic device 101F executing the first flow 60 via, for example, the cable 11 in Fig. 1. In this case, the photographic device 101F executing the first flow 60 may record the image information of the photographic device 101R and may communicate with the user terminal 102 to transmit the image information. In the following embodiment, the photographing device 101 as the photographing device 101F connects to the user terminal 102 via communication and communicates with it, and when data transmission and reception is required between the photographing device 101R and the user terminal 102, the photographing device 101F performs communication to relay the transmission and reception of data between the photographing device 101R and the user terminal 102.

[0051] Next, the first flow 60 will be described. The control unit 403 acquires image information (including date, time, camera identification number, etc.) from the camera 402 (S600) and stores it (S610). The steps described below can be applied to various different image recording processes and video recording processes performed by the photographing device 101, and specific descriptions of the image recording processes and video recording processes will be omitted here.

[0052] The communication processing unit 414 sequentially transmits the plurality of pieces of image information of the image capturing device 101 to the user terminal 102, with which a connection has already been established, via the in-vehicle network 104, such as the vehicle's Wi-Fi (registered trademark), based on, for example, the event recording conditions described below (S620). The event recording conditions will be described in detail with reference to FIGS. 6 and 25. When there are two photographing devices 101F and 101R in the vehicle, the multiple image information transmitted by the photographing device 101 to the user terminal 102 may be transmitted, for example, by one of the photographing devices 101F, 101R, in such a manner that only the image information captured by the photographing devices 101F, 101R is displayed on a preview screen described below, or the multiple image information transmitted by the photographing devices 101F, 101R may always be transmitted.

[0053] The control unit 403 measures traveling information such as the speed and acceleration of the vehicle 10 using the acceleration sensor 405 (S630), and stores this speed and acceleration information (S640). Step S620 may be performed after steps S630 and S640. Alternatively, in another simpler embodiment, step S620 may be omitted. If step S620 is omitted, the photographing device 101 transmits image information upon the start of step S640 below.

[0054] The acceleration information in steps S630 and S640 is the forward / backward component of the traveling direction of the vehicle 10 obtained from the acceleration sensor 405, but a three-axis composite value other than gravitational acceleration can also be used. More specifically, the control unit 403 combines the obtained speed and acceleration with information obtained from various sensors of the image capturing device 101 to generate recording information. The control unit 403 associates the recording information with the image information obtained from the camera 402 and stores them in the SD card 406 (storage unit) (S640). For example, the control unit 403 associates the date, time, camera identification number, type of front or rear, and driving information (speed, acceleration, position coordinates, etc.) and stores them in the storage means.

[0055] The control unit 403 analyzes the acquired image information and executes a vehicle determination process to determine whether the vehicle 10 satisfies the event recording conditions (S650). The event recording conditions and the vehicle determination process will be described later with reference to a separate drawing. The event recording conditions are conditions determined based on the state of the vehicle 10, such as whether an approaching vehicle is present either in front or behind. However, as will be described later, the event recording conditions may be other conditions.

[0056] The control unit 403 determines whether the state of the vehicle 10 satisfies the event recording conditions based on the acquired image information (S660). If the state of the vehicle 10 does not satisfy the event recording conditions (No in S660), the control unit 403 repeats steps S630 to S650. If the state of the vehicle 10 satisfies the event recording conditions (Yes in S660), the control unit 403 transmits event notification information (notification information notifying that the recording conditions have been met, or information for storing the transmitted plurality of pieces of image information in the terminal) to the user terminal 102. The event notification information is notification information notifying that the vehicle 10 has met the recording conditions, and is data that serves as a trigger for the image information recording module 310 of the user terminal 102 to start recording the image information transmitted from the photographing device 101 when the vehicle 10 has met the event recording conditions.

[0057] Particularly advantageously, the vehicle determination process (S650) in this embodiment may perform event determination, particularly determination of dangerous driving such as tailgating from behind, on images recorded by the image capturing device 101. In the vehicle determination process (S650), the control unit 403 executes vehicle determination process, for example, by analyzing acquired image information and determining whether a vehicle is present in front of or behind the vehicle 10.

[0058] The determination of whether or not there is an approaching vehicle is based on, but not limited to, detection by a forward collision warning system (FCWS) or a rearward collision warning system (RCWS). For example, when the size of a vehicle in an image captured by the camera 402 is equal to or larger than a predetermined size, it can be determined that the vehicle has approached within a distance of, for example, 30 m. Alternatively, proximity determination can be made by measuring distance using a distance measurement sensor that uses radio waves, sound waves, or the like, using a method other than image processing.

[0059] The above vehicle determination process (S650) makes it possible to start recording an event on the terminal after an event detected by the forward collision warning system (FCWS) or rearward collision warning system (RCWS) occurs. This makes it possible, for example, for the device to start recording when the Rear Collision Warning System (RCWS) detects that a vehicle behind is approaching abnormally, or for the device to start recording when the Front Collision Warning System (FCWS) detects that a vehicle ahead is suddenly braking. According to this embodiment, it is possible to reliably store image information of an event in which image information after the event occurs is more important.

[0060] The proximity range for determining whether or not a vehicle is tilting can be made variable. For example, the proximity range setting can be changed depending on the speed of the vehicle 10. For example, when the vehicle 10 is traveling at a high speed, the distance between the vehicles increases, so the proximity range can be set wide, and when the vehicle is traveling at a low speed, the distance between the vehicles decreases, so the proximity range can be set narrow. A more specific example of the vehicle determination process will be described in more detail below, particularly in conjunction with FIG.

[0061] If the vehicle determination result shows that the state of the vehicle 10 does not satisfy the event recording condition (No in S660), the control unit 403 returns the first flow 60 to S630. If the vehicle determination result indicates that the state of the vehicle 10 satisfies the event recording conditions (Yes in S660), the control unit 403 adds index information related to the event to the image information from the camera 402 stored in a storage unit of the image capturing device 101, such as the SD card 406 (S670). The index information can be created based on information such as the type of event that occurred, numerical values ​​characteristic of each event, for example, a sudden change in acceleration, the distance to the preceding and following vehicles, the GPS coordinates of the location where the event occurred, and the date and time of the event.

[0062] The index information related to the event makes it easy to call up the photographing device image information related to each individual event stored in the memory unit of the user terminal 102 from the photographing device image information stored in the memory unit of the photographing device 101, such as the SD card 406. The control unit 403 adds index information relating to the event to the image information stored in the storage unit of the photographing device 101 (S670), and then transmits event notification information to the user terminal 102 (S680).

[0063] The control unit 403 of the photographing device 101 can use the index information already attached to the photographing device image information recorded on the SD card 406 of the photographing device 101 as event notification information to be sent to the user terminal 102 or as part of the event notification information. In this case, the control unit 403 of the photographing equipment 101 does not need to incur the burden of creating new event notification information to send to the user terminal 102, or such burden is reduced, which is significantly more advantageous in terms of computational load and manufacturing costs than a case in which a dedicated configuration is provided for creating notification information for the start of recording of event video to the auxiliary storage device 302 for the user terminal 102.

[0064] Furthermore, when index information is used as event notification information, for example, the photographing device control module 314 of the user terminal 102 can call up from the photographing device 101 the photographing device image information recorded on the SD card 406 of the photographing device 101, which has index information corresponding to the event notification information sent to the user terminal 102.

[0065] In the first flow 60, the index information is added to the captured image information after it is determined that the event recording conditions are met (S670), but it may also be added when the image is stored in the photographing device in S610. It is also possible to configure the system so that index information is not used.

[0066] Based on the comparison result, the photographing device control module 314 can instruct the photographing device 101 to search for and extract photographing device image information (second image information) recorded on the SD card 406 or the like of the photographing device 101 that corresponds to the event video recorded in the auxiliary storage device 302 of the user terminal 102, and if there is a plurality of corresponding photographing device image information, to list them, etc. After that, the photographing device control module 314 can instruct the photographing device 101 to transmit the searched photographing device image information to the user terminal 102. Furthermore, if image information photographed by the photographing device 101 is stored on the cloud, it can also be searched for and displayed based on the index information.

[0067] The photographing device image information is, for example, image information including an event of an event video recorded in the auxiliary storage device 302 of the user terminal 102, and is image information of a longer duration or higher image quality, or of a longer duration and higher image quality. The photographing device image information may also include audio information, which is particularly preferable when the image information recording module 310 stores image information without audio information in the auxiliary storage device 302.

[0068] In particular, the photographing device image information may be, for example, image information recorded from a period before the state of the vehicle 10 satisfied the event recording condition, i.e., image information from before the event recorded in the auxiliary storage device 302 of the user terminal 102. The photographing device image information may also be image information captured during a period including before and after the state of the vehicle 10 satisfied the event recording condition.

[0069] The event notification information may also be different data depending on the event to be recorded by the user terminal 102, the settings of the user terminal 102, and the settings of the image capturing device 101. For example, if an abnormal rear approach is an event to be recorded by the user terminal 102, the control unit 403 can create the event notification information based on data regarding the distance to the vehicle behind, for example. Furthermore, for example, if an abnormal forward approach is an event to be recorded by the user terminal 102, the control unit 403 can create event notification information based on data relating to the distance to the vehicle ahead, for example.

[0070] The control unit 403 can notify the user terminal 102 of the occurrence of an event such as rearward collision or forward collision by transmitting event notification information relating to abnormal rearward collision or forward collision. The control unit 403 can also select, for each event, whether or not to transmit event notification information to the user terminal 102. The event notification information does not necessarily have to be common to all events.

[0071] If the control unit 403 does not receive confirmation information of receipt of the event notification information from the user terminal 102 after sending the event notification information (No in S690), the control unit 403 returns the first flow 60 to S680, and the control unit 403 sends the event notification information to the user terminal 102 again. If the control unit 403 receives the receipt confirmation information from the user terminal 102 in S690 (Yes in S690), the control unit 403 determines whether or not to end the first flow 60 (S695).

[0072] If the control unit 403 does not end the first flow 60 (No in S695), the control unit 403 returns to S630 of the first flow 60. If the control unit 403 ends the first flow 60 (Yes in S695), the control unit 403 ends the first flow 60. Examples of cases in which the control unit 403 ends the first flow 60 (Yes in S695) include when the photographing device cooperation module 313 releases the connection with the user terminal 102 or when the photographing device control module 314 outputs an end command for the first flow 60.

[0073] In the example of FIG. 6, the control unit 403 continues to execute image recording until the end of the first flow 60 (S610), regardless of whether an event occurs or whether notification information is sent to the user terminal 102 or not. In particular, steps S630-S660 enable the control unit 403 to determine whether at least one of an event recording condition based on an impact occurring to the vehicle 10 and an event recording condition based on the approach of another vehicle to the host vehicle 10 is satisfied. Then, the control unit 403 can transmit an event recording notification to the user terminal 102, which triggers recording of the event video in the auxiliary storage device 302 for the user terminal 102, without requiring any operation by the user.

[0074] According to this embodiment, for example, while a driver is driving his own vehicle 10, a sensor (VOICE: rear abnormal approach warning) detects aggressive driving by a following vehicle, causing the driver or passengers to become anxious or confused, saying, "Wow! They're tailgating me!" The rear image after the detection is automatically sent to a smartphone and saved on the smartphone (the drive recorder itself also records images from the front and rear), giving the driver and passengers the peace of mind that "it's all properly recorded, so it will serve as a backup just in case."

[0075] FIG. 7 shows an example of a second flow 70 of image recording by the photographing device 101, output of event notification information, and addition of index information. The steps S700, S720 to S750, S756, and S760 to S795 of the second flow 70 correspond to the steps S600, S620 to S650, S670, and S660 to S695 of the first flow 60 described above, and therefore will not be described again. Unlike the example of Figure 6, in the example of Figure 7, the control unit 403 acquires image information from the camera 402 (S600) and transmits the acquired image information to the user terminal 102 (S720) without storing the acquired image information in the memory unit of the photographing device 101 until the photographing device recording conditions are met.

[0076] Thereafter, similar to the example of Figure 6, the control unit 403 measures driving information such as the speed and acceleration of the vehicle 10 (S730), stores this speed and acceleration information (S740), and performs vehicle determination processing (S750) to determine whether the vehicle 10 satisfies the photographing equipment recording conditions and event recording conditions.

[0077] After the vehicle determination process (S750), if the vehicle 10 satisfies the photographing device recording conditions (first recording conditions) (Yes in S752), the control unit 403 records the image information acquired from the camera 402 in the memory unit of the photographing device 101 (S754), and further adds index information to the image information (S756), and proceeds to step S760. If the vehicle 10 does not satisfy the photographing device recording conditions (No in S752), the control unit 403 proceeds directly to step S760.

[0078] Thereafter, the control unit 403 executes steps S760, S780, S790, and S795 in the same manner as the first flow 60 of FIG. 6, and if the vehicle 10 satisfies the event recording condition (second recording condition) (Yes in S760), it sends event notification information to the user terminal 102 to ensure that multiple image information, for example, from after the point in time when the event recording condition was satisfied, is stored in the auxiliary memory device 302 of the user terminal 102 (S780, S790).

[0079] In another embodiment, for example, when an approaching vehicle from behind the vehicle 10 is set as an event recording condition, the control unit 403 can issue an alarm using a sound effect such as a "beep" or voice such as "Please watch your rear" or "Dangerous driving behind has been detected," as will be described later using Figure 27, and can also transmit information for storing multiple image information in the auxiliary storage device 302 of the user terminal 102.

[0080] In the example of Fig. 7, when the vehicle 10 satisfies the photographing device recording conditions, the control unit 403 records image information in the memory unit of the photographing device 101, so the photographing device 101 can use the capacity of the memory unit more efficiently than in the example of Fig. 6. In addition, as a result, for example, the image information recorded in the memory unit of the photographing device 101 can be recorded with higher image quality than the image information recorded in the memory unit of the photographing device 101 in the example of Fig. 6.

[0081] 7, the control unit 403 can separately set the photographic device recording conditions for recording image information in the memory unit of the photographic device 101 and the event recording conditions for recording image information in the auxiliary storage device 302 of the user terminal 102. Preferably, each setting can be changed depending on the content of each event.

[0082] As a result, for example, the control unit 403 can record image information related to various types of dangerous driving, such as "abnormally close from behind," "abnormally close from ahead," "dangerously cutting in," "sudden approach," "dangerous cutting in" by other vehicles from the left and right lanes, "excessive honking" or "abnormal passing / high beams" by other vehicles traveling around the vehicle 10, and "obstruction of overtaking" by a vehicle in front, in the memory unit of the imaging device 101, and the user terminal 102 can record only image information related to, for example, "abnormally close from behind" from among these types of dangerous driving.

[0083] In yet another embodiment, the control unit 403 records the image of the photographing device after acquiring the image of the photographing device (S600, S610) as in the example of Figure 6, and further, when an event occurs that corresponds to the photographing device recording condition of S752 in the example of Figure 7, it is possible to change the recording image quality of the image information according to the priority of each event, and record image information of higher image quality than the image information recorded in S610.

[0084] 6 and 7, the following operations are possible: The photographing device 101 performs event recording, and transmits trigger information for executing the event recording, causing the user terminal 102 to also execute event recording. If the photographing device 101 detects the occurrence of an event while it is continuously transmitting video information to the user terminal 102, it transmits information to the user terminal 102 as a trigger for executing event recording. In this embodiment, transmitting video information sequentially means transmitting frames of image information captured by the photographing device 101 in chronological order. The transmitted image information is displayed in motion on a preview screen.

[0085] For each event type, it is possible to set whether or not to send trigger information to the user terminal 102 to cause the event to be recorded. Trigger information is sent only for the set event type, and not for others. Depending on the event type, information such as video may be recorded only in the image capturing device 101, or may be recorded in both the image capturing device 101 and the user terminal 102. In this case, it is more desirable to send trigger information for tailgating from behind. Since the user terminal 102 can also be made to issue a warning, the transmission of trigger information for recording an event and the transmission of trigger information for outputting a warning can be switched. Note that in this embodiment, the term "warning" may be used to mean a notification with a lower level than an alarm.

[0086] The transmission of image information from the photographing device 101 to the user terminal 102 can be switched between a setting that allows continuous transmission when connected via Wi-Fi or the like, and a setting that starts transmission when the occurrence of a specific event is detected. For example, the photographing device 101 transmits image information only in the case of rear-end tilt, and the user terminal 102 records the event of the transmitted image information (and may also display it).

[0087] FIG. 8 shows an example of a flow 80 of recording image information of the photographing device 101 by the user terminal 102. 8, the photographing device cooperation module 313 of the user terminal 102 searches for and connects to a corresponding photographing device 101 (S800). The connection with the corresponding photographing device 101 may be executed when the corresponding photographing device 101 is turned on, or when the image information recording module 310 of the user terminal 102 receives an input of a command to connect with the photographing device 101. For more information on this point, please refer to the detailed explanation already given in relation to FIG. 6.

[0088] In this embodiment, the description will be given assuming that at least one of the photographing devices 101 is, for example, a drive recorder, but the photographing devices 101 are not limited to drive recorders, and the photographing device cooperation module 313 of the user terminal 102 can also establish connections with three or more photographing devices 101. For example, the photographing device cooperation module 313 of the user terminal 102 can also establish connections with photographing devices provided at positions that can determine whether the vehicle 10 is being pulled over from the side.

[0089] If the connection between the user terminal 102 and the photographing device 101 is not established (NO in S805), the photographing device cooperation module 313 of the user terminal 102 repeats S800. When a connection between the user terminal 102 and the photographing device 101 is established (Yes in S805), the user terminal 102 receives image information from the photographing device 101 using the communication control unit 307 and the display control module 312 (S810). At this point, the user terminal 102 has not yet stored the image information from the photographing device 101. Note that, in another embodiment, S810 may be omitted. When S810 is omitted, the image information from the photographing device is received, for example, after S820, which will be described later.

[0090] The display control module 312 of the user terminal 102 displays the image information from the photographing device 101 on the output device 305, such as a display of the user terminal 102 (S815). The output device 305, such as a display of the user terminal 102, may hereinafter be simply referred to as the "screen." In another embodiment, the display control module 312 may omit the execution of S815, or may selectively execute it for each event.

[0091] The photographing device cooperation module 313 of the user terminal 102 receives the event notification information from the photographing device 101 via the communication control unit 307 (S820). Note that in another embodiment, S820 may be omitted. When S820 is omitted, for example, an image analysis module (not shown) of the user terminal 102 may analyze the image received from the photographing device, and a vehicle determination module (not shown) of the user terminal 102 may determine the state of the vehicle 10.

[0092] When an event occurs in the vehicle 10 and the state of the vehicle 10 satisfies the event recording condition (Yes in S825), the user terminal 102 checks whether or not a warning setting is made for the event in question (S830). In another embodiment, the reception of the event notification information in S820 may be the condition for the state of the host vehicle to satisfy the event recording condition in S825 (Yes in S825).

[0093] The event is related to an event determined in the vehicle determination process S650 of the first flow 60, which has already been described using Fig. 6. This event may be, for example, various types of dangerous driving, such as "abnormal rear approach," "abnormal front approach," "dangerous narrowing," "sudden approach," "dangerous cutting in" by other vehicles from the left or right lane, "excessive honking" or "abnormal passing or high beams" by other vehicles traveling around the vehicle 10, or "obstruction of overtaking" by a vehicle ahead. The control unit 403 of the image capturing device 101 can detect rearward tailgating, forward tailgating, side tailgating, sudden acceleration by a vehicle behind, or sudden braking by a vehicle ahead, by determining "abnormal rearward approach," "abnormal front approach," "dangerous narrowing," and "sudden approach." Furthermore, the vehicle determination process S650 can also determine whether a tire has blown out or whether a wheel has fallen off or come off the road.

[0094] The image information recording module 310 of the user terminal 102 may determine that the event recording condition is satisfied by receiving event notification information from the photographing device 101. In another embodiment, the image information recording module 310 may determine whether the state of the vehicle 10 satisfies the event recording condition by performing a process similar to the vehicle determination process S650 in the first flow 60 of FIG. 6 on the image information received from the photographing device 101.

[0095] If a setting to issue a warning for the event that has occurred is present (Yes in S805), the warning module 311 of the user terminal 102 issues a warning according to each event. For example, when event notification information about an approaching vehicle behind is received, the user terminal 102 may output a voice warning such as "Watch your rear" from the output device 305 such as a speaker (S835).

[0096] After the warning is output or if there is no setting to issue a warning for the event that has occurred (No in S805), the image information recording module 310 of the user terminal 102 checks the recording setting for the event (S840).

[0097] If there is a setting to record image information for the event that has occurred (Yes in S840), the user terminal 102 uses the image information recording module 310 to store the image information of the event that has occurred in the auxiliary storage device 302 for the set length of time (the length of time of the multiple image information to be stored) (S845, S850).

[0098] After recording the image information or if there is no setting to record image information for the event that has occurred (No in S840), the image information recording module 310 of the user terminal 102 checks (S860) whether to end the image information recording flow 80. If the image information recording flow 80 is to be continued (No in S860), the image information recording module 310 of the user terminal 102 executes S815. Examples of cases in which the image information recording flow 80 is terminated (Yes in S860) or the image information recording module 310 terminates the image information recording flow 80 (Yes in S860) include when the photographing device linkage module 313 disconnects from the user terminal 102 or when the image information recording module 310 receives an instruction to terminate the image information recording flow 80.

[0099] As a result of the above steps, in the example of Fig. 8, the photographing device 101 transmits image information to a user terminal 102 such as a smartphone, and the user terminal 102 can store the image information transmitted from the photographing device 101 in the auxiliary storage device 302. In particular, it becomes possible to automatically transfer and store images to a smartphone in relation to aggressive driving, strong impacts, sudden braking, abrupt steering, etc.

[0100] In particular, the user terminal 102 can store video of an event that has occurred immediately after the event actually occurs. Such post-event video recording is particularly important for reckless driving events such as tailgating. According to this embodiment, the user terminal 102 and the photographing device 101 can provide the user with important information after such an event occurs, without the user having to operate the user terminal 102 by himself. Furthermore, since the image information is stored in the user terminal 102, the user is not forced to take the trouble of removing the SD card from the photographing device 101, inserting it into a computer, and displaying it on a viewer, and the user can easily check the video captured by the photographing device 101 using only the user terminal 102.

[0101] By S815, the user terminal 102 can provide the user with images of the area around the vehicle 10 captured by the photographing device 101 on the user terminal 102, and by S845, can record image information of the image displayed when the state of the vehicle 10 satisfies the event recording conditions for each event in a memory unit for the user terminal 102. Furthermore, the user terminal 102 can also accept a setting for each event as to whether or not image information is to be recorded in the storage unit for the user terminal 102.

[0102] If the vehicle 10 satisfies the event recording conditions (S825) based on the event notification information (S820) received from the photographing device 101, the image information recording module 310 of the user terminal 102 can record image information of the state in which the vehicle 10 satisfies the event recording conditions in the auxiliary storage device 302 for the terminal (S845).

[0103] Furthermore, the photographing device 101 can record image information corresponding to the event video stored in the user terminal 102, which is saved for a longer period of time and with higher image quality. For example, if one of the multiple photographing devices 101 is a drive recorder, it can record multiple pieces of image information from the start to the end of a drive.

[0104] As will be described later, the photographing device control module 314 of the user terminal 102 can also instruct the photographing device 101 to transmit image information stored in the photographing device 101 corresponding to the event video in the user terminal 102.

[0105] FIG. 9 shows an example of a setting flow 90 for the photographing device 101 by the user terminal 102. In the example of Figure 9, the photographing device collaboration module 313 of the user terminal 102 accepts setting input by the user using the input device 304 and output device 305 of the user terminal 102 for the photographing device 101 connected to the user terminal 102 (S900).

[0106] The photographing device control module 314 of the user terminal 102 transmits the setting input received in S900 to the photographing device 101 via the communication control unit 307 and the network 104 (S910). The photographing device control module 314 of the user terminal 102 receives a confirmation notification regarding the setting input from the photographing device 101 (S920), and ends the setting flow 90.

[0107] Examples of settings of the photographing device 101 by the user terminal 102 will be described in more detail with reference to FIGS. 9, the photographing device control module 314 of the user terminal 102 can perform various settings for individual events on the photographing device 101. This allows the user terminal 102 to provide the user with the possibility of easily changing the settings of the photographing device 101 according to various driving conditions and surrounding conditions.

[0108] FIG. 10 shows an example of a flow 100 for stopping the display of image information by the user terminal 102. In the example of Figure 10, the display control module 312 of the user terminal 102 displays the image information received from the photographing device 101 in the display area of ​​the screen (hereinafter sometimes simply referred to as the "screen"), which is the output device 305 of the user terminal 102, and displays operation elements in the operation area of ​​the screen of the user terminal 102 (S1000). Examples of the display area and the operation area will be described in more detail with reference to FIG.

[0109] If the first set time has not elapsed (No in S1010), the display control module 312 continues to display the image information on the output device 305. If the no-operation time has elapsed for the first set time (Yes in S1010), the display control module 312 stops displaying the display area while continuing to display the operation area (S1020). For example, the display control module 312 may determine whether the first set time has elapsed since the start of displaying the screen, or may determine whether the no-operation time, which is the time during which no operation is performed on the user terminal 102 (input device 304), has elapsed for the first set time.

[0110] If the second set time has not elapsed (No in S1030), the display control module 312 continues to display only the operation area on the output device 305. If the second set time has elapsed (Yes in S1030), the display control module 312 stops displaying the operation area and the display area (S1040). For example, the display control module 312 may determine whether the second set time has elapsed since the start of displaying the screen, or may determine whether the second set time has elapsed since the user terminal 102 (input device 304) was not operated.

[0111] The display stop flow 100 shown in FIG. 10 will be described in more detail in conjunction with FIGS. According to this embodiment, the user terminal 102 can record events that are difficult for the driver to directly see without using a mirror or monitor, particularly rear-end collisions, without requiring any active operation or action by the user.

[0112] Furthermore, when the vehicle 10 is subjected to dangerous driving such as tailgating, the driver is forced to pay the utmost attention to ensuring the safe running of the vehicle, and therefore it is extremely difficult to record the corresponding event information in the memory unit for the user terminal 102 in order to back up image information related to the dangerous driving or to ensure evidence that can be submitted immediately. Even in such a case, the driver is not legally permitted to operate the user terminal 102 while driving.

[0113] However, according to this embodiment, even when the vehicle is exposed to such dangerous driving, the user terminal 102 can reliably record the corresponding event video without requiring any active operation or action by the driver.

[0114] FIG. 11 is an example of a screen showing some of the operations for setting reference lines 1101 and 1102 for event determination. FIG. 12 is an example of a screen showing a part of the operation for setting reference lines 1201 and 1202 for event determination, following the example of the operation shown in FIG.

[0115] The display control module 312 can accept the user's selection of "Front Reference Position Setting" or "Rear Reference Position Setting" in the "Active Safety Setting" menu (see FIG. 26). In the "front reference position setting" and "rear reference position setting", the photographing device cooperation module 313 can accept input for setting reference lines 1101, 1102, 1201, and 1202 for event determination by the photographing device 101. Then, the photographing device control module 314 sets the accepted setting input in the photographing device 101.

[0116] The procedure for "setting the front reference position" will be described below using the examples of FIGS. 11 and 12, but the imaging device control module 314 can also perform "setting the rear reference position" using a similar procedure.

[0117] When the display control module 312 accepts the user's selection of "Front Reference Position Setting" in the "Active Safety Settings" menu 1100, it displays the image information received from the photographing device 101 connected by the photographing device linkage module 313 in the display area 1110 on the output device 305 of the user terminal 102. In the examples of Figures 11 and 12, the display control module 312 displays, in addition to the image transmitted from the photographing device 101, red lines 1101 and 1201 on the screen, which are reference lines for event determination, and white lines 1102 and 1202 (two dashed lines in a V shape), which are also reference lines, on the screen, within the display area 1110.

[0118] The display control module 312 further displays buttons 1104 to 1109, a decision button, and a cancel button, which are operation elements, in an operation area 1120 on the output device 305 of the user terminal .

[0119] In response to user input via buttons 1104 to 1109, the photographing equipment control module 314 of the user terminal 102 can move two V-shaped white lines (reference lines) 1102 and one red line (reference line) 1101 extending horizontally, which are displayed superimposed on the image from the photographing equipment 101 in the display area 1110, up and down and left and right, and can also narrow or widen the width of the two white lines (reference lines) 1102. For proper event determination, it is preferable that the two white lines (reference lines) 1102 overlap with the white line 1103 on the road, and it is also preferable that the red line (reference line) 1101 overlap with the horizon (horizontal line).

[0120] When the photographing device control module 314 receives an input to the "Confirm" button in the operation area 1120, it confirms the input setting of the reference line to the photographing device 101 and ends the input setting. When the photographing device control module 314 has finished inputting and setting the reference line into the photographing device 101, the display control module 312 displays the "Active Safety Settings" menu 1100 again (see FIG. 26).

[0121] For example, if the photographing equipment 101 is a drive recorder with an LCD (an example of a display unit), the photographing equipment control module 314 of the user terminal 102 can adjust the image displayed on the LCD and the reference line for event determination in accordance with the user input results using buttons 1104 to 1106 displayed in the operation area 1120 of the user terminal 102, in a manner similar to the procedures illustrated in Figures 11 and 12.

[0122] FIG. 13 is an example showing properly set reference lines 1301, 1302, and D along with imaginary decision lines A, B, and C. The example in Figure 13 shows the actual image after the reference line has been set for the image transmitted from the rear image capturing device 101R, and below that, illustrations of "good examples" and "bad examples" of the reference line settings are shown.

[0123] After the reference line is appropriately set by the user terminal 102, the photographing device 101 uses the set reference line and virtual judgment lines A, B, and C set between them to determine dangerous driving, such as tailgating from behind. Note that the virtual judgment lines A, B, and C are not actually displayed on the display of the user terminal 102, and are not included in the recorded image information. However, these virtual judgment lines can also be displayed or set depending on the design and settings. Moreover, the line D shown in the example of FIG. 13 is a reference line D corresponding to the red lines (reference lines) 1101 and 1102 in the screen shown in FIGS. 11 and 12, and is preferably adjusted so as to overlap with the horizontal line.

[0124] If the rear vehicle is present in the area defined by the two reference lines 1301, 1302, the reference line D, and the judgment line C, the photographing device 101 determines that an appropriate distance is maintained between the vehicle 10 and the rear vehicle.

[0125] The control unit 403 of the photographing device 101 can use the first and second judgment times in addition to the reference line to determine dangerous driving. The control unit 403 of the photographing device 101 can set the first and second judgment times to fixed values, for example, 5 seconds. FIG. 27 (described later) shows an example in which both the first and second judgment times are set to 5 seconds. For example, the control unit 403 of the photographing device 101 can further strengthen measures against aggressive driving by using two-stage discrimination using a first determination time and a second determination time.

[0126] In another embodiment, the first and second determination times may be variable values, which may be changed using a lookup table or the like depending on, for example, the speed of the host vehicle 10, the speed difference between the host vehicle 10 and the vehicle behind, the distance or positional relationship between the host vehicle 10 and the vehicle behind, the driving environment such as whether the vehicle is on an ordinary road or an expressway, and the like.

[0127] For example, if a rear vehicle exists in a first determination area defined by the reference line and the determination lines B and C for a first determination time, the image capturing device 101 determines that a rear approaching vehicle exists. The control unit 403 can also change the distance between the judgment lines B and C, the area of ​​the first judgment region, and the first judgment time, for example, using a reference table, depending on the speed of the vehicle 10, the speed difference between the vehicle 10 and the vehicle behind, the distance and positional relationship between the vehicle 10 and the vehicle behind, the driving environment such as whether the vehicle is on a public road or a highway, etc.

[0128] Similarly, if a rear vehicle is present in a second judgment area defined by the reference line and judgment lines A and B for a predetermined second judgment time, the photographing equipment 101 determines that a vehicle approaching abnormally from behind is present. The control unit 403 of the photographing equipment 101 can also change the distance between the judgment lines A and B, the area of ​​the second judgment area, and the second judgment time, for example, using a reference table, depending on the speed of the vehicle 10, the speed difference between the vehicle 10 and the vehicle behind, the distance and positional relationship between the vehicle 10 and the vehicle behind, the driving environment such as whether the vehicle is on a public road or a highway, etc.

[0129] As an example of determining the approach of a rear vehicle using the first judgment area and the second judgment area, when the vehicle 10 is traveling at a speed between 50 km / h and 80 km / h, if a rear vehicle is detected within the second judgment area for 5 seconds, a rear approach warning is issued, and if the vehicle is detected for another 5 seconds, i.e., a total of 10 seconds, an abnormal rear approach warning is issued and an event is recorded; also, when the vehicle 10 is traveling at a speed of 81 km / h or more, if a rear vehicle is detected within the first judgment area and the second judgment area for 5 seconds, a rear approach warning is issued, and if the vehicle is detected for another 5 seconds, i.e., a total of 10 seconds, an abnormal rear approach warning is issued and an event is recorded.

[0130] In this way, it is possible to issue a warning that a vehicle is present in the area behind the vehicle depending on the speed and approach distance, and by using the event recording settings described below, image information related to rear approach warnings and abnormal rear approach warnings can be stored in the memory unit for the user terminal 102. The control unit 403 of the front photographing device 101F performs image processing and judgment on the video information received from the rear photographing device 101R to determine the proximity of a vehicle approaching from behind. However, if the rear photographing device 101R is configured to have the control unit 403, the rear photographing device 101R may also perform the proximity determination of a vehicle approaching from behind.

[0131] FIG. 14 shows an example of switching between display layouts 1420 and 1430 according to the orientation of the screen of the user terminal 102. The example of Figure 14 shows switching between the display layout of display area 1401 and operation area 1402 when the screen of user terminal 102 is in portrait orientation and the display layout of multiple operation elements (operation parts) 1403 to 1410 of display area 1401 and operation area when the screen of user terminal 102 is in landscape orientation.

[0132] The display control module 312 of the user terminal 102 displays a display area 1401 and an operation area 1402 (1403 to 1410) on the output device 305 such as a screen (display) of the user terminal 102. The operation area 1402 is an area made up of operation elements 1403 to 1410 for inputting input operations for the image information recording module 310 and settings for the photographing device 101, for example. The display area 1401 is an area for displaying, for example, image information transmitted from the photographing device 101. Note that various examples of operation areas and display areas are also shown in other drawings.

[0133] Depending on the orientation of the user terminal 102, the display control module 312 of the user terminal 102 switches between arranging the display area 1401 and the operation area 1402 of the image information recording module 310 in separate areas on the screen of the user terminal 102, or arranging the operation area superimposed on the display area. More specifically, when the screen of the user terminal 102 is in portrait orientation, the display control module 312 displays the display area 1401 and the operation area 1402 in a display layout in which they are arranged in separate areas within the screen, and when the screen of the user terminal 102 is in landscape orientation, the display control module 312 displays the operation area 1402 in a display layout in which it is arranged on top of the display area 1401.

[0134] Furthermore, when the operation area is made up of multiple operation elements (operation parts) 1403 to 1410, the display control module 312 can also arrange each of the multiple operation elements 1403 to 1410 on the display area 1401, spaced apart from each other.

[0135] In particular, when an image is displayed when an event occurs, the parts that are important or of greatest interest to the user are often located in the center. Therefore, when the screen is in landscape orientation, the display control module 312 can display the individual operation elements 1403 to 1410 by arranging them along the four sides of the screen, as in the example of Figure 14, thereby maximizing the visible area and maximizing the visibility of the important or highly interesting parts.

[0136] FIG. 15 shows an example of a preview screen 1500 of the user terminal 102. 15 shows a preview screen of the image information recording module 310 of the user terminal 102 when the output device 305, such as a display, is in portrait orientation. In this preview screen, the display control module 312 can change the display layout of the display area 1510 and operation areas 1520 and 1530 according to the orientation of the screen of the user terminal 102, just like in FIG.

[0137] The display control module 312 can call up and display a setting screen or the like of the image information recording module 310 from the preview according to the example of FIG. Furthermore, when the user terminal 102 is connected to a plurality of image capturing devices 101, the display control module 312 can accept settings regarding which image capturing device 101 should display the video on the preview screen.

[0138] The display control module 312 can also display images from multiple image capturing devices 101 (101F, 101R) in picture-in-picture format on the output device 305. For example, if the image capturing device 101F is a front camera and the image capturing device 101R is a rear camera, the display control module 312 can display image information from the rear camera over the entire display area and display image information from the front camera superimposed on a portion of the image display from the rear camera.

[0139] The image information displayed on the output device 305 can be switched in various ways, for example, as follows. The front photographing device 101F aggregates image information from the front photographing device 101F and the rear photographing device 101R and transmits it to the user terminal 102, and the display control module 312 of the user terminal 102 selects one of the image information captured by the front photographing device 101F and the rear photographing device 101R and transmitted from the front photographing device 101F, and switches the displayed image. The front photographing device 101F and the rear photographing device 101R each transmit image information to the user terminal 102, and the display control module 312 of the user terminal 102 selects either the image information captured by the front photographing device 101F transmitted from the front photographing device 101F or the image information captured by the rear photographing device 101R transmitted from the rear photographing device 101R, and switches the displayed image. The front photographing device 101F aggregates the image information of the front photographing device 101F and the rear photographing device 101R, and performs image switching processing. In other words, the front photographing device 101F selects one of the image information captured by both the front photographing device 101F and the rear photographing device 101R, transmits the selected image information to the user terminal 102, and the user terminal 102 displays the transmitted image information. Furthermore, the display control module 312 may use the above configurations depending on the event recording settings described below.

[0140] For example, the display control module 312 displays the camera image (front, rear, picture-in-picture) selected by the user on the preview screen, but can also forcibly switch to the rear camera image if approaching from behind is detected.

[0141] The three buttons 1501 to 1503 arranged at the bottom of FIG. 15 will be described. Data confirmation button 1501: Displays a confirmation screen for the files in the auxiliary storage device 302 of the user terminal 102. One-touch recording button 1502: Saves the video sent from the image capturing device 101 and displayed on the preview screen. Settings button 1503: Calls up a screen for making various settings. Examples of setting screens that can be called up directly or indirectly from the settings button 1503 will be described in more detail in relation to FIGS.

[0142] The four buttons 1504 to 1507 arranged at the top of FIG. 15 will be described. Screen switching button 1504: Switches between multiple image capturing devices 101. For example, the screen switches in the order of front camera → rear camera → picture-in-picture. Snapshot button 1505: Saves the image sent from the image capturing device 101 and displayed on the preview screen as a still image. Power saving mode button 1506: Decreases the brightness of the screen of the user terminal 102. In particular, darkens the screen of the user terminal 102 while keeping it displayed. Microphone on / off button 1507: Switches whether or not audio is to be included in the data stored in the photographing device 101.

[0143] The display control module 312 has a display area and an operation area on the output device 305 of the user terminal 102 . The photographing device cooperation module 313 can display images from the photographing device 101 in the display area of ​​the output device 305 of the user terminal 102. In addition, the photographing device control module 314 can accept setting inputs for the photographing device 101 from the user via multiple operation elements displayed in the operation area. Note that the input from the user using the multiple operation elements displayed in the operation area may be a setting input related to the user terminal 102 only.

[0144] FIG. 16 shows an example of an operation for displaying setting screens 1601 and 1602 for the image capturing device 101 from the preview screen. 16, the user terminal 102 can provide the user with the possibility of setting the photographic device 101 in a simple and intuitive manner. In particular, the user can be provided with the possibility of setting the photographic device 101 via the setting screens 1601 and 1602, provided that the setting is limited to the settings for the various modules of the user terminal 102.

[0145] FIG. 17 shows an example of setting menus 1702 to 1704 that are displayed when the "Smartphone Event Recording Settings" menu shown in FIG. 16 is selected. In the example of FIG. 17, when "Smartphone event record settings" is selected from the "System settings" menu 1701, the display control module 312 displays a setting menu 1702 for "Smartphone event record selection" and "Smartphone event record settings." The "Smartphone Event Recording" menu provides options for events to be recorded on the user terminal 102. In the "smartphone event recording settings" setting menu 1703, the image information recording module 310 accepts the selection input of the event to be saved. In the "smartphone event recording settings" setting menu 1704, "rear-end abnormal approach recording" is set to ON.

[0146] The communication control unit 307 of the user terminal 102 can receive image information from the photographing device 101, with which a connection has been established by the photographing device cooperation module 313, via the network 104, before or immediately after an event occurs. The image information recording module 310 records the image information of the selected event based on the "smartphone event recording selection."

[0147] In the example of FIG. 17, the options in the "Smartphone Event Record Selection" menu are exemplified as "G-sensor Recording," "One-touch Recording," and "Rear-end Abnormal Approach Recording." An example of an event related to the "G sensor" is an event in which the G sensor detects an impact of a certain level or more, such as a collision of the vehicle 10. An example of an event related to "one-touch recording" is an event caused by a button operation by the user. An example of an event related to the "abnormal rear approach record" is an event in which a rear vehicle is detected for 10 seconds while the vehicle is moving. More specifically, an example is an event in which a rear vehicle is present in the first or second determination area for a certain period of time, for example, 10 seconds, as already described with reference to FIG. 13.

[0148] Other events can also be added. For example, the "smartphone event recording selection" menu may include various types of dangerous driving as candidates for events to be automatically saved, such as "abnormal approach from the front," "sudden approach (sudden acceleration of a rear vehicle, sudden braking of a front vehicle)," "dangerous cutting in," "dangerous cutting in," "excessive honking," "abnormal passing / high beams," "sudden braking of a front vehicle," and "obstruction of overtaking by a front vehicle."

[0149] For example, for each event set as dangerous driving in the image capturing device 101, the image information recording module 310 can individually accept an ON / OFF setting for automatic saving. In this specification, "automatic saving" refers to saving of image information without requiring the user to operate the user terminal 102 during or after an event occurs.

[0150] FIG. 18 is an example of a screen 1800 in which an event is displayed. The example of Figure 18 shows an example of image information displayed on the screen of the user terminal 102 when the user terminal 102 activates the image information recording module 310 for recording video from the photographing device 101 and the photographing device 101 detects the occurrence of a set or selected event.

[0151] In the figure, the video screen labeled with the text "Video of being tailgated" displays, for example, a video showing the driver being tailgated by a vehicle behind, transmitted from the photographing device 101R that actually filmed the event. For example, if different image information from the other photographing device (another photographing device, front camera) 101F is displayed in the display area of ​​the preview screen, the display control module 312 switches the video displayed in the display area of ​​the preview screen to the video transmitted from the photographing device (rear camera) 101R that filmed the event. To make it clear that event recording is being performed, the user terminal 102 may display a screen such as that shown in FIG.

[0152] When an event occurs for which the recording setting is on, the display control module 312 of the user terminal 102 can display the corresponding event on the output device 305 of the user terminal 102 without requiring any operation by the user. When the user terminal 102 is in an appropriate position, for example, in the driver's line of sight while driving, the user terminal 102 can provide the driver with an opportunity to check video related to the event without requiring any operation by the driver himself.

[0153] When an event such as rear-end collision occurs, it is extremely useful and important from the perspective of driving safety for users, particularly drivers, to be able to check information about the event without having to perform any additional operations themselves.

[0154] FIG. 19 shows an example of a setting screen 1900 of the user terminal 102. In the example of FIG. 19, a setting screen 1900 displays a selection menu for the image information recording module 310 relating to "Video" and "System." The image information recording module 310 can execute “smartphone event recording settings” and “smartphone APP screen settings” for the system for recording images of the photographing device 101 .

[0155] When the display control module 312 receives a selection of "Smartphone event recording settings," the setting screen 1900 transitions to, for example, a setting screen 3100 for "Event storage selection" and "Event recording settings" in Fig. 31. When the display control module 312 receives a selection of "Smartphone APP screen settings," the setting screen transitions to, for example, a setting screen 2000 for "Smartphone APP screen settings" in Fig. 20.

[0156] The display control module 312 can present the user with the possibility of setting the recording target, recording time, video display time, and the like of the image information from the image capturing device 101 on these setting screens 2000 and 3100. The setting screens 2000 and 3100 will be described later with reference to FIGS.

[0157] FIG. 20 shows an example of a setting screen 2000 for two types of screen settings. 20, the display control module 312 of the user terminal 102 displays a "Smartphone APP Screen Settings" menu. The "Smartphone APP Screen Settings" menu includes "Camera Video Display Time" and "Power Saving Screen Transition Time."

[0158] The "camera image display time" is the length of time for which the image received from the image capturing device 101 is displayed on the preview screen of the user terminal 102. The "power-saving screen transition time" is the length of time from when the image information recording module 310 is started until the entire screen of the user terminal 102 is switched to a state with less power consumption, for example, until the entire screen is hidden. In FIG. 20, the camera image display time is set to 1 minute, and the power-saving screen transition time is set to 30 seconds.

[0159] FIG. 21 shows an example of a selection screen 2100 for camera image display time. 21, the display control module 312 can accept a user's selection of a desired camera video display time from among 30 seconds, 1 minute, 2 minutes, 5 minutes, 10 minutes, 30 minutes, and continuous display. In another embodiment, it is also possible to accept a user's direct input of the camera video display time.

[0160] This camera image display time may correspond to the first setting time of the display stop flow 100 in Figure 10. When the selected camera image display time has elapsed, the display control module 312 stops displaying the image information from the photographing device 101 on the screen of the user terminal 102. If the user selects continuous display, the image from the photographing device 101 is displayed continuously on the preview screen of the user terminal 102.

[0161] FIG. 22 shows an example of a power-saving screen transition time selection screen 2200. 22, the display control module 312 can accept a user's selection of a desired power-saving screen transition time from among 30 seconds, 1 minute, 2 minutes, 5 minutes, 10 minutes, 30 minutes, and continuous display. In another embodiment, it is also possible to accept a user's direct input of the camera image display time.

[0162] This power-saving screen transition time may correspond to the second setting time in the display stop flow 100 in FIG. 10. When the selected power-saving screen transition time has elapsed, the display control module 312 stops displaying the entire screen of the user terminal 102, or reduces the brightness of the entire screen (keeping the screen displayed but dimming it). In this case, the process of "reducing brightness" also includes the process of turning the screen completely black. If the user selects continuous display, these transitions are not performed, and the image from the photographing device 101 is continuously displayed on the preview screen of the user terminal 102.

[0163] The options shown in Figures 21 and 22 may increase or decrease in number depending on the selection result of one of the options. For example, if the user selects 5 minutes in the example of Figure 22, the options shown in the example of Figure 21 may be only 30 seconds, 1 minute, 2 minutes, and continuous display.

[0164] When the image information recording module 310 receives the settings for the “camera image display time” and the “power-saving screen transition time,” it stores the settings as setting information 320 in the auxiliary storage device 302 . The display control module can call the saved setting information regarding the "camera image display time" and "power saving screen transition time" in order to execute the display stop flow 100 of FIG.

[0165] FIG. 23 shows examples of screen displays 2301 and 2302 before and after the camera image display time has elapsed. 23, after the camera video display time has elapsed, the user terminal 102 displays a screen display 2302 of image information that consumes less power for screen display than the image information from the photographing device 101, instead of a screen display 2301 of image information from the photographing device 101. For example, after the camera video display time has elapsed, the display control module 312 stops displaying the image information from the photographing device 101 in the display area and turns the display completely black. The display control module 312 can also add text information such as "Video Display" to the image information that consumes less power, as shown in Fig. 23. The user terminal 102 can also display a notification screen that is more intuitively recognizable by using marks, symbols, images, or the like other than text.

[0166] The display control module 312 can switch from a screen display 2302 of image information with lower power consumption to a screen display 2301 of image information from the photographing device 101 in response to a trigger from the photographing device 101 or a tap on the screen by the user. 23, the display control module 312 can reduce power consumption associated with displaying an image from the photographing device 101 in the display area 2304 while still allowing the user to input using the operation area 2303. Note that the user terminal 102 can record a corresponding event when an event occurs in any of the screen states shown in FIG.

[0167] FIG. 24 shows examples of screen displays 2401 and 2402 before and after the power-saving screen transition time has elapsed. 24, after the power-saving screen transition time has elapsed, the user terminal 102 changes not only the image information from the photographing device 101 but also the screen display 2401 of the user terminal 102 to a screen display 2402 that consumes less power, for example, a screen display in which the brightness of the entire screen is reduced. Particularly preferably, the user terminal 102 turns off the display of the entire screen of the user terminal 102, thereby reducing the power consumption related to the screen display of the user terminal 102 as much as possible.

[0168] The display control module 312 can also return the entire screen display in the lower power consumption state to the preview screen of FIG. 15 or the screen immediately before transitioning to the screen in the lower power consumption state, in response to a trigger from the photographing device 101 or a tap by the user on the screen. In the example of Fig. 24, the user terminal 102 can reduce power consumption related to screen display as much as possible. Note that the user terminal 102 can record a corresponding event when an event occurs in any of the screen states of Fig. 24.

[0169] FIG. 25 shows examples of warning target event selection screens 2502 to 2504. In the example of Figure 25, the display control module 312 of the user terminal 102 displays a "Settings" menu 2501 on the output device 305 of the user terminal 102, which has the options "Video Resolution," "Settings," and "Active Safety." When the user taps to select "Active Safety," the display control module 312 transitions to an "Active Safety Settings Screen" 2502, which has the options "Front Reference Position Setting," "Rear Reference Position Setting," "Forward Vehicle Distance Warning," "Departure Delay Warning," "Lane Departure Warning," "Rear Approach Warning," "Rear Abnormal Approach Warning," and "Rear Abnormal Approach Record."

[0170] On the “Active Safety setting screen” 2502, the photographing device control module 314 of the user terminal 102 can accept settings for the safe driving support function (Active Safety) of the photographing device 101 with which a connection has been established by the photographing device cooperation module 313. The safe driving support function is a function executed by the image capturing device 101 for the purpose of assisting the driver in making decisions and reducing the burden of driving.

[0171] On the "Active Safety Settings Screen," it is possible to individually set ON / OFF the safe driving support functions of "Forward Vehicle Distance Warning," "Departure Delay Warning," "Lane Departure Warning," "Rear Approach Warning," "Rear Abnormal Approach Warning," and "Rear Abnormal Approach Record." For example, a warning for a safe driving support function that is set ON can be output by at least one of the image capturing device 101 and the user terminal 102.

[0172] In the selection screen 2503, all event warnings and rear-end abnormal approach records are set to OFF, and in the selection screen 2504, all event warnings and rear-end abnormal approach records are set to ON. When the "Rear Abnormal Approach Recording" shown in Figure 25 is set to ON, the image information recording module 310 of the user terminal 102 can record image information of, for example, rear-end tailgating in the auxiliary memory device 302 when the vehicle 10 meets the event recording conditions, in accordance with the image information recording flow 80 of Figure 8.

[0173] In addition to the "Rear Abnormal Approach Record," the "Active Safety Settings Screens" 2502-2504 may have options for recording multiple different events, such as "Front Abnormal Approach Record," "Rapid Approach Record," "Side Abnormal Approach Record," "Abnormal Cut-in Record," and "Overtaking Obstruction Record." In this case, the image information recording module 310 can also accept a priority setting for a plurality of events, regarding which event should be given priority in recording when those events occur simultaneously.

[0174] FIG. 26 shows an example of the response of the image capturing device 101 and the indicator display when rear approach is detected. 26, when the front camera device 101F detects the approach of a vehicle from behind based on the video captured by the rear camera device 101R, the camera device 101F emits a beep sound and blinks an indicator attached to the camera device 101F. The sound effect and indicator display can be set based on the recording settings of the camera device 101F or the user terminal 102, the type of event, etc., and in this embodiment, the recording lamp indicating recording is blinking quickly in blue.

[0175] For example, when the photographing device 101F recognizes that a vehicle is approaching from behind, in addition to or instead of the photographing device 101F making the above-mentioned response and displaying an indicator, the photographing device 101R behind it may make a similar response and display an indicator.

[0176] In addition, in the configuration of FIG. 26, the image of the rear photographing device 101R is transmitted to the front photographing device 101F, and the front photographing device 101F performs event determination such as proximity, but the rear photographing device 101R may be configured to perform this event determination. In addition, if the front photographing device 101F outputs a warning, for example, by voice, when a vehicle approaches from the front, and the rear photographing device 101R outputs a warning, for example, by voice, when a vehicle approaches from the rear, the driver and passengers can easily recognize that an oncoming vehicle is approaching from the direction of the warning sound.

[0177] These warnings enable the photographic device 101 to notify the user, for example by outputting sound or light from the photographic device 101, that recording of the corresponding event has begun on the user terminal 102 or the photographic device 101.

[0178] FIG. 27 shows an example of the response of the image capturing device 101 and the indicator display when rear approach and abnormal rear approach are detected. 27, if the photographing device 101 continues to detect a vehicle approaching from behind for five seconds, the photographing device 101 issues a voice warning saying, "Watch your rear." At this point, the user terminal 102 has not yet started recording the event.

[0179] If the photographing device 101 continues to detect a vehicle approaching from behind for another 5 seconds, the photographing device 101F issues a voice warning saying "Dangerous driving from behind has been detected" and changes the indicator display to a warning state. The photographing device 101 also changes the indicator display to a recording state, and the user terminal 102 starts recording the event. 26, when an event is detected, the user terminal 102 can also notify the event by sound, light, screen display, etc., in addition to the photographing device 101. For example, instead of a voice warning, the user terminal 102 can output an event sound such as a "beep."

[0180] In the example of Figure 27, the user terminal 102 starts event recording after the second warning (abnormal rear approach warning), but in another embodiment, the user terminal 102 may perform event recording both after the first warning (abnormal rear approach warning) and after the second warning (abnormal rear approach warning). In yet another embodiment, the first warning (rear approach warning) may be omitted, and a warning and event recording may be performed at the timing of the abnormal rear approach warning.

[0181] In yet another embodiment, when an event that is subject to a warning in Figures 26 and 27 is detected, the warning module 311 of the user terminal 102 may issue a notification similar to the warning issued by the photographing device 101, such as by displaying a screen using sound, light, text, etc. Furthermore, the photographing device cooperation module 313 of the user terminal 102 can set for each event and for each photographing device 101 which events these notifications should be sent for.

[0182] FIG. 28 shows an example of a selection screen 2800 for a terminal storage unit. In the example of FIG. 28, the image information recording module 310 can accept a user's selection from the options of "folder in smartphone" and "folder in microSD" shown in the "smartphone event save destination" menu.

[0183] However, the "smartphone event save destination" menu may include another auxiliary storage device 302 as an option. The other auxiliary storage device 302 may be, for example, a cloud storage device connected to the user terminal 102 via the network 104. Furthermore, in another embodiment, the image information recording module 310 may also accept the user's own addition of a destination to save events.

[0184] FIG. 29 shows an example of a selection screen 2900 for selecting the storage time of the image information received from the photographing device 101. In FIG. In the example of FIG. 29, the "smartphone event recording time" can be selected from 10 seconds, 20 seconds, 30 seconds, and 60 seconds.

[0185] In another embodiment, the user can input and set the recording time themselves. For example, if the social networking service or the like to which the user is registered sets a maximum video duration for uploaded videos, the image information recording module 310 may record the event for the preset duration. This allows the user terminal 102 to record image information that meets the requirements of the upload destination without the user having to process the image information during or after the event.

[0186] 29 may present as options suitable recording times for various SNS services, suitable recording times designated by public institutions to which evidence images are to be submitted, etc. By accepting such options from the user in advance, even if the user encounters an event such as reckless driving, the user can record the data he or she desires in the auxiliary storage device 302 of the user terminal 102 without imposing the psychological burden of editing or checking images of the reckless driving on the user.

[0187] The communication control unit 307 of the user terminal 102 can also upload image information stored in an appropriate format for the purpose of posting content to an external server 103 that provides SNS services, reporting or submitting evidence to an external server of a public institution, providing information or backing up data to an external server of a private company, etc. In either embodiment, the user terminal 102 does not require the user to process the image information after or during the event in order to upload the image information. Also, the image information recording module 310 of the user terminal 102 can accept settings on the setting screen of the image information recording module 310 regarding which events are to be sent to which external servers.

[0188] Furthermore, when uploading image information to the external server 103, the user terminal 102 may also add index information, for example, as described in Fig. 7, before uploading the image information. Particularly advantageously, the user terminal 102 may also upload the image information after performing processing such that at least a portion of the index information can be confirmed on the image information.

[0189] As a result, even if the user terminal 102 or the photographing equipment 101 is damaged and it becomes difficult to transfer data after the event, the user terminal 102 can reliably save, submit, publish, etc., image information of important events and the circumstances of the event occurrence without the user having to operate the user terminal 102 during or after the event occurs.

[0190] In another embodiment, when the user terminal 102 uploads image information to an external server, the user terminal 102 may add information stored as setting information 320 in the auxiliary storage device 302 of the user terminal 102 to the image information to be uploaded. The added setting information 320 may be, for example, information about physical characteristics such as height, weight, and blood type of the user who owns the user terminal 102 and his / her family, friends, or passengers, medical history, medications taken, allergies, or emergency contact information. This allows the user terminal 102 to add information previously stored in the user terminal 102 to the image information captured by the image capturing device 101 and upload it to an external server.

[0191] For example, if vehicle 10 falls off its wheel, image information about the fallen wheel can be sent to a server of an organization that provides road services to which the driver subscribes, along with index information consisting of GPS coordinates or the like indicating the current location of vehicle 10.

[0192] The image information recorded on the user terminal 102 is event record image information triggered by the occurrence of an event, so the user does not have to process the image information and can easily create impactful image information, which can be sent from a smartphone to an external server and posted. For example, it is possible to post video suitable for publication on a social networking service (SNS). Event record video is impactful and short in length, making it suitable for SNS.

[0193] FIG. 30 shows examples of display screens 3001 and 3002 when image information received from the photographing device 101 is being stored. During event recording, the corresponding event video is displayed on the display screen. In the example of FIG. 30, both pieces of image information received from two photographing devices 101 (for example, the front photographing device 101F is called the front camera and the rear photographing device 101R is called the rear camera) are displayed in a picture-in-picture format. As already described with reference to FIG. 15, the display of the picture-in-picture image can be switched by tapping the screen switching button, for example, from front camera image → rear camera image → picture-in-picture image in which image from the rear camera is displayed within image from the front camera → picture-in-picture image in which image from the front camera is displayed within image from the rear camera. Various combinations of switching patterns are possible depending on the number and type of photographing devices 101, and the user can also set a desired pattern.

[0194] For example, if the user terminal 102 receives image information from four image capture devices, the screen can be divided into four sections for display. Alternatively, the user terminal 102 can display only the image information related to the event. To notify the user that an event is actually being recorded, the user terminal 102 can display notification information such as "Event recording in progress" or "Steam recording" along with a countdown of the set event recording time. For example, in FIG. 30, display screen 3001 displays the text information "Event recording in progress 18," notifying the user that 18 seconds of event recording remain. Display screen 3002 displays the text information "Steam recording 28," notifying the user that 28 seconds of event recording remain.

[0195] The screen of the user terminal 102 can also display a more intuitively recognizable notification screen using marks, symbols, images, etc. other than text. In addition to displaying a notification screen on the screen, the user terminal 102 can also output corresponding voices, sound effects, etc.

[0196] FIG. 31 is an example of a setting screen 3100 for event storage selection and event recording settings. In the example of Fig. 31, the user terminal 102 displays two items on the screen: "Select Smartphone Event Record" and "Select Smartphone Event Record." The "Select Smartphone Event Record" menu is the same as that described in relation to Fig. 17, and therefore a description thereof will be omitted.

[0197] The "smartphone event recording selection" menu has options: "smartphone event recording time," "smartphone event overwrite," and "smartphone event save destination." The image information recording module 310 accepts the user's selection of these options.

[0198] In the example of FIG. 31, the image information recording module 310 accepts the selection of 30 seconds for "smartphone event recording time," ON for "smartphone event overwrite," and the smartphone itself for "smartphone event save destination." The image information recording module 310 can provide the user with very concise options via the "Smartphone Event Recording Selection" menu and the "Smartphone Event Recording Selection" menu regarding the recording settings for individual events sent from the photographing device 101 and how to record them in the auxiliary storage device 302 for the user terminal 102.

[0199] FIG. 32 shows an example of a notification screen 3200 regarding insufficient capacity of the auxiliary storage device 302. In FIG. 32, the screen of the user terminal 102 displays a notification screen having text information saying "Event cannot be recorded because the smartphone's storage capacity is insufficient," but the content of the text information can be set as appropriate.

[0200] Furthermore, the screen of the user terminal 102 can display a more intuitively recognizable notification screen using marks, symbols, images, etc. other than text. In addition to displaying a notification screen on the screen, the user terminal 102 can also output corresponding voices, sound effects, etc. By displaying such a notification screen, the user terminal 102 can notify the user of the operation status of the photographing device 101 quickly and reliably.

[0201] FIG. 33 shows an example of a notification screen 3300 that notifies the user that the photographing device 101 is accepting an operation via an operation element provided on the photographing device 101. In FIG. The operation elements are, for example, buttons, switches, slide knobs, rotary knobs, etc. provided on the photographing devices 101F and 101R, touch panel operation screens, buttons, icons, etc. displayed thereon. In Figure 33, the screen of the user terminal 102 displays a notification screen 3300 having text information (predetermined display information, information related to the operating status of the photographing equipment) that reads "Photographing equipment in operation," but the content of the text information can be set as appropriate. The text information may be information that notifies the process being executed by the photographing device 101, such as "SD card formatting in progress."

[0202] While the notification screen 3300 is displayed, the display control module 312 restricts input of operations via the operation area of ​​the user terminal 102 to the photographic device 101. In another embodiment, while the notification screen 3300 is displayed, the display control module 312 may always be able to accept some operations, such as operation elements in the operation area that do not conflict with operations on the photographic device 101, for example, operations that can only be operated on the user terminal 102. Furthermore, the display control module 312 may not accept operations for only a part of the period during which the photographic device 101 is being operated.

[0203] Furthermore, the screen of the user terminal 102 can display a more intuitively recognizable notification screen using marks, symbols, images, etc. other than text. In addition to displaying a notification screen on the screen, the user terminal 102 can also output corresponding voices, sound effects, etc. By displaying such a notification screen, the user terminal 102 can notify the user of the operation status of the photographing device 101 quickly and reliably.

[0204] FIG. 34 shows examples of two external views (A) and (B) of the image capturing device 101F. In the example of external appearance (A) in Fig. 34, the image capturing device 101F is designed as a drive recorder 3401 for capturing images of the area in front of the vehicle 10. The drive recorder 3401 is shown as seen from the front of the vehicle 10 through the windshield. The drive recorder 3401 is connected to a cable 3402 for electrically connecting to, for example, the image capturing device 101R at the rear via a connection part 3402a. This cable 3402 corresponds to the cable 11 in Fig. 1 and is connected to the image capturing device 101R at the rear on the side opposite to the drive recorder 3401.

[0205] The housing of the photographing device 101F is generally columnar (or cylindrical). An imaging lens 3404 of the photographing device 101F (drive recorder 3401) is disposed on the vehicle-facing side of the photographing device 101F. On the housing of the photographing device 101F, on the opposite side of the position where the imaging lens 3404 is disposed, LEDs for notifying various information, operation buttons, and a microphone hole are disposed. The drive recorder 3401 is further connected to a cable 3403 for connecting to a multi-battery or a power supply unit with a voltage monitoring function via a connection section 3403a, but can function without the cable 3403.

[0206] 34(B), the image capturing device 101F is also designed as a drive recorder 3411 for capturing images of the area ahead of the vehicle 10. The drive recorder 3411, cable 3412, and connection part 3412a have similar configurations to those in the image capturing device 100 shown in FIG. 34(B), and therefore, a description thereof will be omitted. As is clear from a comparison with the example of appearance (A) in FIG. 34, the appearance design of the photographing device 101 can be varied in many ways.

[0207] [Configuration of image recording device] FIG. 35 is a diagram showing another example of the appearance of the image capturing device 101, which is a drive recorder with an LCD as the display unit described above. The operation unit 3515 is one of the user interfaces for inputting various instructions to the photographing device 101. As an example of the operation unit 3515, as shown in Fig. 35, there are various operation buttons arranged on a predetermined surface of the housing 3524. Details will be described later.

[0208] The display unit 3516 is configured by, for example, a liquid crystal display (LCD). The control unit 403 controls the display surface of the display unit 3516 to display, in real time, image data captured by the imaging unit (camera) 402, or to read and display image data recorded in a storage device such as an SD card 406.

[0209] The power supply 411 receives power from an external source and supplies a predetermined power supply voltage to devices and equipment within the photographing device 101. The power supply 411 includes a DC jack 3531 for attaching a power cable for connecting to an external power supply, for example.

[0210] As shown in FIG. 35 , the external configuration of photographing device 101 includes a flat, substantially rectangular housing 3524. Housing 3524 stands upright when attached to a predetermined position on a vehicle, such as a car. In the upright position, lens 3542 of built-in photographing unit (camera) 402 is disposed on front surface 3541, which is located forward in the direction of travel of the vehicle. Photography unit (camera) 402 photographs the area ahead in the direction of travel of the vehicle. Above lens 3542 on front surface 3541 of housing 3524, through-hole 3581 for speaker 409 built into housing 3524 is provided. Furthermore, below lens 3542 on front surface 3541 of housing 3524, through-hole 3571 for a microphone built into housing 3524 is provided.

[0211] A joint rail 3523 and a camera jack 3521 are provided on an upper surface 3544 of the housing 3524. One end of a bracket, which is an example of an attachment member (not shown), is detachably attached to the joint rail 3523. The other end of the bracket serves as an attachment portion for attachment to, for example, a predetermined position above the windshield of the vehicle, the vehicle's rearview mirror, or the ceiling inside the vehicle cabin. This attachment portion is affixed and fixed to, for example, the windshield of the vehicle, which is an example of an attachment location. In this way, the photographing device 101 is placed at a predetermined position inside the vehicle cabin. Furthermore, when the photographing device 101 is placed at the predetermined position and is in the normal position as described above, the front surface 3541 faces the windshield of the vehicle, and the photographing unit (camera) 402 photographs the area ahead of the vehicle.

[0212] The camera jack 3521 is connected to one end of a camera cable (not shown). The other end of this camera cable is connected to another external imaging device. The external imaging device may, for example, be one that captures an area different from the imaging area of ​​the imaging unit (camera) 402, and in this embodiment, a cable for communication with the imaging device 102R is connected to the camera jack 3521. The camera jack 3521 may be compatible with, for example, the USB Type C standard, and may be a terminal for communication with the imaging device 101R according to the Ethernet (registered trademark) standard.

[0213] The rear surface of the housing 3524 is provided with a display surface of the display unit 3516, an operation unit 3515, and a power lamp. In the example shown in Fig. 35(b), the operation unit 3515 has a plurality of operation buttons arranged vertically in an area to the right of the display surface of the display unit 3516. In this embodiment, four operation buttons are arranged from top to bottom: first operation button 3501, second operation button 3502, third operation button 3503, and fourth operation button 3504.

[0214] The first operation button 3501 is a button for switching the image displayed on the display surface of the display unit 3516, and a button for instructing formatting of a storage device such as the SD card 406. As described above, the photographing device 101 has a function of connecting a separate external photographing device (not shown) using a camera cable connected to the camera jack 3521, and displaying image data captured by the external photographing device on the display surface of the display unit 3516. When the first operation button 3501 is pressed for a short time, the control unit 403 switches the display between images captured by the photographing unit (camera) 402 built into the photographing device 101 and images captured by the external photographing device. When the first operation button 3501 is pressed for a long time, the control unit 403 executes formatting of a storage device such as the SD card 406.

[0215] The second operation button 3502 is a button that displays a selection screen for selecting image data to be played. When the second operation button 3502 is pressed, the control unit 403 displays a selection screen for selecting and playing video data recorded on the SD card 406.

[0216] The third operation button 3503 is a button that displays a setting menu. This setting menu is hierarchical, and by appropriately operating each operation button, a menu screen of a lower layer is displayed, and various settings can be made on the desired setting screen.

[0217] Fourth operation button 3504 is a button for starting / stopping recording. For example, if fourth operation button 3504 is pressed for a short time during recording using the continuous recording function, control unit 403 detects this and pauses recording, and if fourth operation button 3504 is pressed for a short time during the pause, control unit 403 resumes recording. Furthermore, if fourth operation button 3504 is pressed for a long time, control unit 403 changes the frame rate during recording.

[0218] A DC jack 3531 and a recording medium insertion slot 3525 are provided on the right side 3543 of the housing 3524. The DC jack 3531 is used to supply power to the photographing device 101. For example, a cigarette plug cord connector can be connected to the DC jack 3531, and the cigarette plug cord can be connected to a vehicle's cigarette lighter socket to receive power. It is recommended to use an OBD adapter that can be connected to the vehicle's OBDII connector instead of the cigarette plug cord.

[0219] The OBDII connector, also known as a diagnostic connector, is connected to the vehicle's ECU and periodically outputs various vehicle information. By connecting this OBDII adapter to the OBDII connector on the vehicle body and connecting the connector attached to the end of the OBDII adapter's connection cord to the DC jack 201, power is supplied to the image capture device 101 and the control unit 403 periodically acquires various vehicle information. The recording medium insertion slot 3525 is an insertion slot for inserting an SD card, which is an example of a recording medium. A reader / writer is mounted at the back of this recording medium insertion slot 3525, allowing recording media such as the SD card 406 to be attached to and detached from the reader / writer.

[0220] A fifth operation button 3505, which is an example of the operation unit 3515, is provided on the left side surface 3545 of the housing 3524. This fifth operation button 3505 is an input button for manually instructing the start of recording, and when this fifth operation button 3505 is pressed, the control unit 403 may perform event recording, which starts recording image data captured by the imaging unit (camera) 402 etc. and sound collected by the microphone etc.

[0221] Furthermore, a program according to the present invention causes the user terminal 102 to realize the above-described functions of each module (each means) of the user terminal 102. Another program according to the present invention causes the photographing device 101 to realize the above-described functions of each module (each means) of the photographing device 101.

[0222] [Other embodiments] Furthermore, the embodiments of the present invention may be as follows: The embodiments may be combined as appropriate. (A) In one embodiment of the present invention, when both a warning from the photographing device 101 (in this embodiment, the photographing device 101F) and a warning from the user terminal 102 (more specifically, the warning module 311) are provided, a problem occurs in that if a sound (e.g., a warning voice) is emitted from both the photographing device 101 and the user terminal 102, a sound for the same event may be emitted twice with a time difference by the photographing device 101 and the user terminal 102.

[0223] Therefore, it is preferable that both the photographing device 101 and the user terminal 102 emit sounds in response to the same event, but that the sounds (i.e., the content of the notification) are different. Although event notification information is usually sent from the photographing device 101 to the user terminal 102, it may also be sent from the user terminal 102 to the photographing device 101. In this way, it is possible to reduce the number of sensors for detecting events in the photographing device 101 (for example, a G sensor such as an acceleration sensor), thereby simplifying the configuration and reducing costs.

[0224] Both the photographing device 101 and the user terminal 102 may be configured to detect an event, send event notification information to each other, and record image information based on the event notification information received from another device (here, the other device). In addition, both the photographing device 101 and the user terminal 102 each emit a sound in response to the same event, exchange event notification information with each other, and record image information, etc. In this case, it is preferable that the device that detects the event emits a sound, and the device that has not detected the event and received event notification information from another device does not emit a sound.

[0225] It is preferable to provide a function that allows the user to set which device, the photographing device 101 or the user terminal 102, will emit sound for each event. This setting function may be provided, for example, in the user terminal 102, which may notify this setting to the photographing device 101. In this case, it is preferable to provide a setting item so that when one of the devices is selected to emit sound, the other devices are automatically set to mute.

[0226] For events detected by the photographing device 101, a notification (e.g., warning) threshold value set to a higher threshold value than the detection threshold value for the occurrence of the event (e.g., acceleration) may be provided on the user terminal 102, and the user terminal 102 may not notify an event that exceeds the detection threshold value for the event occurrence but does not exceed the notification threshold value. Both threshold values ​​may be set by an application program installed on the user terminal 102, and the user terminal 102 may have a function to send threshold value information to the photographing device 101.

[0227] It is preferable to provide a function for automatically adjusting the notification language setting of the photographing device 101 to the language setting of the user terminal 102. The language setting of the user terminal 102 can be set, for example, from an application program installed in the user terminal 102. In this way, it is possible to output sounds such as warnings from each device in a language that the user can understand, without having to set a language setting for each device individually. In this way, a warning function that is highly convenient for the user can be provided.

[0228] (B) In one embodiment of the present invention, the user terminal 102 may be equipped with the following functions related to speech recognition. The photographic device 101 may not have a voice recognition function, or even if it has a voice recognition function, it may have a function to recognize a voice different from that of the user terminal 102. The photographic device 101 may also have a function to perform processing within the photographic device 101 based on the results of voice recognition performed by an application program on the user terminal 102.

[0229] Based on the results of the voice recognition, the user terminal 102 may generate a command to cause the photographing device 101 to execute a process and transmit the command to the photographing device 101. An example of the process is event recording. In this case, event recording may be executed when a predetermined phrase such as "Record it!" is recognized in an application program of the user terminal 102. In particular, the photographing device 101 or the user terminal 102 may record image information with the recognized phrase attached as a tag. For example, if the photographing device 101 or the user terminal 102 recognizes "tailgating," it may attach the tag "tailgating" to the image information. In this way, it is possible to search for image information using the tag as a search key, for example.

[0230] Recently, price competition for packet communication fees has intensified for smartphones and other devices, making constant tethering no longer a problem. If a smartphone or other device is brought into a vehicle, it can be used as a router, making it easy to utilize voice recognition functions in a cloud computing environment, for example. Therefore, voice recognition is performed on the user terminal 102, converting the voice into a string of characters, and sending the string of characters to the image capture device 101. The image capture device 101 then adds the received tag to the video containing the current or immediately preceding range, records the image information, and allows a viewer or the like to search for and display the video with the tag added.

[0231] The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0232] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations, functions, etc. may also be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in a memory, a storage device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0233] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. The above-described embodiments disclose at least the configurations described in the claims.

[0234] The above-described embodiment discloses at least the following configurations. (1) A terminal, a communication control means for receiving a plurality of pieces of image information transmitted from a photographing device disposed in the vehicle; an image information recording means for storing the plurality of pieces of image information received from the photographing device in a storage means for the terminal when the state of the vehicle satisfies a predetermined recording condition; A terminal having: With this configuration, image information captured by the imaging device installed in the vehicle when the recording conditions are met can be recorded on the terminal without the user having to operate the terminal to issue a command to record it. Therefore, the user can use the terminal to check the video when the recording conditions are met, without having to remove the recording medium such as an SD card installed in the imaging device and insert it into a PC, and display the video on a viewer installed on the PC, which is convenient for the user. Even in the case of an event such as a rear-end collision that is difficult for the driver to see directly without using a mirror or monitor, the terminal can record image information related to the event in a storage device for the terminal without the driver having to operate the terminal or the photographing equipment after or during the event. Furthermore, for example, since the driver does not need to operate the terminal himself to record image information, he or she can concentrate on driving, and can respond to dangerous driving such as tailgating in a safer state and with a more reassured or calm mind. (2) the terminal has a display means for displaying a screen and a display control means, the communication control means sequentially receives the plurality of pieces of image information from the photographing device; the display control means displays the received plurality of pieces of image information in a display area on the screen; If the recording condition is satisfied while the plurality of pieces of image information are being displayed, the terminal stores the plurality of pieces of image information being displayed in a storage means for the terminal. With this configuration, the user can check the images of the area around the vehicle captured by the imaging device on a terminal at hand, such as a smartphone, without operating the imaging device. Furthermore, the user can save the image information of the image displayed when the recording conditions were met in the storage unit for the terminal without having to check the imaging device again. For example, if a serious event such as aggressive driving or an accident occurs, the situation can be easily checked and displayed on a terminal such as a smartphone. (3) The image information recording means is a terminal that stores the plurality of pieces of image information when notification information notifying that the recording conditions have been met is received from the photographing device. By configuring the terminal in this manner, when the terminal receives notification information, the reception of the notification information is used as a trigger to record the image information received from the photographing device as image information when the recording conditions are met, so that the user can easily check it later. (4) The display control means is a terminal that displays an operation area for operating the photographing device or the terminal on the screen. With this configuration, the terminal can provide the user with instructions on how to operate not only the terminal itself but also the photographic equipment via the screen, allowing the user to operate and set up the terminal and photographic equipment as if they were an integrated system. (5) The display control means is a terminal that stops displaying the display area while continuing to display the operation area after a first set time has elapsed. With this configuration, the terminal can reduce power consumption related to displaying the display area after the first set time has elapsed. Because the main power consumption related to screen display is often due to the display of image information in the display area, the terminal can reduce main power consumption related to screen display while maintaining a state in which it can accept user operations and settings of the terminal via the currently displayed operation area. As a result, it is possible to prevent problems such as the image information not being stored in the terminal storage unit due to the terminal's battery running out. (6) The display control means is a terminal that stops displaying the display area and the operation area after a second set time has elapsed. With this configuration, the terminal can reduce power consumption related to displaying the display area and operation area after the second set time has elapsed. When the user does not look at the terminal screen for a long period of time, such as while driving, the display area as well as the operation area can be stopped, thereby enabling more effective power saving. (7) The display control means is a terminal that accepts changes to the first set time and the second set time. With this configuration, the user can set the desired power consumption savings for each occasion with simple operations, depending on the user's own driving style, driving conditions, remaining battery capacity of the terminal, and the like. (8) When the photographing device is operated using an operating element on the photographing device side, the display control means displays specified display information in the display area and restricts input of operations to the photographing device using the operating area of ​​the terminal. This configuration prevents interference between the terminal and the photographic device when inputting operations to the photographic device. The photographic device is often located in a location that is difficult for the user to reach, such as near the top edge of the windshield or rear window, where the user's line of sight is not obstructed. This allows the user to easily check the operation and processing status of the photographic device at hand, without having to directly operate the photographic device itself, which is convenient. For example, the user can easily and reliably check the status of the rear camera without having to move to check the status of the photographic device. (9) The display control means is a terminal that displays information relating to the operating state of the photographing device as the predetermined display information. With this configuration, the user can know on the terminal whether the photographing device is being operated by the operation element on the photographing device. For example, if a passenger is operating the rear camera, the driver can check whether the operation is being performed as intended while remaining seated in the driver's seat. (10) A terminal in which the display control means switches between arranging the operation area and the display area in separate areas on the screen or arranging the operation area overlapping the display area, depending on the orientation of the terminal. This configuration allows the device to easily switch between the display area and operation area depending on the screen orientation, providing better visibility and operability for the user. For example, when taking a screenshot, the user can easily capture an event image from a camera displayed across the entire device screen simply by changing the orientation of the device. This is significantly more advantageous for the user in terms of effort, image quality, safety, etc. than when using the device's camera to directly capture video of a tailgating incident or other incident captured on the LCD screen of a conventional liquid crystal display (LCD) dashcam. (11) the operation area has a plurality of operation portions, When the display control means displays the operation area superimposed on the display area, the display control means arranges each of the plurality of operation parts so as to be spaced apart from each other within the display area and superimposed on the display area. With this configuration, when the operation area is displayed overlapping the display area, the visibility of the display area can be improved. Also, by increasing the spacing between the operation parts of the operation area, the user will feel less cramped when performing individual operations, and the terminal can simultaneously provide the user with comfortable operability and good visibility. Alternatively, displaying an operation area (operation portion) superimposed on a display area can significantly improve the degree of freedom in layout design and the degree of freedom in customization by the user. (12) The terminal has a photographing device linking means, and sets the photographing device based on inputs to the display area and the operation area. By configuring the terminal in this way, the terminal can provide the user with the possibility of not only setting the terminal but also individual settings of the photographic device. For example, the photographic device may not have a high-performance user interface compared to the terminal, or the photographic device itself may not have an output device such as a display, but the user can easily make various settings of the photographic device at hand via the display of the terminal. (13) The photographing device linking means is a terminal that sets a reference line in the photographing device for outputting that the state of the vehicle satisfies a predetermined recording condition based on an input to the operation area. By configuring it in this way, even if the photographing device itself is designed without an output device such as a display, the user can easily make detailed settings for the reference line of the photographing device at hand via the display of the terminal. Compared to using the screen of a camera fixed to a vehicle, the user can set the device in a more comfortable position or in a more easily visible or visible state (e.g., with less influence from ambient light). Also, the UI of a smartphone or other device is often more comprehensive than that of a camera, which improves user operability. (14) The photographing equipment determines at least one of the distance or positional relationship to at least one of a vehicle in front of the vehicle or a vehicle behind the vehicle based on the set reference line, and notifies the terminal that the vehicle's state meets specified recording conditions. The device can not only receive notifications from the device but also easily recognize that image information of events related to such notifications has been recorded. (15) If the state of the vehicle satisfies a predetermined recording condition while another image information from another photographing device is being displayed in the display area, the display control means of the terminal switches the image information displayed in the display area to the image information from the photographing device that has detected that the state of the vehicle has satisfied the predetermined recording condition. With this configuration, the terminal can display image information that is problematic or of particular interest to the user on the screen. For example, even in the case of an event such as a rear-end collision that is difficult for the driver to visually check directly without using a mirror or monitor, the user can check image information related to the event on the terminal at hand without having to operate the terminal individually each time, which can provide the user with good driving safety and a sense of security. (16) The display control means is capable of displaying images from a camera in front of the vehicle and images from a camera in rear of the vehicle, and if the display means detects that another vehicle is approaching from behind the vehicle while displaying the image from the camera in front of the vehicle, the display means switches to the image from the camera in rear of the vehicle. With this configuration, the device can record image information about rear-end collisions in the device's storage device without the user operating the device or camera after or during the collision, even if the collision is difficult for the driver or passenger to see directly without using a mirror or monitor. While driving, the user can check the highly accurate detection of rear-end collisions on the screen display at hand, without constantly checking the behavior of a following vehicle suspected of being a rear-end collision, and can also save the image information in the device. With conventional devices, especially when there is only one driver in the vehicle, even if the driver pays attention to the rear and recognizes the collision early, it is difficult to immediately record the image information on the device at hand. With this device, the user does not need to pay more attention to the rear than necessary or manually start recording the image information. (17) the control means of the photographing device determines whether or not a recording condition based on at least an impact occurring to the vehicle is satisfied, and whether or not a recording condition based on an approach of another vehicle to the vehicle is satisfied, The image information recording means is a terminal that records the plurality of pieces of image information in a storage means for the terminal when a recording condition based on the approach of another vehicle to the vehicle is satisfied. This configuration makes it possible to selectively use recording conditions based on the impact of a collision accident or the like, and recording conditions based on proximity before a collision accident, which is a situation where an accident is likely to occur. For example, image information of events where the aftermath of the event is more important, such as road rage or dangerous driving, can be preferentially stored in the terminal storage unit. (18) The terminal has a warning means for outputting a warning that the state of the vehicle has satisfied a predetermined recording condition using at least one of sound, light, and text. With this configuration, the terminal can issue a warning to the user regarding the vehicle status determined by the image capturing device. For example, if the image capturing device is a rear camera, the user can more reliably receive warnings related to the rear camera from the terminal. Furthermore, if image information of events related to warnings about aggressive driving or dangerous driving is recorded on the terminal, the user (the driver) can be freed from focusing on securing evidence and can focus on safe driving. (19) The image information recording means is a terminal that stores the image information not including audio information in the storage means. With this configuration, the terminal can store image information with a smaller capacity, and the capacity of the storage unit for the terminal can be used efficiently. (20) the image information recording means stores the image information to which the index information has been added in the storage means; A terminal that acquires second image information corresponding to the plurality of image information stored by the photographing device based on the index information. With this configuration, the terminal can easily search for and acquire image information related to image information stored in the terminal storage unit from the image information stored in the storage unit for the photographing device based on the index information. Also, the user can easily recognize the contents of the image information recorded in the terminal from the index information, even from the list on the saved screen. (twenty one) The second image information includes audio information. With this configuration, even if a smaller volume of image information that does not include audio information is stored in the terminal storage unit, if, for example, it becomes necessary later to grasp a more detailed situation from the video, the terminal can obtain the audio information from the second image information. Because the user has both image information that includes audio information and image information that does not include audio information related to the same event, there is no need for the user to process the image information to determine whether or not audio information is included. (twenty two) The second image information is a terminal having image information for a period before the state of the vehicle satisfied a predetermined recording condition. This configuration allows the device to obtain image information from a period prior to the image information stored in the device's storage unit, allowing the user to check the behavior of the target vehicle before it is determined to be aggressive driving, and to submit it as evidence if necessary. For example, even if it is difficult to determine whether the target event was intentional or accidental based on image information obtained only after the event occurred, the user can add behavior prior to that time to the determination. This allows the user to understand in more detail the situation when the recording conditions were met. (twenty three) The image information recording means is a terminal that sets the length of time for storing the plurality of pieces of image information, and controls the time for storing the plurality of pieces of image information based on the set length of time. With this configuration, after storing the image information in the storage unit for the terminal, the terminal can provide the image information of the length desired by the user without the user having to process the length of the image information. This also allows the user to upload image information of an appropriate playback length to SNS etc. without processing it. (twenty four) The communication control means is a terminal that transfers the stored image information to an external server. With this configuration, the terminal can also save image information stored in the terminal's storage unit on an external server, so that even if the terminal or the photographing equipment is damaged after an event occurs and image information serving as a record or evidence is lost, image information of highly important scenes can be reliably saved. For example, if the external server that can be set on the terminal is operated by a public investigative agency or a private road service, it will be possible to quickly report or contact the relevant party along with the specific situation. (twenty five) A program for causing the terminal to realize the functions of each of the means described above. With this configuration, the above control can be performed by providing a program to an existing terminal. (26) A photographing device disposed in a vehicle, a transmitting means for transmitting information on the captured images to an external terminal; a detection means for detecting a state of the vehicle; a control means for transmitting information for storing the transmitted plurality of pieces of image information in the terminal when the state of the vehicle satisfies a predetermined recording condition; Photographing equipment having With this configuration, image information captured by the imaging device installed in the vehicle when the recording conditions are met can be recorded on the terminal without the user having to operate the terminal to issue a command to record it. Therefore, the user can use the terminal to check the video when the recording conditions are met, without having to remove the recording medium such as an SD card installed in the imaging device and insert it into a PC, and display the video on a viewer installed on the PC, which is convenient for the user. The photographing device can record image information related to an event on a terminal without the driver having to operate the photographing device or terminal after or during the event, even in the case of an event such as rear-end collision, which is difficult for the driver to see directly without using a mirror or monitor. This eliminates the need for the driver to operate the photographing device to record image information on the terminal, allowing the driver to concentrate on driving and respond to dangerous driving such as rear-end collision in a safer state with a more reassuring and calm mind. (27) the information for storing the plurality of pieces of image information in the terminal is notification information for notifying that the recording condition has been satisfied, the transmitting means sequentially transmits the captured image information; The control means is an imaging device that transmits the notification information when the recording condition is satisfied. By configuring in this manner, the photographing device notifies the terminal by notification information that the recording conditions have already been met, which serves as a trigger for the terminal to start storing image information, so that the terminal can store image information without the need to determine the recording conditions at the terminal. (28) The control means is a photographing device that, when the state of the vehicle satisfies the specified recording condition, stores the plurality of image information captured during a period including before and after the time when the recording condition was satisfied in a storage means for the photographing device. This configuration allows the image capture device to store longer-term image information, including the period before and after the image information stored in the terminal. This allows the user to obtain image information from the period before and after the image information stored in the terminal's memory. The user can, for example, check the behavior of the target vehicle and the words and actions of the target vehicle's user before and after the determination of aggressive driving, and can submit such information as evidence if necessary. For example, even if it is difficult to determine whether an event was intentional or accidental based on image information obtained only after the event occurred or for a short period of time before and after the event, the behavior prior to the event can be added to the information needed to make a decision.Furthermore, if the behavior of the driver of the target vehicle becomes an issue at a later date, the necessary image information can be provided to the user at that stage. (29) The control means When a first recording condition is satisfied, the plurality of pieces of image information are stored in a storage means for the photographing device; When a second recording condition is satisfied, the photographing device transmits information for causing the terminal to store the plurality of pieces of image information after the second recording condition is satisfied. With this configuration, the photographing device can record image information under different recording conditions for the photographing device and the terminal, allowing the user to set more appropriate conditions depending on the entity to be stored (the photographing device or the terminal). For example, by setting the second recording condition to a relatively extreme condition, such as a very dangerous condition, the user can prioritize leaving image information that is expected to cause a big stir on social media or the like on the terminal. (30) The photographing device issues a predetermined warning when a vehicle approaches the vehicle from behind as the second recording condition, and transmits information for storing the plurality of image information in the terminal. By configuring it in this way, the photographing device can issue a warning about the approach of a vehicle behind and also prompt the terminal to store image information related to the approach of the vehicle behind.As a result, not only can the photographing device issue a warning about a detected event, but the terminal can also record image information about detected events such as rear-end collisions based on highly accurate judgments made based on the performance of the photographing device, even if the terminal itself does not have the configuration or function to determine the distance or positional relationship to the vehicle behind. This not only enables users to quickly recognize events such as rear-end collisions, which tend to be delayed due to warnings, but also allows users to secure image information of interest on their own device without having to operate the device individually, allowing users to concentrate on safe driving with the peace of mind that image information that can serve as evidence is secured. (31) A program for causing the photographing device to realize the functions of each of the means described above. With this configuration, the above control can be performed by providing a program to existing photographing equipment. [Explanation of symbols]

[0235] 10...vehicle, 101, 101F, 101R...photography equipment, 102...user terminal, 103...external server, 104...network, 301...main memory device, 302...auxiliary memory device, 310...image information recording module, 311...warning module, 312...display control module, 313...photography equipment linkage module, 314...photography equipment control module, 320...setting information, 321...image information, 303...processor, 304...input device, 305...output device, 306...camera unit, 307...communication control unit, 401...microcomputer, 402...camera, 403...control unit, 404...photography equipment controller

Claims

1. A photographing device disposed in a vehicle, a transmitting means for transmitting information on the captured images to an external terminal; a detection means for detecting a state of the vehicle; a control means for transmitting information for storing the transmitted plurality of pieces of image information in the terminal when the state of the vehicle satisfies a predetermined recording condition; and The control means When a first recording condition is satisfied, the plurality of pieces of image information are stored in a storage means for the photographing device; When a second recording condition is satisfied, information for storing the plurality of pieces of image information after the second recording condition is satisfied is transmitted to the terminal. Filming equipment.

2. When a vehicle approaches the vehicle from behind as the second recording condition, a predetermined warning is issued and information for storing the plurality of pieces of image information in the terminal is transmitted. The imaging device according to claim 1 .

3. 3. A program for causing the photographing device to realize the functions of each of the means according to claim 1 or 2.

Images