Reproducing device, reproducing method, program thereof, recording device, recording device control method, and the like

Abstract

To provide a recording device and a method for controlling the recording device that a user can view a video which has been processed based on acquired information.SOLUTION: An automatic driving video folder in which video data taken during automatic driving is stored, and a manual driving video folder in which video data taken during manual driving is stored, are generated, and video data taken during automatic driving is stored in the automatic driving video folder, and video data taken during manual driving is stored in the manual driving video folder. Also, a picture quality of the video data to be stored will be different depending on a determination result of whether it is in the automatic driving or the manual driving.SELECTED DRAWING: Figure 19

Description

[Technical field]

[0001] The present invention relates to, for example, a playback device, a playback method, and a program therefor, as well as a recording device and a method for controlling the recording device. [Background technology]

[0002] There are known in-vehicle video recording devices that capture video of the vehicle's surroundings using a camera installed in the vehicle and record the surrounding video and the vehicle speed when the vehicle is subjected to an impact due to an event such as sudden braking, sudden steering, or a collision, and there are also known playback devices that play back the video captured by such drive recorders (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5966131 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the invention described in Patent Document 1 only plays back captured images, such as images taken before and after the occurrence of an event.

[0005] The present invention aims, for example, to increase the possibility of displaying an image more suitable for the user than ever before. The purpose of the present invention is not limited to this, and the applicant intends to obtain rights to configurations that aim to obtain effects from parts of the configuration disclosed in this specification and drawings, etc., through divisional applications, amendments, etc. For example, this specification discloses a problem in which the part described as "can be" in this specification is read as "the problem is that...". The problems are described as independent, and the applicant intends to obtain rights to the configuration for solving this problem alone through divisional applications, amendments, etc. Even if the problem is understood implicitly from the description in the specification, the applicant intends to include part of the configuration described in this specification in the claims by amendment or divisional application. [Means for solving the problem]

[0006] (1) A playback device that plays back recorded video data, processes the recorded video data based on acquired information, and performs control to display an image based on the processed video data.

[0007] In this way, the user can view not the recorded image itself, but the image processed based on the acquired information. The processing is suitable for the user viewing the image, and an image suitable for the user can be obtained. Since the recorded image is not simply processed, but the recorded image data is processed based on the acquired information, the possibility of displaying an image more suitable for the user viewing the image is increased. For example, a playback device that plays back video data recorded in a vehicle may process the recorded video data based on information acquired in the vehicle, and control the display of an image based on the processed video data. In particular, the recorded video data may be processed based on information acquired in the vehicle at the same time as the video data recorded in the vehicle, and control the display of an image based on the processed video data. The same time period may be within a predetermined time range, and the predetermined time range may be, for example, several seconds, and in particular, the same time period may be the same. The information to be acquired may be information that changes in accordance with the driving operation of the driver of the vehicle, and may be information relating to the behavior of the driver himself / herself. The information relating to the driver's own state may be information that has a relationship between the driver's own state and the image recorded in the vehicle. The relationship may be an objective relationship, but it is particularly preferable to use a subjective relationship of the driver himself. The subjective relationship may be a relationship of what the driver thinks it looked like, for example, information necessary to estimate that it "looked like this." However, it is preferable that the information to be acquired is information other than the vehicle speed.

[0008] The recorded video data may be, for example, video data recorded by a recording device, or may be, for example, video data recorded on a memory card such as an SD (Secure Digital) card, a micro SD card, or an XQD (registered trademark) card. The acquired information may be, for example, information input by a user viewing the video. By having the user input information so that processing suitable for the user viewing the video is performed, and the playback device acquires that information, the possibility of displaying video suitable for viewing by the user who is viewing the video increases. The acquired information may also be predetermined information. Information may be acquired by reading such predetermined information. The processing may be, for example, such that the video data before and after the processing is changed.

[0009] (2) The processing may include processing to display the user's field of view in the image.

[0010] In this way, the user's visual field range in the image becomes easier to understand. Since the user's visual field range is known, for example, it becomes easier to understand the parts that the user cannot see and the parts that the user can see. The user's visual field range may be fixed, may be determined according to the user's age, or may be measured and determined for each user. In particular, the processing process may be a process of displaying the user's visual field range in the image based on the acquired information. In particular, a processing process may be performed to display the user's visual field range in the image based on the range based on information for adjusting the visual field range input from the visual field range input means and the acquired information. Displaying the visual field range in the image may be displayed in a manner in which the inside and outside of the visual field range can be distinguished. For example, the visual field range may be displayed by surrounding it with a frame, or the outside of the visual field range may be displayed by filling in or blurring it. The image may be, for example, an image recorded by a recording device using a wide-angle camera, or an image recorded by a recording device using a spherical camera.

[0011] (3) If there is an object that requires attention in the image outside the user's field of view, the user may be notified.

[0012] In this way, the user can easily know that there is an object that the user should be aware of in an area that the user cannot see. Whether or not there is an object that the user should be aware of in an image outside the user's field of view can be determined by, for example, detecting an object in an image in each frame of a recorded image by image recognition or the like, and judging that an object is present in the image of the frame when the range of the detected object is within the range outside the field of view, and judging that an object is not present when the detected object is not present. In particular, the object detection process may omit at least a part of the search for the object by image recognition within the field of view in the image of the frame, and preferably does not omit this. In the image of the frame, the search for the object by image recognition may be performed only in the range outside the field of view. For example, if the image is acquired by a recording device, it is possible to know that there is an object that the user should be aware of in a place that the user cannot see, and this increases the possibility that the user will be able to move his / her head and pay attention to the area that is not visible when driving. The object to be aware of may be, for example, a pedestrian, bicycle, automobile, motorcycle, etc. The notification of this may be particularly preferably made by displaying the position in the image.

[0013] (4) The processing includes displaying the user's line of sight at the time of recording the video data on the video. It is advisable to use this processing method.

[0014] In this way, it becomes easier to understand where the user was looking when the video data was recorded. In particular, the processing process may be a processing process that displays the user's gaze on the video when the video data was recorded based on the acquired information. In particular, the user's gaze on the video may be displayed based on the user's gaze input from an input means that inputs the user's gaze on the video when the video data was recorded and the information acquired in the previous period. For example, when the video data was recorded, the user's eye movement may be recorded and the user's command may be displayed on the video based on the record. The gaze may be, for example, the direction in which the user is looking, and may be the line of sight. Displaying the gaze on the video may mean, for example, displaying the gaze as dots on the video in sequence, or displaying the trajectory of the gaze on the video. The user's gaze may be, for example, photographed by a photographing means when the video data was recorded and the gaze may be detected from the photographed image obtained by the photographing means, or the user may wear a gaze detection device such as glasses for gaze detection and the gaze may be detected by the gaze detection device such as glasses.

[0015] (5) The processing may be such that the field of view is moved in the direction in which the user is facing when the video data is recorded and displayed on the video.

[0016] In this way, the direction in which the user is facing when the video data is recorded can be known, and the field of view range in that direction can be easily understood. It is also easy to see which part of the video was visible when the video data was recorded. In particular, it is preferable to obtain information indicating the direction detected by a detection means that detects the direction in which the user is facing, and to move the field of view range based on the obtained information and display it in the video. The field of view range can be moved by processing the frames so that the range inside and outside the field of view range can be distinguished for each certain number of frames. For example, the field of view range can be moved by moving the field of view range relatively, and the field of view range can be moved relatively by moving the video.

[0017] (6) For example, the processing may be a processing for displaying in the video a range that is not viewed by the user for a certain period of time when the video data is recorded.

[0018] In this way, the range that the user does not look at during a certain period of time becomes easy to understand. In particular, it is preferable to obtain information obtained from a detection means that detects the range that the user does not look at during a certain period of time, and display the range that the user does not look at during the certain period of time from the obtained information in the image. It is preferable to detect the user's gaze, and detect the range that the user does not look at during the certain period of time from the detected information of the user's gaze and the information of the certain period of time. The user's gaze may be detected, for example, by photographing the user with a photographing means when recording the image data, and the gaze may be detected from the photographed image obtained by the photographing means, or the user may wear a gaze detection device such as glasses for gaze detection, and the gaze may be detected by the gaze detection device such as glasses. For example, if the image data is recorded by a recording device, the certain period of time may be the time during which the user is stopped at an intersection, the time during which the user turns right or left at an intersection, the time during which the vehicle changes lanes, etc. Displaying the range that the user does not look at on the image may mean, for example, filling in or blurring the range that the user does not look at.

[0019] (7) For example, the processing may include processing to display on the video whether the user's line of sight at the time the video data was recorded matches the direction in which the user should be looking.

[0020] In this way, it becomes easier to know whether the user is looking in the direction they should be looking or not. In particular, information is obtained from an input means for inputting information indicating whether the user's line of sight at the time the video data was recorded matches the direction the user should be looking, and based on the obtained information, it is determined whether the user's line of sight at the time the video data was recorded matches the direction the user should be looking. It is preferable to use a processing process in which whether the user's line of sight at the time of recording the video data and the direction in which the user should look are aligned is displayed on the video. It is also preferable to use a processing process in which information input from an input means for inputting information indicating the user's line of sight at the time of recording the video data and information input from an input means for inputting information that the user should look at are acquired, and whether the user's line of sight at the time of recording the video data and the direction in which the user should look are aligned is displayed on the video based on the acquired information. It is preferable to use a processing process in which information obtained from a detection means for detecting whether the user's line of sight at the time of recording the video data and the direction in which the user should look are aligned is acquired, and whether the user's line of sight at the time of recording the video data and the direction in which the user should look are aligned is displayed on the video based on the acquired information. For example, if the video data is recorded by a recording device, the direction in which the user should look is predetermined depending on the surrounding situation. It is also possible to calculate the degree of alignment between the user's line of sight and the direction in which the user should look.

[0021] (8) For example, when the video data is being recorded by a recording device, with regard to whether the user's line of sight at the time the video data is recorded coincides with the direction in which the user should look, the direction in which the user should look may differ depending on the operation of driving the car at the time the video data is recorded.

[0022] In this way, it becomes easy to understand that the direction the user looks at differs depending on the driving of the car or the user's driving operation when the video data is recorded. Since the direction the user looks at differs depending on the driving of the car or the user's driving operation when the video data is recorded, it becomes easy to understand whether the user looks at the direction that should be looked at depending on the driving of the car or the driving operation of the car. In particular, it is preferable to determine the direction that the user should look at based on information from an input means that inputs information about the driving of the car or the user's driving operation when the video data is recorded. The direction that the user should look at, for example, when the driving operation of the car is driving straight, should be the direction that the user should look at approximately the front, and when changing lanes, the direction that the user should look at is the room mirror, the door mirror of the driving lane on the side where the lane is to be changed, the side window on the side where the lane is to be changed, etc. In addition, when driving a car, when you turn the steering wheel relatively greatly after operating the turn signal in order to turn right or left, you should look diagonally forward and diagonally backward on the side on which you operated the steering wheel, and when you turn the steering wheel relatively slightly after operating the turn signal in order to change lanes, it is a good idea to look at the rearview mirror, the door mirror of the driving lane on the side on which you are changing lanes, and the side window on the side on which you are changing lanes.

[0023] (9) For example, the processing may involve obtaining information indicating whether the order in which the user viewed the video data matches the order in which it should be viewed when the video data was recorded, and displaying the information on the video based on the obtained information.

[0024] In this way, it is easy to know whether the order in which the user viewed the images matches the order in which they should be viewed in advance. In particular, based on information from a detection means for detecting whether the order in which the user viewed the images matches the order in which they should be viewed in advance when the image data is recorded, information indicating whether the order in which the user viewed the images matches the order in which they should be viewed in advance may be displayed on the image. The order in which the user viewed the images when the image data was recorded can be known, for example, by analyzing image data obtained by shooting a user driving a car and detecting the user's line of sight in a time series. The order in which the images should be viewed in advance can be known based on information input from an input means for inputting information indicating the order. For example, the recorded image data may be played back, and the order in which an instructor or the like should view the images represented by the played back image data may be input from the input means. The order in which the images should be viewed in advance may vary, for example, depending on the situation at the time of recording. For example, if the image data is from the time of starting a car, an example of the order in which the images should be viewed is, for example, to look forward, look at the room mirror or the rear, look at the right door mirror or fender mirror, and look at the right window. The degree of agreement between the order in which the user viewed the images and the order in which they should be viewed in advance may be calculated.

[0025] (10) For example, the processing may be processing for displaying in the image the part that the user should look at.

[0026] In this way, it becomes easier for the user to see from the image where they should look. In particular, it is a good idea to acquire information input from an input means for inputting information indicating where the user should look, and to display on the image where the user should look based on the acquired information. For example, if the image data is an image seen from the driver's perspective while driving, it becomes easier for the driver to see where they should look, which is often useful for driving instruction. For example, if the image data indicates where the driver should look at the rearview mirror, a mark may be added to the image part of the rearview mirror, or if the door mirror is where the driver should look, a mark may be added to the image part of the door mirror.

[0027] (11) For example, if the user does not see a part that he or she should see when recording video data, an image of that scene may be displayed.

[0028] In this way, it becomes easier to understand what kind of scene the user is not looking at the places he should be looking at. In particular, it is preferable to display a screen of a scene where the user is not looking at the places he should be looking at, based on information input from an input means for inputting information indicating that the user is not looking at the places he should be looking at when recording the video data. To determine whether the user is not looking at the places he should be looking at, gaze information obtained from a detection device that detects the user's gaze can be used. For example, if the video data represents a video of the user while driving, it becomes easier to understand what kind of scene the user is driving in and when he is not looking at the places he should be looking at, which is more likely to be useful in improving driving skills. Also, for example, if the video data represents a video of the user while driving, evaluation can be made based on an image of the scene where the user is not looking at the places he should be looking at.

[0029] (12) For example, the speed at which a portion of an image where there is no change in the movement of the subject may be displayed may be made faster than the speed at which a portion of the image where there is a change in the movement of the subject is displayed.

[0030] In this way, it is more likely that the entire video can be viewed quickly. In particular, to determine whether a video portion has no change in the subject's movement or a video portion has a change in the subject's movement, a detection means for detecting whether the video portion has no change in the subject's movement or a video portion has a change in the subject's movement may be used. The detection means may detect the difference in the images between frames, and if the difference is less than a certain level, it is determined that there is no change in the subject's movement, and if the difference is more than a certain level, it is determined that there is a change in the subject's movement. For example, if the video portion with no change in the subject's movement is not relatively important, and the video portion with a change in the subject's movement is relatively important, it is more likely that the relatively unimportant part can be viewed in a shorter time, and the relatively important part can be viewed at length. A video portion with no change in the subject's movement does not only mean a part with no change in the subject's movement at all, but also includes a part with almost no change in the subject's movement or a part with a change in the subject's movement below a certain level. Also, a video portion with a change in the subject's movement does not only mean a part with even a small amount of subject movement, but also a part with a change in the subject's movement above a certain level.

[0031] (13) For example, video data may be recorded by a recording device, and the user's driving may be evaluated based on video portions in which there is a change in the subject's movement.

[0032] In this way, it becomes easier for the user to evaluate the driving of the car based on the image part where there is a change in the movement of the subject. In particular, to determine whether the image part is one where there is no change in the movement of the subject or one where there is a change in the movement of the subject, a detection means for detecting whether the image part is one where there is no change in the movement of the subject or one where there is a change in the movement of the subject can be used. The detection means is It is advisable to detect the image difference between frames, and if the difference is less than a certain level, there is no change in the subject's movement, and if the difference is more than a certain level, there is a change in the subject's movement. In particular, it is advisable to use an evaluation method in which the evaluation is performed by analyzing the video, and the closer the driving is to following predetermined traffic rules, the higher the evaluation, and the closer the driving is to not following predetermined traffic rules, the lower the evaluation. For example, when a car is stopped at a traffic light, it can be considered that it is more important to evaluate the locations where the subject's movement changes depending on whether the car is turning right or left or changing lanes than to locations where the subject's movement does not change. Driving evaluation can be performed in such situations.

[0033] (14) For example, video data may be recorded by a recording device, and the video may be displayed based on the portion of the video that includes the customer.

[0034] In this way, it becomes easier to view the video based on the video portion that includes the customer. The video data can be analyzed to determine whether the video portion includes the customer, and if a person other than the driver is detected inside the vehicle, the video portion can be determined to include the customer. For example, if a recording device is installed in a commercial vehicle such as a bus or taxi, and video data is recorded by such a recording device, the video containing the customer can be displayed in the commercial vehicle.

[0035] (15) For example, based on a keyword that describes the situation of a video portion, video portions related to the keyword may be found from the video.

[0036] In this way, it becomes easier to find the image portion related to the keyword from within the image. In particular, it is preferable to detect the keyword input from an input means for inputting the keyword that represents the situation of the image portion, and to find the image portion related to the detected keyword by analyzing the image with an analysis means. The situation of the image portion may be, for example, an abstract situation such as bright or dark, or a situation in which a specific subject was included, such as including a bullet train or a red car.

[0037] (16) For example, the video data is recorded by a recording device, and based on a keyword that describes the condition of the car at the time the video data was recorded, video portions related to the keyword are found from within the video.

[0038] In this way, it is more likely that video portions related to keywords that represent the vehicle's status at the time of recording can be found. It is advisable to detect keywords inputted into an input means for inputting keywords that represent the vehicle's status at the time of recording of the video data, and to find video portions related to the detected keywords by analyzing the video with an analysis means. Keywords that represent the vehicle's status may be, for example, that the vehicle accelerated, that the vehicle exceeded a certain speed, or that the blinker was turned on.

[0039] (17) For example, the video data may be recorded by a recording device, and a warning may be issued when the user is not looking in the correct direction while driving a car.

[0040] In this way, it is easier to warn the user that they are not looking in the direction they should be looking. In particular, it is preferable to warn the user based on information input by an input means that inputs information indicating that the user is not looking in the direction they should be looking. Such an input means should display the user's gaze, detected by a detection means that detects the user's gaze while driving, on the reproduced image, and input information that determines that the displayed gaze is not looking in the predetermined direction. The warning may be, for example, by attaching a warning mark to the displayed image, by applying a specific color to part or all of the displayed image, or by outputting an alarm. It is good.

[0041] (18) For example, the video data may be recorded by a recording device, and the processing may be a processing for displaying the video while the user is driving a car, distinguishing between an area visible to the user and an area not visible to the user.

[0042] In this way, the range that the user can see and the range that the user cannot see can be easily understood. In particular, it is preferable to use a processing process that displays the image by distinguishing between the range that the user can see and the range that the user cannot see, based on information input from an input means that inputs information indicating the range that the user can see. For example, it is preferable to blur the range that the user cannot see, fill it with a semi-transparent color, or fill it completely.

[0043] (19) For example, the processing may involve processing the recorded video data so as to change the display range of the video depending on the orientation of a device that displays the video based on the processed video data.

[0044] In this way, it becomes easier to view the image according to the orientation of the device that displays the image. It is preferable to change the display range of the image based on information from a detection means that detects the orientation of the device that displays the image based on the processed image data. For example, it is advisable to record the entire 360-degree range when recording, and then display the image according to the orientation of the device that displays the image.

[0045] (20) For example, the video data may be recorded by a recording device, and the processing may involve processing the recorded video data so as to change the display range of the video with the vehicle's travel direction as the reference direction, depending on the orientation of a device that displays the video based on the processed video data.

[0046] In this way, it is highly possible to change the display range of the image based on the traveling direction of the car, and to view the image according to the orientation of the device that displays the image. It is preferable to change the display range of the image based on information from a detection means that detects the orientation of the device that displays the image based on the processed image data. It is possible to view not only images in the traveling direction when the car is traveling, but also images in directions other than the traveling direction.

[0047] (21) For example, it is advisable to display information regarding the condition of the vehicle at the time the video data was recorded.

[0048] In this way, the vehicle's status can be easily understood. The information relating to the vehicle's status may be, for example, the vehicle's speed, acceleration, or position. In particular, based on information input from an input means for inputting information relating to the vehicle's status at the time of recording of the video data, the information may be displayed on the display screen of the display device by the display control means. The information relating to the vehicle's status may be displayed superimposed on the video, or may be displayed in a separate area from the area in which the video is displayed. For example, the information relating to the vehicle's status may be displayed in the area above, below, to the left, or above the video.

[0049] (22) For example, video data may be recorded by a recording device, and an image of a map showing the range of movement of the vehicle at the time the video data was recorded may be displayed, and video based on the video data recorded by the recording device may be displayed at a specified position on the displayed map image.

[0050] In particular, it is preferable to detect the range of movement of the vehicle when the video data was recorded by the detection means, acquire image data representing an image of a map showing the detected range of movement by the acquisition means, display the map image represented by the acquired image data on the display screen of the display device by the display control means, specify any position of the displayed map image by the designation means, find the video data recorded at the specified position from the video data by the image analysis means, and display the found image on the display screen of the display device by the display control means. In this way, it becomes easier to view the video at the specified position from the map image. Even if the time of the video you want to view is unknown, as long as the position is known, you can view the video at that position.

[0051] (23) For example, it is preferable to display at least one of an image based on video data recorded by a user's recording device at a specified position in the displayed map image and an image based on video data recorded by a third party's recording device at a specified position in the displayed map image.

[0052] In this way, it is highly possible to view not only images based on video data recorded by the user's recording device, but also images based on video data recorded by a third party's recording device. In particular, a command requesting video data recorded by a third party's recording device at a specified position in the displayed map image is transmitted from the transmitting means to a server or the like on which such video data is recorded, and the video data transmitted from the server or the like in response to the command is received by the receiving means, and the video based on the video data recorded by the third party's recording device is displayed on the display screen of the display device by the display control means. Both the video based on the video data recorded by the user's recording device and the video based on the video data recorded by the third party's recording device may be displayed simultaneously, or may be displayed by switching between them. The video based on the video data recorded by the third party's recording device may be of the same season as the season in which the video data was recorded by the user's recording device, or may be of a different season. Furthermore, the video based on video data recorded by a third party's recording device may show the same weather as when the video data was recorded by the user's recording device, or it may show different weather.

[0053] (24) For example, a name including the location where the recorded video data was shot may be displayed as a name for identifying the recorded video data.

[0054] In this way, a name including the shooting location can be used as a name to identify the video data. This makes it easier to find the video data of the shooting location. In particular, it is preferable to detect the shooting location from data indicating the recording position of the recorded video data by a detection means, change the detected name including the shooting location to a name that identifies the recorded video data by a name change means, and display the changed name on the display screen of the display device by a display control means as a name that identifies the recorded video data. In particular, it is preferable to change the file name that stores the recorded data to a name that includes the shooting location of the recorded video data by the file name change means. For example, the file name of the video file that stores the video data may be a name that includes the shooting location. The name of the shooting location itself may be a name that identifies the video data.

[0055] (25) For example, a plurality of the video data may be recorded, and names that identify the video data shot at the same location may be displayed together.

[0056] In this way, it is more likely that it will be easier to tell whether the image data was taken at the same location. In particular, it is advisable to control the display device by the display control means so that names that identify the image data taken at the same location are displayed together, for example, consecutively on the display screen. Locations with many shadow areas can be considered to be everyday locations such as commuting to work or school, while locations with few identical locations can be considered to be non-everyday locations such as travel destinations. This increases the likelihood that video data from everyday locations and video data from non-everyday locations will be easily selected.

[0057] (26) For example, it may be a good idea to give priority to names that appear less frequently.

[0058] In this way, names with a low occurrence frequency can be displayed with priority, making it easier to select video data corresponding to the names. In particular, it is preferable to control the display device by the display control means so that names with a low occurrence frequency are displayed with priority on the display screen. Whether the occurrence frequency is low or not may be determined, for example, when the occurrence frequency is equal to or lower than a predetermined threshold. Display with priority means, for example, that when multiple identifying names are displayed in a vertical list format, the names are displayed at the top, and that when the names are displayed in order, the names are displayed in the early order.

[0059] (27) For example, it may be a good idea to display an image of a map of the location where the photo was taken.

[0060] In this way, it becomes more likely that a map of the shooting location can be viewed in addition to the image based on the video data. In particular, it is preferable that a map image of the shooting location is found by the search means based on the data indicating the location of the shooting location, and the found map image is displayed on the display screen of the display device by the display control means.

[0061] (28) For example, the video data may be recorded by a recording device, and an image of a map showing the range of movement of the vehicle at the time the video data was recorded may be displayed, and information regarding the operation of the vehicle at the time the video data was recorded may be displayed on the map image.

[0062] In this way, it becomes easy to understand what kind of vehicle operation was performed at the location shown in the map image. By looking at the information about the vehicle operation displayed on the map image, it is possible to know what kind of vehicle operation is likely to be required when driving a car at the location shown in the map image. It is preferable to find a map image showing the range of vehicle movement from data showing the range of movement by the search means, control the display device by the display control means to display the found map image on the display screen, input information about the vehicle operation from the input means, and control the display device by the display control means to display the input information on the map image. The vehicle operations may be, for example, operation of a turn signal, sudden steering, sudden braking, etc.

[0063] (29) For example, the video data may be recorded by a recording device, and an image based on the processed video data may be displayed, along with information based on a comparison between a third party's driving and the user's driving at the location where the video data was recorded.

[0064] In this way, it becomes easier to compare the third party's driving with the user's driving. This comparison increases the possibility that the user can objectively evaluate his / her own driving skills in comparison with the third party's driving. In particular, data indicating the recording location of the video data is input from the input means, information indicating the state of the third party's driving at the recording location indicated by the input data is acquired by the acquisition means, and the acquired information about the third party's driving is used to compare the user's driving with the third party's driving by the comparison means, and the display control means controls the display device so as to display information based on the comparison on the display screen. The information based on the comparison between the third party's driving and the user's driving may be, for example, a ranking of the third party's driving with that of the user, an average evaluation value of the third party's driving with an evaluation value of the user's driving, etc.

[0065] (30) For example, the video data may be recorded by a user's recording device, and the video based on the processed video data and the video based on the video data recorded on the recording device in response to the driving of a third party at the location where the video data is recorded may be displayed.

[0066] In this way, it becomes easier to compare the footage of the user's driving with the footage of a third party's driving. By watching the third party's driving, the user is more likely to be able to understand how good or bad his or her own driving is. In particular, it is preferable to input data indicating the recording location of the video data from the input means, acquire the video data recorded according to the third party's driving at the recording location indicated by the input data by the acquisition means, and control the display device by the display control means to display on the display screen the video based on the acquired video data recorded according to the third party's driving and the video obtained according to the user's driving. The recording location of the video data is, for example, the location where the user recorded the video data obtained by driving a car in which a recording device is installed.

[0067] (31) For example, information obtained through at least one of road-to-vehicle communication and vehicle-to-vehicle communication is recorded in association with video data, and control is performed to display the information obtained through at least one of road-to-vehicle communication and vehicle-to-vehicle communication in association with the video based on the processed video data.

[0068] In this way, the information obtained by at least one of the road-to-vehicle communication and the vehicle-to-vehicle communication becomes easier to understand. In particular, it is preferable to read the information obtained by at least one of the road-to-vehicle communication and the vehicle-to-vehicle communication by the reading means, and control the display device by the display control means so as to display the read information on the display screen. The information obtained by at least one of the road-to-vehicle communication and the vehicle-to-vehicle communication may be, for example, information obtained based on ITS (Intelligent Transport Systems). The information obtained by the road-to-vehicle communication may be, for example, information transmitted from a roadside unit installed on the road, and the information obtained by the vehicle-to-vehicle communication may be information communicated between vehicles. Displaying in association with the image may mean that the information is associated with the image, and may be displayed on the image, may be displayed in a separate area from the image, or may be displayed when the image is displayed.

[0069] (32) For example, the processing may be a processing for correcting the color to a memory color.

[0070] In this way, it becomes easier to obtain an image based on memory colors. In particular, it is a good idea to use this as a correction means for correcting the image to memory processing. For example, it becomes more likely that an image relatively close to the user's memory will be obtained. Memory colors can be, for example, colors that the user remembers as an image, or colors that the user remembers for a specific thing. Correction to memory colors can be done, for example, by changing brightness, hue, saturation, etc. through user operation while watching the image.

[0071] (33) For example, the processing may be a processing for blurring at least a portion of an image based on the recorded video data.

[0072] In this way, it is highly likely that at least a part of the image is blurred. In particular, it is preferable to perform the processing using a blurring means that performs a process of blurring the image. A specified range of the image may be blurred, or the image outside the specified range may be blurred. It may be possible to switch between blurring the specified range and blurring the outside the specified range.

[0073] (34) For example, the communication robot may be further provided with a communication robot that faces a display screen displaying the video when outputting audio about the video based on the processed video data, and faces a viewer of the video based on the processed video data when outputting audio to the viewer.

[0074] In this way, it is highly likely that a communication robot can be obtained that faces the display screen displaying the image when audio is output for the image, and faces the viewer when audio is output for the viewer of the image. In particular, the communication robot should have audio output means, head rotation means capable of moving its head between the display screen and the viewer, a determination means for analyzing the image displayed on the display screen and determining whether audio should be output for the image displayed on the display screen or to the viewer, and a head rotation control means for controlling the head rotation means so that the head faces the display screen displaying the image when audio is output for the image, and faces the viewer when audio is output to the viewer of the image. When audio is output for the image, the head faces the display screen displaying the image, so that the viewer of the image will pay attention to the image, and when audio is output to the viewer, the head faces the viewer, so that the viewer of the image will be able to listen carefully to the audio output from the communication robot. It is preferable that the communication robot can, for example, face the display screen and the viewer, and can output audio.

[0075] (35) For example, it is advisable to perform control to change the manner in which the image based on the processed image data is displayed based on information about roads.

[0076] In this way, it becomes easier to change the display mode of the image based on the information about the road. In particular, it is preferable to analyze the image represented by the image data, detect the information about the road contained in the image, and change the display mode of the image based on the processed image data based on the detected information about the road. The information about the road may be, for example, the width of the road, road signs, etc. For example, the brightness, hue, saturation, etc. of the image may be changed based on the information about the road, or the image may be displayed from a different camera angle to change the display mode.

[0077] (36) For example, the present invention may be a recording device that records the video data to be played back on the playback device.

[0078] In this way, it is more likely that the video data to be played back in the playback device can be recorded. For example, the playback device may be a drive recorder that records the video data to be played back in the playback device. The recording device may be installed in a four-wheeled vehicle such as an automobile, a two-wheeled vehicle such as a motorcycle, or a bicycle.

[0079] (37) For example, the recording device may perform control to record the video data in high image quality based on specified conditions.

[0080] In this way, it becomes easier to record video data in high image quality based on the specified conditions. In particular, it is preferable to specify conditions using the specification means, input the specified conditions using the input means, and control the recording device using the recording control means so that the video data is recorded in high image quality based on the input conditions. The specified conditions may be, for example, location, time, etc. High image quality may be, for example, a frame rate higher than a certain threshold value, or a resolution higher than a certain resolution.

[0081] (38) For example, at least one of the playback device and the recording device may have a designation means for designating conditions, and control may be performed to record the video data in high image quality based on the conditions designated by the designation means.

[0082] In this way, it is more likely that video data can be recorded in high quality based on the conditions specified by the specification means. The designated conditions may be input, and the recording control means may control the recording device so that the video data is recorded in high image quality based on the input conditions. When the reproducing device is equipped with a designation means, for example, data representing the conditions designated by the designation means may be recorded on a recording medium on which the video data is recorded, such a recording medium may be loaded into the recording device, and the data representing the designated conditions may be read from the recording medium, thereby recording the video data in high image quality in the recording device based on the designated conditions.

[0083] (39) For example, the condition may be either that the speed of the vehicle in which the recording device is installed exceeds a certain level, or that the vehicle in which the recording device is installed has traveled through a designated location.

[0084] In this way, it becomes easier to record video data in high image quality when the speed of the car exceeds a certain level or when the car has traveled through a designated location. In particular, it is preferable to record the speedometer of the car using a camera provided in the recording device, analyze the captured image using an analysis means, and perform high image quality recording when the speedometer determines that the car's speed has exceeded a certain level using a determination means. Also, the location where the car is traveling may be detected by a position detection means, and high image quality recording may be performed when the determination means determines that the car is traveling through a designated location.

[0085] (40) For example, when the video data is recorded by the recording device, information obtained by at least one of the road-to-vehicle communication and the vehicle-to-vehicle communication may be recorded in association with the video data.

[0086] In this way, it is more likely that information obtained by at least one of road-to-vehicle communication and vehicle-to-vehicle communication can be recorded in association with video data. In particular, it is preferable to receive information transmitted by at least one of road-to-vehicle communication and vehicle-to-vehicle communication by a receiving means, and record the received information in association with video data obtained by filming. In particular, it is preferable to record information transmitted by at least one of road-to-vehicle communication and vehicle-to-vehicle communication in the header of a video file that stores the video data obtained by filming.

[0087] (41) For example, it is preferable to control the transmission of data representing a first calculated value, which is uniquely determined from the video data recorded by a recording device, to a blockchain system.

[0088] In this way, it is highly likely that data representing the first calculated value uniquely determined from the video data can be transmitted to the blockchain system. It is possible to record the data representing the first calculated value in the blockchain. In particular, it is preferable to calculate the first calculated value uniquely determined from the video data recorded by the recording device, and control the transmission means to transmit the data representing the calculated first calculated value to the blockchain system. The calculation means for calculating the first calculated value may be provided in the recording device or in the playback device, and the transmission means may also be provided in the recording device or in the playback device. Since the blockchain is relatively unlikely to be tampered with, by recording the data representing the first calculated value in the blockchain, it is found that there is a high possibility that the data representing the first calculated value has not been tampered with, and it is highly likely that the data representing the first calculated value can be used to prove that the video data has not been tampered with.

[0089] (42) For example, the video data recorded by the recording device is video data based on MPEG, and the portion immediately before a P frame is used as a unit for dividing the video data, and control is performed to record data representing a first calculated value that is uniquely determined for each unit in the block chain.

[0090] In this way, it is highly possible to record data representing the first calculated value for each unit by dividing the image immediately before the P frame in the blockchain. Since a P frame cannot be reproduced as a single image without an I frame, if the first calculated value is calculated for each unit by dividing the image immediately before the P frame, it is highly possible that the calculation of the first calculated value will become more difficult. In particular, it is preferable to control the transmission means so that the unit immediately before the P frame is used as a unit for dividing the image data, a first calculated value that is uniquely determined for each unit is calculated, and the data representing the calculated first calculated value is transmitted to the blockchain system.

[0091] (43) For example, it may be possible to control the recording of data representing a second calculated value that is uniquely determined from the event data in the blockchain.

[0092] In this way, it is highly possible to record data representing the second calculated value that is uniquely determined by the event data in the blockchain. It is possible to make it difficult to tamper with the second calculated value. In particular, it is advisable to control the transmission means so that the second calculated value that is uniquely determined from the event data is calculated by the calculation means and the transmission means transmits data representing the calculated second calculated value to the blockchain system. By comparing the second calculated value with the event data, it is possible to determine whether the event data has been tampered with, and since it is difficult to tamper with the second calculated value, it is also difficult to tamper with the event data.

[0093] (44) For example, it is advisable to record data indicating whether the video data recorded in the recording device was taken during manual driving or automatic driving in association with the video data.

[0094] In this way, it is more likely that it will be possible to determine whether the video data is from manual driving or automatic driving. In particular, it is advisable to input identification data from an input means, which indicates whether the video data is recorded in the recording device during manual driving or automatic driving, and record the input identification data in association with the video data. In some cases, it is advisable to record the identification data in the header area of ​​the video file in which the video data is recorded. Manual driving, for example, is when a person drives a car by operating the steering wheel, and automatic driving is when a person does not operate the steering wheel but instead operates the steering wheel by machine control.

[0095] (45) For example, it is advisable to change the image quality of the video data recorded by the recording device during manual driving and automatic driving.

[0096] In this way, it is highly possible that the image data during manual driving and the image data during automatic driving have different image quality. In particular, it is preferable to input identification data indicating whether the vehicle is being driven manually or automatically from the input means, and change the image quality of the image data recorded when the identification data indicates that the vehicle is being driven manually and the image quality of the image data recorded when the identification data indicates that the vehicle is being driven automatically by the change means. The image data may be recorded with a high image quality during manual driving and a low image quality during automatic driving, or conversely, the image data may be recorded with a low image quality during automatic driving and a high image quality during automatic driving.

[0097] (46) For example, it would be good to have a communication robot operate during autonomous driving.

[0098] In this way, it becomes more likely that the communication robot can be operated during autonomous driving. During autonomous driving, for example, the user does not need to drive, so the user can communicate with the communication robot without interfering with driving. In particular, it is preferable that the input means of the recording device inputs identification data indicating that the vehicle is in autonomous driving mode, and that the transmission means of the recording device transmits an operation command to the communication robot in response to the input of such identification data to the input means.

[0099] (47) For example, it is advisable to display an image based on recorded video data, and perform control to change the display mode of the image depending on whether or not it is likely that an event will occur.

[0100] In this way, it becomes easier to change the display mode of the image in response to the determination that the probability of an event occurring is high. In particular, it is preferable to provide a display device in a recording device provided in a vehicle, display an image on the display screen of the display device, analyze the image by an analysis means while the vehicle is traveling, and control the display device by a display control means so as to change the mode of the image displayed on the display screen in response to the determination by the determination means that the probability of an event occurring in front of the vehicle is high as a result of the analysis. For example, by recognizing that the display mode of the image has changed, it is possible to know that an event is likely to occur, and therefore it is possible to prepare for the event. For example, when the image data is taken by a drive recorder installed in a traveling vehicle, the change in the image mode indicates that, for example, some event, such as a pedestrian or a car suddenly jumping out, may occur in a few seconds.

[0101] (48) For example, it is advisable to record data representing the communication status of communication radio waves received by a communication device.

[0102] In this way, it becomes easier to record data representing the communication state of the communication radio waves. In particular, it is preferable to receive the communication radio waves using a communication device compatible with receiving the communication radio waves, and record data representing the communication state of the communication radio waves. In particular, the data representing the communication state should indicate whether communication is in progress or communication is interrupted. For example, the communication radio waves should be radio waves based on LTE (Long Term Evolution), Wi-Fi, or Bluetooth (registered trademark). When displaying an image, the display mode may be changed according to the communication state.

[0103] (49) For example, the recording mode of the video data may be changed based on radio waves for communication received by a communication device.

[0104] In this way, it becomes easier to change the recording mode of the video data based on the communication radio waves. In particular, it is preferable to receive the communication radio waves using a communication device capable of receiving the communication radio waves, and change the recording mode of the video data based on the received communication radio waves. In particular, the recording mode may be any mode as long as it changes the way of recording. For example, when the communication radio waves are being received, a recording mode with high image quality may be used, and when the communication radio waves are not being received, a recording mode with low image quality may be used. The reverse may also be used.

[0105] (50) For example, it is advisable to record video data showing an image of a driver who has received a roll call in association with the video data obtained by the driver's driving.

[0106] In this way, it becomes more likely that the video data showing the image of the driver who has received the roll call can be recorded in association with the video data obtained by the driver's driving. In particular, it is advisable to input the video data showing the image of the driver who has received the roll call by the input means, and record the input video data in association with the video data obtained by filming the driver's driving by a camera installed in the recording device. In particular, it is advisable to store these video data in the same folder. For example, it becomes possible to display the video obtained by the driver's driving in association with the video of the driver who has received the roll call. For example, it becomes possible to simultaneously display the video of the driver who has received the roll call and the video of the driver's driving.

[0107] (51) For example, it is advisable to perform control so as to associate and record the video data obtained by a recording device installed in the truck driven by the driver with the video data obtained by a recording device installed in the forklift operated by the driver.

[0108] In this way, it is easy to record the video data when driving the truck and the video data when driving the forklift in association with each other. In particular, it is preferable to transmit the video data obtained by filming the driving of the truck driver by a camera installed in a recording device attached to the truck to the server, and transmit the video data obtained by filming the driving of the forklift driver by a camera installed in a recording device attached to the forklift to the server, and record these video data in association with each other on the server. In particular, it is preferable to create a video folder for each driver on the server, and store the video data when the driver drove the truck and the video data when the driver drove the forklift in the same video file. Also, when the truck was driven, the video data when the truck was driven may be recorded on a recording medium loaded in a recording device attached to the truck, and when the forklift was driven, the recording medium may be removed from the recording device attached to the truck and loaded into a recording device attached to the forklift, and the video data when the forklift was driven may be stored in the same video file as the video file in which the video data when the truck was driven is stored. For example, it becomes possible to compare the video of the driver driving the truck with the video of the driver driving the forklift.

[0109] (52) For example, it would be a good idea to control the display so that information that drivers should pay attention to is superimposed on the real scenery seen from a moving car.

[0110] In this way, it becomes more likely that information that one should pay attention to when driving can be displayed superimposed on the real scenery. When actually driving, one can be made aware of information that one should pay attention to while viewing the real scenery. In particular, it is advisable to display information that one should pay attention to when driving on a mixed reality (MR) device while viewing the real scenery through the MR device. In particular, it is advisable to have a user wear MR device glasses, show the real scenery through the glasses, and display information that one should pay attention to when driving in a semi-transparent form on the surface of the glasses, superimposed on the real scenery.

[0111] (53) For example, it is advisable to carry out control to notify a user that the vehicle has traveled through the same place as a previous vehicle.

[0112] In this way, it becomes easy to know whether the vehicle has been driven before. Since it is known that the vehicle has been driven before, new developments can be considered, such as whether to drive the same route or a different route. In particular, data indicating the previous drive location is recorded on the recording medium loaded into the recording device, and the previous drive location represented by the recorded data is compared with the current drive location. If they match, the vehicle is notified that it is driving in the same location as the previous drive location by outputting a voice from the voice output device, displaying the information on the display screen of the display device, etc. The current drive location can be detected by using a GPS (global positioning system) receiver provided in the recording device.

[0113] (54) For example, it is advisable to photograph the speedometer of a car and control the recording of the video data based on the photographed speedometer.

[0114] In this way, it is more likely that video data can be recorded based on the captured speedometer. In particular, it is preferable to install a camera in the recording device in a position inside the vehicle where the speedometer can be captured, capture the vehicle's speedometer with the camera, analyze the captured image with an analysis means, and start recording the video data when the speedometer indicates a speed that satisfies the conditions for recording the video data. For example, it is preferable to record the video data when the vehicle's speedometer exceeds 80 km.

[0115] (55) For example, a recording device may be attached to an automated guided vehicle and control the recording of the video data based on the detection of the approach of an object around the automated guided vehicle.

[0116] In this way, it becomes more likely that video data can be recorded based on the detection of the approach of a surrounding object of the automatic guided vehicle. In particular, it is preferable to record the video data obtained by the camera captured by the recording device when the detection means detects the approach of an object surrounding the automatic guided vehicle. For example, when the automatic guided vehicle comes into contact with a surrounding object, the situation can be understood.

[0117] (56) For example, it is preferable that the recording device is attached to a forklift and controls the recording of the video data based on at least one of the operation of the forklift and the driver getting on the forklift.

[0118] In this way, it becomes easier to record video data based on at least one of the forklift operation and the driver getting on the forklift. In particular, it is preferable to start recording video data in response to the detection means detecting at least one of the forklift operation and the driver getting on the forklift. In particular, it is preferable to detect whether the forklift has operated or whether the driver has gotten on the forklift by analyzing the video data obtained by filming the inside of the forklift using a camera provided in the recording device, and start recording video data in response to the detection. The operation of the forklift may be, for example, the movement of the forks. For example, the recording of video data may be stopped when the driver gets off the forklift.

[0119] (57) Event recording trigger information may be recorded in the header of the file that stores the video data.

[0120] This increases the possibility of recording event record trigger information in the header of the file. For example, this increases the possibility of using the trigger information to perform various processes.

[0121] (58) For example, a playback method for playing back recorded video data may be characterized in that the recorded video data is processed based on acquired information, and a control is performed to display an image based on the processed video data.

[0122] In this way, it is more likely that the user will be able to see video that has been processed based on the acquired information, rather than the recorded video itself. By processing the video in a way that suits the user viewing the video, it is possible to obtain video that suits the user. Since the recorded video data is processed based on the acquired information, rather than processing the recorded video, it is more likely that video that is more suitable for the user viewing the video can be displayed.

[0123] (59) In the reproducing method, it is preferable to control a recording device so as to record the video data to be reproduced.

[0124] This increases the likelihood that the video data to be played back can be recorded in the recording device.

[0125] (60) For example, a computer-readable program for playing back recorded video data, which processes the recorded video data based on acquired information and controls the computer to display a video based on the processed video data. It is advisable to use a program that can be read by the data.

[0126] In this way, the user can view not the recorded image itself, but image that has been processed based on the acquired information. By processing the image in a way that suits the user viewing the image, it is possible to obtain an image that suits the user. Since the recorded image is not processed, but the recorded image data is processed based on the acquired information, there is a greater possibility that an image that is more suitable for the user viewing the image can be displayed.

[0127] The inventions shown in (1) to (60) above can be combined in any way. For example, at least a part of the configuration of at least one of the inventions from (2) onwards may be added to all or a part of the configuration of the invention shown in (1). In particular, it is preferable to add at least a part of the configuration of at least one of the inventions from (2) onwards to the invention shown in (1). In addition, any configuration may be extracted from the inventions shown in (1) to (60) and the extracted configurations may be combined. The applicant of this application intends to obtain rights to inventions including these configurations. Effect of the Invention

[0128] In this way, the user can view not the recorded image itself, but image that has been processed based on the acquired information. By processing the image in a way that suits the user viewing the image, it is possible to obtain an image that suits the user. Since the recorded image is not processed, but the recorded image data is processed based on the acquired information, there is a greater possibility that an image that is more suitable for the user viewing the image can be displayed. The effects of the invention of this application are not limited to this, and effects obtained from the configuration disclosed in the specification and drawings, etc. are also disclosed, and the intention is to obtain rights to the configuration that achieves the effects by divisional applications, amendments, etc. For example, in this specification, the phrase "can" is a description that clearly indicates the effect that is achieved, and there are also parts that show the effect even without the phrase "can." Also, there are effects that can be understood from the configuration even without such a description. [Brief description of the drawings]

[0129] [Figure 1] FIG. 2 is a side view of the drive recorder. [Diagram 2] FIG. 2 is a block diagram showing the electrical configuration of the drive recorder. [Diagram 3] 2 shows the file structure of a video file. [Figure 4] 13 is a flowchart showing a recording process procedure. [Diagram 5]13 is a flowchart showing a procedure for generating header data. [Figure 6] 13 is a flowchart showing a procedure of a recording process of video data. [Figure 7] 13 is a flowchart showing a data output processing procedure. [Figure 8] 13 is a flowchart showing a data output processing procedure. [Figure 9] 1 is an example of an image displayed on a display screen. [Figure 10] FIG. 2 is a block diagram showing the electrical configuration of a personal computer. [Figure 11] 13 is a flowchart showing a playback process procedure. [Figure 12] 13 is a flowchart showing a playback process procedure. [Figure 13] 13 is a flowchart showing a playback process procedure. [Figure 14] 1 is an example of a playback window. [Figure 15] 1 is an example of a playback window. [Figure 16] 1 is an example of a playback window. [Figure 17] 1 is an example of a playback window. [Figure 18] 1 is an example of a playback window. [Figure 19] 1 is an example of a playback window. [Figure 20] 1 is an example of a playback window. [Figure 21] 1 is an example of a playback window. [Figure 22] 1 is an example of a playback window. [Diagram 23] 1 is an example of a playback window. [Figure 24] 13 is an example of a play list display area. [Diagram 25] 13 is an example of a play list display area. [Figure 26] An I frame and a P frame are shown. [Figure 27] 13 is a flowchart showing a recording process procedure. [Figure 28] 1 is a flowchart showing the processing procedure of a drive recorder installed in a truck. [Figure 29] 13 is a flowchart showing the processing steps of a drive recorder attached to a forklift. [Diagram 30] 13 is a flowchart showing a recording process procedure. [Diagram 31] 13 is a flowchart showing a processing procedure of an automatic guided vehicle. [Diagram 32] 13 is a flowchart showing a processing procedure of driving evaluation. [Diagram 33] 13 is a flowchart showing a playback process procedure. [Diagram 34] 13 is a flowchart showing a playback process procedure. [Diagram 35] 13 is a flowchart showing a playback process procedure. [Diagram 36] 1 is an example of a map image. [Figure 37] 13 is a flowchart showing a playback process procedure. [Figure 38] 1 is an example of a playback window. [Figure 39] 1 is an example of a playback window. [Diagram 40] This is an example of a personal computer and a communication robot. [Diagram 41] FIG. 2 is a block diagram showing the electrical configuration of the communication robot. [Diagram 42] 1 is a flowchart showing the processing steps of a communication robot. [Diagram 43] This shows a communication robot placed inside a car. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0130] (1) Record processing Figures 1 to 9 show the recording process using a drive recorder.

[0131] FIG. 1 shows an embodiment of the present invention and is a side view of a drive recorder (eg, an example of a recording device) 1.

[0132] The drive recorder 1 has a mounting section 2 and a camera section 4.

[0133] The mounting part 2 is fixed to the windshield (not shown) of the vehicle to which the drive recorder 1 is attached, and allows the memory card to be freely attached and detached. A display screen 3 for displaying the captured image is formed on a part of the outer periphery of the mounting part 2. Furthermore, the mounting part 2 is equipped with a DC (direct current) jack (not shown) that connects to the vehicle via a converter unit (not shown). Furthermore, the mounting part 2 is equipped with predetermined buttons (not shown).

[0134] The camera unit 4 has a built-in front camera (not shown in FIG. 1) that mainly captures the front (the driving direction of the vehicle in which the drive recorder 1 is attached) and a rear camera (not shown in FIG. 1) that mainly captures the rear. The camera unit 4 has a front lens 5 that forms an image of a subject on the imaging plane of the front camera, and a rear lens 6 that forms an image of a subject on the imaging plane of the rear camera. The front camera can capture the forward direction of the vehicle, the left and right directions in front of the vehicle, and the up and down directions in front of the vehicle, while the rear camera can capture the backward direction of the vehicle, the left and right directions in the rear of the vehicle, and the up and down directions in the rear of the vehicle. Therefore, the front camera and the rear camera can capture images in all directions centered on the drive recorder 1.

[0135] The camera unit 4 can move back and forth within a range of a predetermined angle θ with respect to the mounting part 2 as the center, making it possible to take pictures at a relatively suitable angle regardless of the angle of the windshield.

[0136] FIG. 2 is a block diagram showing the electrical configuration of the drive recorder 1. As shown in FIG.

[0137] The overall operation of the drive recorder 1 is controlled by a controller 10 .

[0138] The drive recorder 1 is equipped with a memory card reader / writer 95. When the memory card 25 is inserted in the drive recorder 1, the memory card reader / writer 95 reads data recorded in the memory card 25 and writes data to the memory card 25. An operation program described later is recorded in the memory card 25, and the recorded operation program is read by the controller 10 and installed in the drive recorder 1. The controller 10 reads the operation program installed in the drive recorder 1 and controls each part of the drive recorder 1 according to the operation program. If the operation program of the drive recorder 1 is stored in a recording medium other than the memory card 25, it may be read from such a recording medium. In this case, an interface such as a driver corresponding to the recording medium is provided in the drive recorder. It may also be received by the communication circuit 17 and installed in the drive recorder 1.

[0139] As described above, the drive recorder 1 includes a front camera 11 and a rear camera 12. The image data captured by the front camera 11 and the rear camera 12 are input to the controller 10. The drive recorder 1 also includes a display device having a display screen 3, and a display control device 13 controls the display on the display device 14. The drive recorder 1 also includes a clock 15, a road-to-vehicle / vehicle-to-vehicle information receiving circuit 16, a communication circuit 17, and an acceleration sensor 18. Time data indicating the time output from the clock 15 is output to the controller 10. The road-to-vehicle / vehicle-to-vehicle information receiving circuit 16 receives, for example, data indicating the surrounding conditions transmitted from a roadside device arranged on a road or the like based on an advanced road information system (one example of road-to-vehicle information), data indicating the vehicle information transmitted from a vehicle (one example of vehicle-to-vehicle information), and the like, and outputs the received data to the controller 10. The road-to-vehicle / vehicle-to-vehicle information receiving circuit 16 uses, for example, a communication circuit based on DSRC (Dedicated Short Range Communications). For example, when a communication robot or the like is placed inside the vehicle, the communication circuit 17 communicates with the communication robot or the like. Output data from the communication circuit 17 is input to the controller 10. The acceleration sensor 18 detects the acceleration of the vehicle in which the drive recorder 1 is installed, and outputs the detected acceleration data to the controller 10.

[0140] Furthermore, the drive recorder 1 includes an input device 20 such as buttons, a memory 21 such as a RAM (Random Access Memory), and a GPS (Global Positioning System) receiver 22. The output signal from the input device 20, the output data from the memory 21, and the output data from the GPS receiver 22 are also input to the controller 10.

[0141] FIG. 3 shows the file structure of a video file that stores the video data output from the front camera 11 and the video data output from the rear camera 11.

[0142] The video file includes a header recording area 31 and a video recording area 32. The header recording area 31 is an area for recording header data, and the video recording area 32 is an area for recording video data.

[0143] FIG. 4 is a flowchart showing the recording process procedure of the drive recorder 1.

[0144] The recording process procedure shown in FIG. 4 is repeated, for example, for each frame (or any other frame) shot until the recording process is completed.

[0145] When the engine of the vehicle to which the drive recorder 1 is attached starts, the battery of the vehicle supplies power to the drive recorder. Then, the recording process shown in FIG. 4 starts. Also, a video file is generated in the memory card 25. When the recording process starts, the video data output from the front camera 11 and the rear camera 12 is input to the display control device 13 via the controller 10. The captured video is displayed on the display screen 3 of the display device 14 (see FIG. 9). The video displayed on the display screen 3 is either the video obtained by the front camera 11 or the video obtained by the rear camera 12 according to the program recorded in the memory card 25, but both the video obtained by the front camera 11 and the video obtained by the rear camera 12 may be displayed. Also, the video obtained by the front camera 11 or the video obtained by the rear camera 12 may be displayed as a default display, and a switching button may be provided on the input device 20, and the video obtained by the front camera 11 or the video obtained by the rear camera may be displayed in response to the operation of the switching button.

[0146] First, header data is generated and the generated header data is temporarily recorded in memory 21 (step 41), and then video data is temporarily recorded in memory 21 (step 42). Next, various data, etc. are output (step 43), and the data, etc. temporarily recorded in memory 21 is written to memory card 25 (step 44).

[0147] In Fig. 4, after the generation of header data and the temporary recording of the generated header data in memory 21 (step 41), the temporary recording process of video data in memory 21 is performed (step 42), and then the output process of various data, etc. is performed (step 43), but the order of these processes does not matter. Also, the header data and video data are temporarily recorded in memory 21, the header data and video data recorded in memory 21 are read, and the read header data and video data are written to memory card 25, but if possible, they may be written directly to memory card 25 without being temporarily stored in memory 21. Also, among the generation and recording process of header data (step 41), the recording process of video data (step 42), and the output process of data, etc. (step 43), any desired two or more processes may be performed in parallel.

[0148] When the engine of the vehicle in which the drive recorder 1 is installed is stopped and the power supply to the drive recorder 1 is stopped, the recording process procedure shown in FIG. 4 ends.

[0149] FIG. 5 is a flowchart showing the header data generation processing procedure (the processing procedure of step 41 in FIG. 4), and shows the processing procedure of the controller 10.

[0150] If the vehicle in which the drive recorder 1 is installed is capable of automatic driving, the vehicle transmits data to the drive recorder 1 indicating that it is in automatic driving mode (when the vehicle is not being driven based on the operation of the vehicle's driver). For example, by connecting the drive recorder 1 to the vehicle's in-vehicle network and detecting data output to the in-vehicle network indicating whether the vehicle is in automatic driving mode or manual driving mode (when the vehicle is being driven based on the operation of the vehicle's driver), the controller 10 of the drive recorder 1 recognizes whether the vehicle is in automatic driving mode or manual driving mode. In addition, if the vehicle and the drive recorder 1 are capable of wireless communication and the vehicle transmits data indicating whether the vehicle is in automatic driving mode or manual driving mode, the controller 10 of the drive recorder 1 recognizes whether the vehicle is in automatic driving mode or manual driving mode. By receiving this signal at the communication circuit 17 of the drive recorder 1, the controller 10 of the drive recorder 1 may recognize whether the vehicle is being driven automatically or manually.

[0151] The controller 10 of the drive recorder 1 judges whether the vehicle is in automatic driving or manual driving (step 51). In the case of automatic driving, data indicating that the data recorded on the memory card 25 is video data taken during automatic driving is temporarily recorded in the memory 21 as header data (step 52). In the case of manual driving, data indicating that the data recorded on the memory card 25 is video data taken during manual driving is temporarily recorded in the memory 21 as header data (step 53). For example, data indicating whether video data taken within a certain period of time is data during automatic driving or data during manual driving is temporarily recorded in the memory card 25. Of course, an automatic driving video folder for storing video data during automatic driving and a manual driving video folder for storing video data during manual driving may be generated, and video data taken during automatic driving may be stored in the automatic driving video folder, and video data taken during manual driving may be stored in the manual driving video folder. The time or the storage of video data in the automatic driving video folder or the storage of video data in the manual driving video folder is associated with data indicating whether the video data is during automatic driving or manual driving, and the video data is during automatic driving or the video data is during manual driving.

[0152] Next, the controller 10 checks whether the road-to-vehicle communication information or the vehicle-to-vehicle communication information has been received by the road-to-vehicle / vehicle-to-vehicle information receiving circuit 16 (step 54). When the road-to-vehicle communication information or the vehicle-to-vehicle communication information is received (YES in step 54), data representing the received communication information is temporarily recorded in the memory 21 together with data representing the time when the communication information was received (step 55). Since the time when the communication information was received is known, it is also possible to know what kind of video was being shot at that time, and when playing back the shot video, the road-to-vehicle communication information or the vehicle-to-vehicle communication information can be output at the same time. In this way, for example, the road-to-vehicle communication information or the field-of-view vehicle communication information can be associated with the video data by time. If neither the road-to-vehicle communication information nor the vehicle-to-vehicle communication information is received (NO in step 54), the process of step 55 is skipped and the received information is not temporarily recorded in the memory 21.

[0153] It is also confirmed whether or not radio waves for communication have been received by the communication circuit 17 (step 56). When radio waves for communication are received by the communication circuit 17 (YES in step 56), data representing the communication status of the radio waves for communication is temporarily recorded in the memory 21 (step 57). For example, when the communication circuit 17 receives radar waves emitted by speed enforcement devices installed on the road such as speed cameras, or radio signals used by emergency vehicles, data representing the contents of the signals is temporarily recorded in the memory 21. When radio waves for communication are not received by the communication circuit 17 (NO in step 56), the processing of step 57 is skipped, and data representing the communication status of the radio waves for communication is not recorded in the memory 21.

[0154] Furthermore, the gaze of the vehicle driver is detected by the controller 10 from the video data obtained by the rear camera 12 (step 58). Gaze data representing the detected gaze is also temporarily written to the memory 21 together with data representing the time (step 59). It is possible to know when and in which direction the driver was looking. The eyes are detected from the video, and the gaze is determined from the positional relationship between the pupil, iris, and cornea of ​​the eye (the so-called black part of the eye) and the conjunctiva (the so-called white part of the eye). Similarly, the video data obtained by the rear camera 12 is analyzed to detect the direction in which the vehicle driver is facing, and data representing the detected direction is also temporarily written to the memory 21 together with data representing the time of shooting.

[0155] FIG. 6 is a flow chart showing the procedure for recording video data (the procedure for step 42 in FIG. 4).

[0156] In the process of recording the video data, the controller 10 also checks whether the vehicle is in automatic or manual driving mode (step 61). If the vehicle is in automatic driving mode, video data of relatively low image quality is recorded in the memory 21 (step 62). For example, the controller 10 controls the front camera 11 and the rear camera 12 so that the number of pixels per frame represented by the video data output from the front camera 11 and the rear camera 12 is reduced below a first threshold value (reduced resolution) or the number of frames read per unit time is reduced below a first number of frames (reduced frame rate). This results in video data of relatively low image quality. If the vehicle is in manual driving mode, video data of relatively medium image quality is recorded in the memory 21 (step 63). For example, the front camera 11 and the rear camera 12 are controlled by the controller 10 so that pixel thinning is performed so that the number of pixels per frame represented by the video data output from the front camera 11 and the rear camera 12 becomes greater than the first threshold value, or the number of read frames per unit time is made greater than the first frame number. This results in video data of relatively medium image quality. In this way, the image quality of the video data can be adjusted by adjusting the resolution and frame rate. During automatic driving, video data with relatively low image quality is recorded in the memory 21, and during manual driving, video data with relatively medium image quality is recorded in the memory 21. However, during automatic driving, video data with relatively medium image quality may be recorded in the memory 21, and during manual driving, video data with relatively low image quality may be recorded in the memory 21. It is sufficient that the image quality of the video data recorded during automatic driving is different from the image quality of the video data recorded during manual driving.

[0157] Next, the controller 10 checks whether the specified condition is met (step 64). For example, the vehicle speed exceeds a certain level, the vehicle is traveling in a specified location, and if a high-image-quality designation switch (one example of a designation means) is provided on the input device 20 of the drive recorder 1, the high-image-quality designation switch is operated, which are examples of the specified condition. Whether the vehicle speed exceeds a certain level can be detected by analyzing the image obtained by the rear camera 12 if the drive recorder 1 is attached to the vehicle so that the rear camera 12 can capture the vehicle's speedometer, and can also be detected from the traveled distance calculated based on the GPS receiver 22 and the time it takes to travel that distance calculated based on the clock 15. Whether the vehicle is traveling in a specified location can be determined based on the output data from the GPS receiver 22.

[0158] When the specified condition is met (YES in step 64), relatively high-quality video data is temporarily recorded in memory 21 (step 65). For example, front camera 11 and rear camera 12 are controlled by controller 10 so as to stop pixel thinning of the video data output from front camera 11 and rear camera 12, and to increase the number of read frames per unit time to be greater than the number of frames at medium resolution. This allows relatively high-quality video data to be obtained. If the specified condition is not met (NO in step 64), the process of step 65 is skipped, and the process of recording high-quality video data in memory 21 is not performed.

[0159] Furthermore, it is also confirmed whether a command to change the recording mode has been given to the controller 10 (step 66). If a command to change the recording mode has been given (YES in step 66), the recording mode is changed (step 67), and if no command to change the recording mode has been given (NO in step 66), the recording mode is not changed. For example, as described above, it may be determined that a command to change the recording mode has been given when the communication circuit 17 receives radar waves emitted by speed enforcement devices installed on the road such as speed cameras, or radio waves for communication such as those used by emergency vehicles. When a command to change the recording mode has been given, the recording mode can be changed so that high-quality video data is recorded in the memory 21, for example.

[0160] In the video data recording process shown in FIG. 6, after checking whether the vehicle is in automatic or manual operation (step 61), a check is made as to whether specified conditions have been met (step 64). However, after checking whether specified conditions have been met (step 64), a check as to whether the vehicle is in automatic or manual operation (step 61) may also be made, or both checks (steps 61, 64) may be made in parallel.

[0161] FIG. 7 is a flowchart showing the data output processing procedure (the processing procedure of step 43 in FIG. 4).

[0162] In the data output processing procedure, the controller 10 also checks whether the vehicle is in automatic or manual driving (step 71). In the case of automatic driving, the communication circuit 17 transmits an operation start command to the communication robot (not shown) placed in the vehicle (step 72). In the case of manual driving, the communication circuit 17 does not transmit an operation start command to the communication robot placed in the vehicle. In the case of manual driving, a communication robot stop command may be transmitted. During automatic driving, the driver is not driving the vehicle manually, so the operation of the communication robot does not interfere with driving, but the driver can communicate with the communication robot, so time can be used effectively. In the case of manual driving, the communication robot placed in the vehicle does not operate, so it is possible to prevent the driver from being distracted by the communication robot talking to him / her. In addition, if an operation stop command is transmitted to the communication robot during manual driving, the communication robot can be stopped when it is operating.

[0163] Next, the controller 10 checks whether or not there is a high possibility that an event will occur (step 73). For example, if the road on which the vehicle is traveling is narrow or there is a lot of pedestrian traffic, it is considered that there is a high possibility that the driver will suddenly brake and an event will occur. If there is a high possibility that an event will occur (YES in step 73), the controller 10 controls the display control device 13 to change the display mode of the image displayed on the display screen 3 of the drive recorder 1 (step 74). For example, it is advisable to change the display mode of the image by displaying the word "Caution!" on the display screen or superimposing the entire image displayed on the display screen 3 or a part of the image, such as the periphery, in a semi-transparent red or yellow color. If there is a low possibility that an event will occur (NO in step 73), the display mode of the image displayed on the display screen 3 of the drive recorder 1 cannot be changed.

[0164] Furthermore, when the vehicle is moving (or may be stopped), the controller 10 also checks whether the location where the vehicle is currently moving is the same as the location where the vehicle has previously moved (step 75). In the header recording area of ​​the video file of the memory card 25 attached to the drive recorder 1, GPS data indicating the location where the vehicle has previously moved is recorded. The controller 10 compares the GPS data recorded in the header recording area with the GPS data received by the GPS receiver 22 provided in the drive recorder 1 attached to the moving vehicle, thereby making it possible to check whether the location where the vehicle is currently moving is the same as the location where the vehicle has previously moved. If the vehicle is moving in the same location as before (YES in step 75), the drive recorder 1 notifies the driver that the vehicle is moving in the same location (step 76). If the vehicle is not moving in the same location as before (NO in step 75), the drive recorder 1 does not notify the driver that the vehicle is moving in the same location.

[0165] FIG. 9 is an example of the display screen 3 of the drive recorder 1.

[0166] The display screen 3 of the drive recorder 1 displays an image captured by a front camera 11. An image of the vehicle is displayed. At the bottom of the display screen 3, a character string 7 is displayed saying "You are now traveling in the same place you have traveled before." By looking at this character string 7, the driver knows that he or she is traveling in the same place as before. The driver can tell whether or not he or she is traveling in a place that he or she has the impression of having traveled before due to déjà vu, or whether or not the place is a place that he or she has actually traveled before. It does not have to be an exact match to be the same place as before, as long as it is within a certain range.

[0167] Although the character string 7 indicates that the current location is the same as the location previously traveled, it is also possible to display a predetermined mark or change the display mode of the display screen 3 without displaying the character string 7. When the driver sees such a mark or confirms the change in the display mode, the driver knows that the current location is the same as the location previously traveled. In addition, if the drive recorder 1 is equipped with a speaker, the driver may be notified that the current location is the same as the location previously traveled by outputting a voice or sound from the speaker without displaying the character string 7, etc. In any case, it is sufficient for the driver to know that the current location is the same as the location previously traveled.

[0168] Returning to FIG. 8, when one frame of video data is temporarily recorded in the memory 21, the controller 10 calculates a hash value (an example of a first calculated value uniquely determined from the video data) of the recorded video data (step 77). Data representing the calculated hash value is transmitted to the blockchain system by the communication circuit 17 (step 78). The blockchain system records the data representing the calculated hash value in the blockchain (block transaction). Since the blockchain is difficult to tamper with, it becomes possible to check whether the video data has been tampered with by using the hash value. Data representing the calculated hash value is also recorded in the memory card 25, and when checking whether the video data has been tampered with, the data representing the hash value transmitted to the blockchain system is compared with the data representing the hash value recorded in the memory card 25. If the hash values ​​match, it is determined that the video data has not been tampered with, and if the hash values ​​do not match, it is determined that the video data has been tampered with. It goes without saying that the video data recorded on memory card 25 and the data representing that hash value are associated with each other and recorded on memory card 25. For example, the data representing the hash value is recorded in the header recording area 31 of the video file together with data indicating the time of the video data from which the hash value was calculated.

[0169] In this way, once the header data and video data are temporarily recorded in memory 21, the recorded header data and video data are read from memory 21, the header data is written to the header recording area of ​​the video file, and the video data is written to the video recording area of ​​the video file.

[0170] (2) Recycling Figures 10 to 23 show the playback process using the playback viewer software.

[0171] FIG. 10 is a block diagram showing the electrical configuration of a tablet personal computer (an example of a playback device).

[0172] The overall operation of the tablet-type personal computer (hereinafter referred to as the personal computer) 80 is controlled by a CPU (Central Processing Unit) 80.

[0173] The personal computer 80 is provided with a display device 82. The display device 82 is It is controlled by a display control device 81 which is controlled by a CPU 90. The personal computer 80 is also provided with an acceleration sensor 93, and an output signal from the acceleration sensor 93 is input to the CPU 90. The personal computer 80 further includes an SSD (solid state drive) 94 accessed by the CPU 90, and a memory card reader / writer 95 which reads data etc. recorded on the memory card 25 and writes data etc. to the memory card 25.

[0174] In addition, the personal computer 80 includes input devices 96 such as a keyboard and a mouse, memory 97 such as RAM, and communication circuitry 98 for connecting to a network such as the Internet.

[0175] An operating program, which will be described later, is received by the communication circuit 98 of the personal computer 80 via the Internet and installed in the personal computer 80. The CPU 90 reads and executes the installed program, thereby controlling each part. The operating program may be stored in a recording medium such as the memory card 25, and the operating program may be read from such a recording medium and installed in the personal computer 80.

[0176] Figures 11 to 13 are flowcharts showing the playback process procedure in the personal computer 80. Figures 14 to 24 are examples of playback windows displayed on the display screen of the display device 92 of the personal computer 80.

[0177] The memory card 25 that was attached to the drive recorder 1 is removed from the drive recorder 1 and attached to the personal computer 80. When the operating program installed in the personal computer 80 is started by the user, the display device 92 A playback window 130 shown in FIG. 14 is displayed on the display screen.

[0178] FIG. 14 is an example of the playback window 130.

[0179] A menu bar 131 is formed at the top of the playback window 130, and a video display area 141 is formed below this menu bar 131. Furthermore, below the video display area 141, a map image display area 151, a vehicle information etc. display area 161, and a play list display area 171 are formed.

[0180] The menu bar 131 displays various items, and by selecting each item, a pull-down menu is displayed, allowing detailed instructions to be given. The menu bar 131 displays the character strings "File", "Display", "Playback", "Tools", and "Information", a folder icon 132, a camera icon 133, a printer icon 134, a reel icon 135, an FD (flexible disk) icon 136, a log icon 137, and a gear icon 138. By pulling down the character strings "File", "Display", "Playback", "Tools", and "Information", a pull-down menu corresponding to each item appears, allowing detailed instructions to be given. Clicking on the folder icon 132 specifies a folder, clicking on the camera icon 133 converts to a still image, clicking on the printer icon 134 prints, clicking on the reel icon 135 converts to video, clicking on the FD (Flexible Disk) icon 136 backs up data, clicking on the log icon 137 converts to log data, and clicking on the gear icon 138 changes recording settings. The video display area 141 is an area for displaying video recorded by the drive recorder 1 or the like. The map image display area 151 displays the vicinity of the shooting location of the video displayed in the video display area 141. The area 161 displays an image of a nearby map. The vehicle information display area 161 displays information such as the vehicle speed at the time of recording the video. The vehicle information display area 161 includes a vehicle status display area 162 that displays the status of the vehicle to which the drive recorder 1 that recorded the video displayed in the video display area 141 is attached, an operation button 163 that issues operation commands such as playing and stopping the video, a time display area 166 that displays the recording time of the video displayed in the video display area 141, and an acceleration graph display area 168 that displays a graph of the vehicle's acceleration. The play list display area 171 is an area that displays a list of the file names of the video files recorded in the memory card 25.

[0181] When the folder icon 132 is clicked and a folder in which video files are stored is selected, the file names of the video files stored in that folder are displayed in a clickable manner in a playlist display area 171, as shown in Fig. 14. The video recording start time is used as the file name, but it goes without saying that something other than the recording start time may be used as the file name.

[0182] When one of the file names displayed in the play list display area 171 is clicked using the mouse included in the input device 96, the corresponding video file is read from the memory card 25. Playback of the video data stored in the read video file begins (FIG. 11, step 111). In the header recording area of ​​the video file, various header data are recorded corresponding to the recording time of the video data, and the recorded header data is also read when the video at the recording time is displayed, and various superimposition processes are performed on the video depending on the contents of the read header data.

[0183] When playback of the video data (the video data being played back may be data that was shot and recorded when the user viewing the video was driving, or may be data that was shot and recorded when a user other than the user viewing the video was driving) begins, the video represented by the video data is displayed as a moving image in the video display area 141, as shown in FIG. 15. Also, data representing the GPS information recorded in the video file in which the video data is stored is transmitted via the Internet to a website that transmits data representing an image of a map. Then, image data representing a map in the vicinity of the location where the video displayed in the video display area 141 was recorded is transmitted from the site to the personal computer 80. When the image data representing the map is received by the personal computer 80, an image of the map is displayed in the map image display area 151. An arrow icon 152 representing the location displayed in the video display area 141 is also displayed on the map image. Furthermore, the vehicle status display area 162 of the vehicle information display area 161 displays the vehicle speed, the vehicle's longitudinal acceleration (X), the vehicle's lateral acceleration (Y), the vehicle's vertical acceleration (Z), the latitude (N) indicating the vehicle's position, and the longitude (E) indicating the vehicle's position at the time the image displayed in the image display area 141 was recorded. The time display area 166 displays the recording time of the image displayed in the image display area 141, and the acceleration graph display area 168 displays a graph of the acceleration.

[0184] Since the video data is recorded in all directions, it is possible to display not only the video of the subject seen from the traveling direction of the vehicle, but also the video of the subject behind (opposite to the traveling direction), the subject to the left of the vehicle (left direction based on the traveling direction of the vehicle) or the subject to the right of the vehicle (right direction based on the traveling direction of the vehicle). FIG. 16 shows a playback window 130 displaying the video of the subject behind in the video display area 141, FIG. 17 shows a playback window 130 displaying the video of the subject to the right in the video display area 141, and FIG. 18 shows a playback window 130 displaying the video of the subject to the left in the video display area 141. For example, if the mouse is dragged to the right in the state shown in FIG. 15, the video of the subject in the traveling direction gradually switches to the video of the subject to the right, and the video of the subject to the right is displayed in the video display area 141 as shown in FIG. 17. If the mouse is dragged further to the right in the state shown in FIG. 17, the video of the subject to the right will be displayed. 16, the image of the subject behind is gradually switched to the image of the subject to the left, as shown in FIG. 18, displayed in the image display area 141. When the mouse is further dragged to the right in the state shown in FIG. 16, the image of the subject behind is gradually switched to the image of the subject to the left, as shown in FIG. 18, displayed in the image display area 141. When the mouse is further dragged to the right in the state shown in FIG. 18, the image of the subject to the left is gradually switched to the image of the subject in the traveling direction, as shown in FIG. 15, displayed in the image display area 141. In this way, images of subjects in all directions can be displayed in the image display area 141. In the above embodiment, the mouse is dragged to the right, but when the mouse is dragged to the left, the image of the subject in the traveling direction is gradually switched to the image of the subject to the left, as shown in FIG. 18, as shown in FIG. 16, and then gradually switched to the image of the subject to the rear, as shown in FIG. 16, gradually switched to the image of the subject to the right, as shown in FIG. 17, and gradually switched to the image of the subject in the traveling direction, as shown in FIG. 15.

[0185] In this embodiment, when an image is displayed in the image display area 141, the field of view range is superimposed on the image (step 111 in FIG. 11).

[0186] FIG. 19 is an example of a playback window 130 showing the user's field of view.

[0187] In the video display area 141, a frame 143 indicating the field of view range is displayed by being superimposed (one example of processing to display on the video) by the CPU 90 on the video represented by the video data. The image portion of the area 142 outside the field of view range is blurred by the CPU 90 (hatching is shown to indicate that it is blurred). The field of view range is determined in advance for each user's age, and data indicating the field of view range is recorded in the SSD 94 in advance. By specifying the user's age, data indicating the field of view range corresponding to the specified age is read from the SSD 94, and the frame 143 indicating the field of view range and the area 142 outside the field of view range are blurred as shown in FIG. 19 (one example of control for processing the video data based on the acquired information and displaying the video based on the processed video data). Of course, the field of view range of each user may be detected in advance, and data indicating the field of view range specific to each user may be recorded in the SSD 94 in advance. The frame 143 indicating the field of view range of each user is superimposed on the video. The user knows the field of view and can therefore recognize which areas are visible and which are not.

[0188] The CPU 90 also judges whether or not there is an object (such as a small child) that should be noted in the area 142 outside the visual field (step 113 in FIG. 11). If there is an object that should be noted in the area 142 outside the visual field (YES in step 113 in FIG. 11), a warning (one example of a notification to the user) is issued to the user (step 114). For example, the part where the object that should be noted is displayed is surrounded by a frame or a mark is displayed. Whether or not there is an object that should be noted in the area 142 outside the visual field may be determined based on whether or not the CPU 90 detects that an image of the object is displayed in the area 142 outside the visual field by predetermining the object that should be noted and performing image analysis. Since the user knows that there is an object that should be noted outside the visual field, the user can understand that driving while shaking his / her head as necessary to substantially expand the visual field will lead to the prevention of accidents. If there is no object that should be noted in the area 142 outside the visual field (NO in step 113 in FIG. 11), the process of step 114 is skipped and no warning is issued. If an object requiring attention exists in the area outside the visual field 142, it is sufficient that the user is notified, and therefore the object does not necessarily have to be displayed in the area outside the visual field 142. For example, the user may be notified by outputting a sound.

[0189] Also, a mark 144 indicating the user's line of sight is superimposed on the image display area 141 by the CPU 90 (this is an example of processing for displaying the user's line of sight in the image) (FIG. 11, step 115). Data indicating the user's gaze is recorded in the header recording area 31 of the video file in association with the recording time of the video, so by reading data indicating the user's gaze from the header recording area 31, a mark indicating the user's gaze can be superimposed on the video displayed in the video display area 141 at the recording time of the displayed video. Since the user's gaze is superimposed on the video, it is possible to confirm whether the user is looking at the subject (for example, a traffic light, an oncoming vehicle, a pedestrian, etc.) that they should be looking at.

[0190] In the example shown in FIG. 19, the frame 143 of the field of view and the mark 144 indicating the user's line of view are displayed regardless of the user's line of view. That is, the frame 143 of the field of view indicates the range that would be the field of view when the user sees the image displayed in the image display area 141, so the frame 143 of the field of view is displayed in the approximate center of the image display area 141 regardless of the user's line of view while driving. For example, even if the user's line of view is not in the center of the image but is at the edge of the image displayed in the image display area 141, the frame 143 of the field of view is displayed in the approximate center of the image display area 141. In contrast, the frame 143 of the field of view may be displayed in the image display area 141 with the user's actual line of view as the center. The field of view when the user is actually driving can be known. In this case, blurring of the area 142 outside the field of view may also be performed. The area within the field of view frame 143 is an example of a range that is visible to the user while the user is driving the vehicle, and the area outside the field of view 142 is an example of a range that is not visible to the user while the user is driving the vehicle, and this is an example of how these ranges are displayed distinctly.

[0191] If the direction in which the user is facing has changed at the recording time of the image displayed in the image display area 142 (YES in step 116 in FIG. 11), the field of view range frame 143 is moved in the changed direction and superimposed on the image (step 117 in FIG. 11). Whether the direction in which the user is facing has changed can be determined based on data that indicates the direction in which the user is facing, which is recorded in the header recording area of ​​the image file. If the direction in which the user is facing has not changed (NO in step 116 in FIG. 11), the field of view range frame 143 is not moved.

[0192] FIG. 20 is an example in which the frame 143 of the field of view is moved.

[0193] Assume that the direction in which the user faces has changed from almost directly ahead in the direction of travel to a lower right direction when the image displayed in the image display area 141 is recorded. Then, the position is moved from frame 143 of the field of view range to frame 143A of the field of view range. Also, the position is moved from mark 144 indicating the line of sight to mark 144A indicating the line of sight. The user can recognize that the field of view has also changed due to the change in the direction in which the user faces. When the direction in which the user faces changes, the relative position of the image displayed in the image display area 141 and frame 143 (143A) indicating the field of view range also changes. For this reason, the relative position of the image and frame 143 indicating the field of view range may be kept unchanged (for example, frame 143 indicating the field of view range may be displayed almost in the center of the image). For example, an image in the direction in which the user faces may be displayed in the same way as changing the image displayed in image display area 141 when dragged with the mouse.

[0194] Next, it is determined whether the user's line of sight matches the direction in which the user should look (step 118 in FIG. 12). If they do not match (NO in step 118 in FIG. 12), a mark indicating the place in which the user should look is displayed on the video display area 141 (step 119 in FIG. 12), and if they match (YES in step 118 in FIG. 12), the mark indicating the place in which the user should look is not displayed on the video display area 141. However, if they match, the user may be notified of the match (by displaying on the video display area 141, outputting a sound, etc.). It goes without saying that the direction in which the user should look differs depending on the driving at the time of recording (right turn, left turn, lane direction, etc.) or the user's driving operation (brake operation, turn signal lever operation, etc.).

[0195] Figure 21 is an example of a display showing marks indicating areas to look at.

[0196] If the user should look at the lower right part of the image displayed in the image display area 141, but the user is not looking at it (for example, the user is looking straight ahead), a mark 144 is displayed in the image display area 141 as the lower right part is the part to look at (this is an example of processing to display the part the user should look at in the image). The CPU 90 determines where the user should look by analyzing the displayed image or reading the recorded event data (e.g., operation of the turn signal lever, sudden braking, etc.) indicating an event at the time of recording of the image, when the header recording area of ​​the image file is recorded. For example, when the turn signal lever is operated to change lanes from the left lane to the right lane, if the user is not looking at the right side mirror, a mark 145 is displayed as the right side mirror as the part to look at. Also, when a predetermined place such as a right turn or left turn needs to be confirmed, such a place may be displayed as the place to look when the user is not looking at it.

[0197] Furthermore, when the user is not looking at a location that he or she should be looking at, the video display in the video display area 141 may be temporarily stopped, and an image (still image) of that scene may be displayed in the video display area 141. In a video, a scene in which the user is not looking at a location that he or she should be looking at ends in an instant, whereas a stopped image is displayed in the video display area 141, so that the user's attention can be more easily drawn.

[0198] Furthermore, for image portions in which there is no (little) change in the subject's movement in the image displayed in the image display area 141 (YES in step 120 in FIG. 12), the CPU 90 relatively speeds up the display speed (step 121 in FIG. 12). For image portions in which there is (large) change in the subject's movement in the image (NO in step 120 in FIG. 12), the display speed is not relatively speeded up. Whether there is (little) or large change in the subject's movement can be determined by comparing the differences between the frames of the image. Image portions in which there is no (little) change in the subject's movement are not considered to be important scenes, such as pedestrians jumping out or oncoming vehicles running into the road, so there is no problem in speeding up the display speed. Also, image portions in which the vehicle's position does not change are considered to be unimportant because the vehicle is stopped and therefore unlikely to cause an accident. For this reason, the display speed may also be speeded up for image portions in which the vehicle's position does not change.

[0199] Furthermore, the CPU 90 also judges whether the orientation of the display device 92 has been changed (step 122 in FIG. 12). If the orientation has been changed (YES in step 122 in FIG. 12), the display range of the image displayed in the image display area 141 is changed according to the orientation (step 123 in FIG. 12). If the orientation has not been changed (NO in step 122 in FIG. 12), the display range of the image is not changed according to the changed orientation. Whether the orientation of the display device 92 has been changed and to what direction it has been changed can be detected by the acceleration sensor 93. Assume that an image in the traveling direction of the vehicle (see FIG. 17) is displayed on the display screen of the display device 92 that was facing in a certain direction. When the display device 92 that was facing in a certain direction is turned to the right, an image in the rightward direction (see FIG. 17) is displayed in the image display area 141 in the same way as when the image displayed in the image display area 141 is dragged rightward. Similarly, when the display device 92, which was facing in a certain direction, is turned to the left, an image to the left (see FIG. 18) is displayed in the image display area 141, just as when the image displayed in the image display area 141 is dragged to the left. Also, when the display device 92, which was facing in a certain direction, is turned to the rear, an image to the rear (see FIG. 18) is displayed in the image display area 141. Changing the display range of the image in this way, with the traveling direction of the vehicle as the reference direction, is an example of processing of image data. It becomes possible to display a pseudo VR (virtual reality) image.

[0200] Furthermore, information regarding the vehicle status at the time the image displayed in the image display area 141 was recorded is also displayed. The CPU 90 determines whether or not information regarding the vehicle status has been recorded in the header recording area 31 of the video file (step 124 in FIG. 12). If information regarding the vehicle status has been recorded (YES in step 124 in FIG. 12), the information regarding the vehicle status is displayed in the video display area 141. If information regarding the vehicle status has not been recorded (NO in step 124 in FIG. 12), the information regarding the vehicle status is not displayed in the video display area 141.

[0201] FIG. 22 is an example in which information regarding the vehicle's status is displayed in the image display area 141. In FIG.

[0202] A speech bubble 146 in the first display mode is displayed in the video display area 141. This speech bubble 146 displays information about the vehicle's status at the recording time of the video displayed in the video display area 141. In the example shown in FIG. 22, the character string "The brakes have been applied!!" is displayed in the speech bubble 146, and it is understood that the brakes of the vehicle (sudden braking) were applied at the recording time of the video displayed in the video display area 141. Data representing information about the vehicle's status is also recorded in the header recording area 31 of the video file in association with time data, and is read from the header recording area 31. However, for example, the CPU 90 may infer the status of the vehicle from a sudden change in acceleration and display it in the speech bubble 146 as information about the vehicle's status. For example, if the acceleration of the vehicle in the longitudinal direction suddenly changes in the negative direction at a certain recording time, it is considered that the brakes have been applied suddenly, so a speech bubble similar to the speech bubble 146 shown in FIG. 22 may be displayed in the video when such acceleration occurs.

[0203] Furthermore, when the user specifies any position in the map image displayed in the map image display area 151 using the mouse (YES in step 126 of FIG. 13), data representing the specified position is sent to the server, and data indicating the latitude and longitude of that position is sent from the server to the personal computer 80. When the data indicating the latitude and longitude are received by the personal computer 80, a scene of the image captured at a position closest to that latitude and longitude is found from among the images displayed in the image display area 141, and the image from that scene is displayed in the image display area 141 (step 127 of FIG. 13). If no position is specified in the map image (NO in step 126 of FIG. 13), the processing of step 127 is skipped, and no image is displayed by specifying a position in the map image. In addition, the image of the position specified in the map image is displayed in the image display area 141, but the image displayed in the image display area 141 may be either or both of the image recorded by the drive recorder installed in the vehicle in which the user is riding and the image recorded by the drive recorder installed in the vehicle of a third party. In this case, data representing the image recorded by the drive recorder installed in the vehicle of a third party is stored in the server, and the image data recording the location of the position specified by the user is read from the server and transmitted to the personal computer 80. When a large amount of data representing the image recorded by the drive recorder installed in the vehicle of a third party is stored in the server, it is preferable to have the user select the image to be displayed on the personal computer 80 from among those images, and to display the selected image on the display screen of the display device 92.

[0204] Furthermore, the CPU 90 determines whether or not information on road-to-vehicle communication or vehicle-to-vehicle communication is recorded in the header recording area 31 of the video file at the recording time of the video displayed in the video display area 141 (step 128 in FIG. 13). If information on road-to-vehicle communication or vehicle-to-vehicle communication is recorded (YES in step 128 in FIG. 13), the information on road-to-vehicle communication or vehicle-to-vehicle communication is displayed in the video display area 141. If information on road-to-vehicle communication and vehicle-to-vehicle communication is not recorded (NO in step 128 in FIG. 13), the information on road-to-vehicle communication or vehicle-to-vehicle communication is not displayed in the video display area 141.

[0205] FIG. 23 shows an example in which information on road-to-vehicle communication (the same applies to vehicle-to-vehicle communication) is displayed in the video display area 141.

[0206] In the video display area 141, a speech bubble 147 is displayed in a second display mode, which is a mode different from the first display mode of the speech bubble 146 described above (see FIG. 22). This speech bubble 147 displays information on road-to-vehicle communication at the recording time of the video displayed in the video display area 141. In the example shown in FIG. 23, the text "There is a moving vehicle ahead on the left" is displayed in the speech bubble 147, and it is understood that there was a moving vehicle ahead of the vehicle on the left at the recording time of the video displayed in the video display area 141. Data representing information on road-to-vehicle communication and vehicle-to-vehicle communication is also recorded in the header recording area 31 of the video file in association with time data, and is read out from the header recording area 31. A user who obtains information on road-to-vehicle communication and vehicle-to-vehicle communication can confirm what kind of vehicle operation was performed (for example, whether the vehicle was decelerated) based on the information when the video was recorded.

[0207] In the playback processing procedure shown in Figure 11, the following processing is performed in the order of whether an object of attention is present outside the field of view (step 113 in Figure 11), whether the direction in which the user is facing has changed (step 113 in Figure 11), whether the user's line of sight is consistent with the direction in which the user should be looking (step 118 in Figure 12), whether there has been no change in the movement of the subject (step 120 in Figure 12), whether the orientation of the display device 92 has been changed (step 122 in Figure 12), whether there is information regarding the vehicle's situation (step 124 in Figure 12), whether a position in the map image has been specified (step 126 in Figure 13), and whether information on vehicle-to-vehicle communication or road-to-vehicle communication has been obtained (step 128 in Figure 13). However, the order of these processing may be arbitrary, and each processing may be performed in parallel.

[0208] (3) Modifications FIG. 24 shows a part of the play list display area 171 contained in the playback window 130.

[0209] In the embodiment described above, the file name of the video file displayed in the play list display area 171 uses the recording date and time of the video data, but in this embodiment, the recording location of the video data (one example of the shooting location) is used as the file name of the video file, rather than the recording date and time of the video data.

[0210] The recording location of the video data is set as the file name of the video file as follows. For example, when a video file is generated in the drive recorder 1, the latitude and longitude of the location where the video file is generated are read by the GPS receiver 22, and the latitude and longitude data are transmitted from the communication circuit 17 to the server via the Internet. The server finds the location of the latitude and longitude, receives data representing the found location via the Internet, and sets the location obtained from the received data as the file name of the video file. However, the latitude and longitude data recorded in the header recording area 31 of the video file corresponding to the recording date and time may be transmitted to the server via the Internet at the time of playback, data representing the found location at the server may be received via the Internet, and the location obtained from the received data may set the file name of the video file. In FIG. 24, the file names are displayed in the order in which they were recorded. Since the location at the time of recording is included in the file name, it is possible to know at a glance where the video file was shot.

[0211] FIG. 25 also shows a part of the play list display area 171 contained in the playback window 130.

[0212] In Figure 25, the file of the video file that contains the recording location with low occurrence frequency is The playlist display area 171 is displayed so that the names of video files that contain recording locations that occur frequently are displayed at the top (top) (this is an example of a case where display is given priority). The file names of video files that contain recording locations that occur frequently are displayed at the bottom. Recording locations that occur frequently are thought to be more likely to be recordings of everyday life, such as recordings made while commuting, whereas recording locations that occur less frequently are thought to be more likely to be recordings of non-everyday life, such as recordings made while traveling. To make it easier to play video files that were recorded in everyday life, the file names of video files that contain recording locations that occur less frequently are displayed at the top.

[0213] Also, files of the same location may be displayed together in the play list display area 171. This makes it easier to distinguish between video files of the same location and video files of different locations. Furthermore, the file name of the video file may include the location at the time of recording, and the file name of the video file may include the recording date and time in addition to the location at the time of recording. Furthermore, a map image of the location corresponding to the video file name displayed in the play list display area 171 may be displayed, for example, in the video display area 141. Since a map is displayed rather than only the name being displayed, it is easier to understand which location the video file is located at. For example, the file name of the video file including the location is transmitted from the communication circuit 98 to a map server (not shown) on the Internet, and the map data transmitted from the map server is received by the personal computer 80, and the map image is displayed in the video display area 141. By selecting the map image displayed in the video display area 141, playback of the corresponding video file may be started.

[0214] FIG. 26 is an example of a frame structure based on MPEG (Moving Picture Experts Group).

[0215] In Fig. 26, "I" indicates an I frame, and "P" indicates a P frame. One I frame is followed by three P frames, and these one I frame and three P frames make up a GOP (Group Of Picture).

[0216] When calculating the hash value as shown in step 77 of Fig. 8, the unit for dividing the video data is the portion immediately before the P frame, not the portion immediately before the I frame, and the controller 10 calculates a hash value for each unit. For example, the controller 10 calculates a hash value for each unit consisting of a P frame, an I frame, and three P frames following the I frame. The hash value calculated in this manner is transmitted to the blockchain system.

[0217] Furthermore, rather than calculating a hash value from video data, when various events (e.g., sudden braking, skidding, etc.) occur in the drive recorder, the controller 10 may calculate a hash value (an example of a second calculated value that is uniquely determined from the event data) from data representing the event, and the calculated hash value may be transmitted to the blockchain system.

[0218] FIG. 27 is a flowchart showing an example of a recording process procedure of the drive recorder 1.

[0219] In this embodiment, when a driver (e.g., a professional driver) is called up, the driver's behavior at the time of roll call is filmed and recorded. The recorded video data is stored in a video file, transmitted to a server for each driver, and stored in the server. The drive recorder 1 accesses the server and reads the video file in which the video data at the time of the driver's roll call is stored (step 181). The controller 10 of the drive recorder 1 records the video file at the time of roll call in the memory card 25 attached to the drive recorder 1 (step 182).

[0220] Next, the driver's name and the date and time are added to the video file that stores the video data during driving. The controller 10 generates a video file with the file name "video_file.mv" (step 183). The driver's name may be pre-stored in the memory 21 of the drive recorder 1, or may be input from the input device 20. When the engine of the vehicle to which the drive recorder 1 is attached starts, recording of video data during driving begins, and the video data continues to be stored in the video file as described above until recording ends (YES in step 185).

[0221] In this way, a video file storing video data during roll call and a video file storing video data during driving are recorded on memory card 25. When such memory card 25 is inserted into personal computer 80, the video during roll call and the video during driving can be displayed in video display area 141 of playback window 130. The video file storing the video data during roll call and the video file storing the video data during driving are associated with each other by the driver's name (which is an example of an association).

[0222] In this embodiment, in the drive recorder 1, the roll call video file and the driving video file are stored in the memory card 25, but the roll call video file and the driving video file may be stored in the memory card 25 during playback. For example, the memory card 25 in which the driving video file is stored is inserted into the personal computer 80, and the file name of the video file is displayed in the play list display area 172. When one of the video file names displayed in the play list display area 172 is selected, data representing the driver's name recorded in the header recording area 31 of the video file corresponding to the selected video file name is transmitted to the server recording the roll call video file, and the roll call video file is transmitted from the server to the personal computer 80. When the roll call video file is received by the personal computer 80, the roll call video represented by the driver's roll call video file and the driving video represented by the driver's driving video file are displayed in the video display area 141. The roll call image and the driving image are preferably displayed side by side at the same time in the image display area 141, but this is not necessarily required.

[0223] Figures 28 and 29 are flowcharts showing an example of the recording process procedure of the drive recorder 1. Figure 28 shows an example of the recording process procedure when the drive recorder 1 is attached to a truck, and Figure 29 is a flowchart showing the recording process procedure of the drive recorder 1 attached to a forklift when the driver of the truck uses a forklift to move the cargo loaded on the truck.

[0224] Referring to Fig. 28, while the truck is driving, video data is recorded by the drive recorder 1 attached to the truck onto the memory card 25 attached to the drive recorder 1 (step 191). When the recording of the video data onto the memory card 25 is completed, data representing the name of the truck driver and data representing the date and time are stored in the header recording area of ​​the video file, and the video file in which the video data is recorded in the video recording area of ​​the video file is transmitted from the drive recorder 1 to the server (step 192). Data representing the driver's name is pre-stored in the memory 21 of the drive recorder 1. Furthermore, a command to associate the video file from the forklift with the video file is transmitted from the drive recorder to the server (step 193). For example, a command is sent from the drive recorder attached to the truck to the server to associate the video file sent from the same location (the location where the truck was stopped) represented by the video data stored in the video file sent from the truck's drive recorder 1, with the video file storing the video data that was started to be recorded by the drive recorder attached to the forklift within a few minutes of the time the truck was stopped.

[0225] Referring to FIG. 29, when the drive recorder 1 attached to the forklift detects that the driver is approaching the forklift, shooting begins using the front camera 11 and rear camera 12 included in the drive recorder 1 (step 201).

[0226] When the video captured by the front camera 11 or rear camera 12 is analyzed and it is detected that the driver has got on the forklift and the forklift has started to operate, or that the driver has got on the forklift (YES in step 202), recording of the video data onto the memory card 25 mounted in the drive recorder 1 attached to the forklift starts (step 203). When recording ends, the video file is sent to the server (step 205).

[0227] In the server, if there are two video files whose recording end time and recording start time are almost the same and whose recording end location and recording start location are almost the same, those video files are treated as, for example, a video file of the same driver switching from a truck to a forklift, as described above. By transmitting the video files representing a series of videos to the personal computer 80, the video of the truck driving and the video of the forklift driving can be displayed in the video display area 141 of the playback window 130.

[0228] In the above embodiment, a truck is driven and then a forklift is driven, but the reverse can be done by driving a forklift and then a truck, and video files for each can be obtained. Also, a series of video files can be obtained in which a forklift is driven to load cargo onto a truck, the truck is then driven to move to a destination, and the cargo is unloaded from the truck at the destination using a forklift.

[0229] FIG. 30 is a flowchart showing an example of a recording process procedure of the drive recorder 1 attached to a vehicle.

[0230] The drive recorder 1 is attached to the upper inside of the vehicle's windshield in a position where it can capture the vehicle's speedometer. When the vehicle's engine starts, the front camera 11 and rear camera 12 start capturing images, and the rear camera 12 captures the vehicle's speedometer (step 211). The image of the vehicle's speedometer is analyzed by the controller 10, and it is determined whether the vehicle's speed is, for example, 30 km / h or more (this is an example of a speed above a certain level, but other speeds may also be used) (step 212). When the vehicle speed is 30 km / h or more (YES in step 212), recording of the image data to the memory card 25 starts (step 213). When the vehicle speed is less than 30 km / h (YES in step 214), recording of the image data to the memory card 25 stops (step 215).

[0231] In this way, when the vehicle speed is 30km / h or more, the captured image data is recorded on the memory card 25, and when the speed is less than 30km / h, the recording of the captured image data on the memory card 25 is stopped. When the vehicle is driven at a certain speed or more, the image data obtained by shooting is recorded on the memory card 25. In this embodiment, when the vehicle speed is 30km / h or more, the captured image data is recorded on the memory card 25, and when the speed is less than 30km / h, the recording of the captured image data on the memory card 25 is stopped. When the vehicle speed is 30km / h or more, the captured image data is recorded on the memory card 25, but even when the speed is less than 30km / h, the recording of the captured image data on the memory card 25 may not be stopped, or the captured image data may be recorded on the memory card 25 regardless of the vehicle speed, and when the speed is less than 30km / h, the recording of the captured image data on the memory card 25 may be stopped.

[0232] FIG. 31 is a flowchart showing an example of a recording process procedure of a drive recorder attached to an automatic guided vehicle (automated guided vehicle).

[0233] The above-mentioned drive recorder 1 is attached to an automatic guided vehicle. A laser sensor that detects the approach of surrounding objects to the automatic guided vehicle is attached to the automatic guided vehicle.

[0234] When the engine of the automatic guided vehicle starts (the power supply of the automatic guided vehicle is turned on), the supply of power to the drive recorder 1 starts. Then, the front camera 11 and the rear camera 12 provided in the drive recorder 1 start taking pictures (step 221). When the laser sensor detects the approach of an object (an example of a surrounding object, which may be another automatic guided vehicle, etc.) to the automatic guided vehicle (YES in step 222), recording of video data to the memory card 25 attached to the drive recorder 1 starts (step 223). When the laser sensor does not detect the approach of a surrounding object to the automatic guided vehicle (NO in step 222), recording of video data to the memory card 25 does not start. Even if a surrounding object accidentally comes into contact with the automatic guided vehicle, the situation can be recorded. In this way, vehicles include not only those with a driver but also those without a driver.

[0235] FIG. 32 is a flowchart showing an example of a processing procedure for driving evaluation.

[0236] A user drives a vehicle equipped with the drive recorder 1, removes the memory card 25 on which the video data is recorded from the drive recorder 1, and inserts it into the personal computer 80. The playback window 130 shown in FIG. 14 is displayed on the display screen of the display device 92, and the video data recorded on the memory card 25 is played back (step 231). If there is movement in the subject of the video being played back (YES in step 232), the video is analyzed, and a driving evaluation is performed from the video (step 233). For example, if the subject is moving and the brakes are applied suddenly, or if pedestrians, bicycles, motorcycles, automobiles, or other vehicles are too close, the driving evaluation is lowered. Whether the brakes are applied suddenly can be determined from the acceleration data, and whether pedestrians or other vehicles are too close can be determined from the relative size of the images of pedestrians and other vehicles in the video. If there is no movement in the subject of the video (NO in step 232), no driving evaluation is performed for that part of the video. The processes of steps 232 and 233 are repeated until the video ends. In video parts where there is no subject movement, the vehicle is considered to be stopped, so there is little point in evaluating the driving. Since driving evaluation is not performed in such places, there is a high possibility that unnecessary evaluations will not be performed. Although driving evaluation is performed when there is subject movement and not performed when there is no subject movement, it is also possible to evaluate driving when there is vehicle movement and not perform driving evaluation when there is no vehicle movement.

[0237] FIG. 33 is a flowchart showing an example of a playback process procedure for video data recorded by the drive recorder 1.

[0238] In this embodiment, the drive recorder 1 is installed in a vehicle that transports passengers, such as a bus, taxi, hire car, or train, but it does not necessarily have to be installed in such a vehicle.

[0239] The memory card 25 that was attached to the drive recorder 1 is removed from the drive recorder 1, and is attached to the personal computer 80. The playback window 130 shown in Fig. 14 is displayed on the display screen of the display device 14 of the personal computer 80, and the video data recorded on the memory card 25 is played back (step 241).

[0240] The video is analyzed by the CPU 90, and the CPU 90 determines whether or not a customer is included in the video (step 242). For example, if there is anyone sitting in the vehicle seat except the driver, it can be determined that a customer is present. If a customer is included in the video (YES in step 242), the video is displayed in the video display area 141 (step 243). If a customer is not included in the video (NO in step 242), the video display in the video display area 141 is stopped. The processing of steps 242 and 243 is repeated until the video ends (NO in step 244). In the above embodiment, if a customer is not included in the video, the video display in the video display area 141 is stopped, but the video may be displayed at a higher speed or the video may be skipped to the part of the video that includes the customer.

[0241] Fig. 34 is a flow chart showing an example of a playback process procedure for recorded video data. It is preferable that the video data is recorded by the drive recorder 1, but it may be recorded by other recording devices. This playback process is to find a desired scene from among the videos.

[0242] The memory card 25 on which the video data is recorded is inserted into the personal computer 80, and the playback window 130 is displayed on the display screen of the display device 92. When the user clicks on the word "Tools" included in the menu bar 131 of the playback window 130, a pull-down menu including a "Search" item appears. When the "Search" item is specified from the pull-down menu, a pop-up window for inputting a keyword appears on the playback window 130. In the pop-up window, the user inputs a keyword representing the situation of the video portion he or she wishes to find using the input device 96 (step 251). The video data is analyzed by the CPU 90, and a video of a scene related to the input keyword is found (step 258). The video of the found scene is displayed in the video display area 141 (step 254). The keyword representing the situation of the video portion input by the user may be not only clear, such as a video containing a bullet train, but also ambiguous, such as a bright video. The video portion corresponding to the ambiguous keyword is found. For example, if the video is a bright video, a video portion with a relatively high brightness is found. Also, a keyword that represents the vehicle's condition when the video data was recorded may be input, and the video of that condition may be found. For example, the condition of the vehicle in which the drive recorder that recorded the video data was installed, such as speeding up, accelerating rapidly, or turning on the blinker, may be used as a keyword to find the video portion that corresponds to the keyword. Data representing the vehicle's condition is recorded together with the recording time in the header recording area 31 of the video file in which the video data is stored, and the video portion may be found using that data. Also, a keyword that searches for video after dynamic movement, such as finding the video portion after a red car has passed a blue car, may be input, and the corresponding video portion may be found.

[0243] FIG. 35 shows a flowchart illustrating an example of a playback process procedure for video data recorded by the tribe recorder 1.

[0244] The memory card 25 that was attached to the drive recorder 1 is removed from the drive recorder 1 and attached to the personal computer 80. A playback window 130 shown in FIG. 14 is displayed on the display screen of the display device 92 of the personal computer 80.

[0245] Data indicating the latitude and longitude of the vehicle's movement position recorded in the header recording area 31 of the video file stored in the memory card 25 is read, and the range of movement of the vehicle is detected by the CPU 90 of the personal computer 80 (step 261). A request command for image data of a map including the detected movement range is sent from the personal computer 80 to the server, and the map image data is sent from the server to the personal computer 80. When the map image data is received by the personal computer 80, an image of the map represented by the received image data is displayed in the video display area 141 (step 262). Furthermore, data representing the vehicle operation information at the time of recording of the video data is read from the header recording area 31 of the video file, and the vehicle operation status performed at each position is displayed on the map image displayed in the video display area 141 (step 264).

[0246] FIG. 35 is an example of a map image displayed in the video display area 141.

[0247] In the map image, the vehicle's movement status is indicated by a dashed line 148. This dashed line 148 representing the movement status is obtained from data indicating the vehicle's movement position recorded in the header recording area 31. Furthermore, in the movement status indicated by the dashed line, the vehicle operation status is displayed by a speech bubble 146. By looking at the map image, the user can relatively easily grasp the range of the vehicle's movement, as well as what location and what type of vehicle operation was performed during that movement.

[0248] 37 is a flowchart showing an example of a playback process procedure for video data recorded by the drive recorder 1. In this embodiment, a video obtained by driving the user and a video obtained by driving a third party are displayed in the video display area 141.

[0249] The memory card 25 installed in the vehicle that the user has been driving is removed, and the removed memory card 25 is installed in the personal computer 80. The video data obtained by the user driving the vehicle is read from the memory card 25 and played back (step 271). In addition, data (data indicating latitude and longitude) indicating the recording location (vehicle movement range) of the video data obtained by the user's driving is read from the header recording area 31 of the video file, and is sent to a server in which a large number of video files obtained by a third party's driving are stored. In the server, a video file that stores video data obtained by driving the same place as the user's driving place is found, and sent to the personal computer 80. It is sufficient to find a video file of a third party that does not completely match the user's moving direction and movement range, but is approximately the same. The found video file of the third party is received by the personal computer 80 (step 273), and the video obtained by the user's driving and the video obtained by the third party's driving are displayed in the video display area 141 (step 274).

[0250] FIG. 38 is an example of the playback window 130.

[0251] The video display area 141 is divided into two video display areas 141A and 141B on the left and right (or above and below). The video display area 141A on the left displays a video obtained by the user's driving (an example of a video based on the processed video data), and the video display area 141B on the right displays a video obtained by a third party's driving represented by a video file found in the server. In the video display area 141A, a balloon 146A is displayed showing the vehicle operation information by the user when the video displayed in the video display area 141A was recorded. It goes without saying that the operation information displayed in the balloon 146A is obtained from the acceleration data and the like recorded in the header recording area 31 of the video file in which the video data is stored. Similarly, the balloon 146B showing the vehicle operation information by the third party when the video displayed in the video display area 141B was recorded is also displayed in the video display area 141B. It is preferable that playback control is performed by CPU 90 so that the display of the images is synchronized so that the position indicated by the image displayed in image display area 141A and the position indicated by the image displayed in image display area 141B approximately coincide with each other.

[0252] A video of the user's driving is displayed in the video display area 141A and a video of the user's driving is displayed in the video display area 141B. Since the user can compare his / her driving with the video of a third party, he / she can improve his / her driving skills by comparing his / her driving with the third party's driving. In particular, the speech bubbles 146A and 146B show the similarities and differences in the operations, so that the user can know whether his / her driving is consistent with or different from that of others.

[0253] Furthermore, the user's driving may be compared with that of a third party, and information showing an evaluation of the user's driving (which is an example of information based on a comparison between the third party's driving and the user's driving) may be displayed in the video display area 141. For example, if the user's driving skills are evaluated as being worse than the third party's driving, warning information such as "Take the third party's driving as a reference and improve your driving skills" may be displayed, and if the user's driving skills are evaluated as being better than the third party's driving, information showing a high evaluation such as "Your driving skills are considered to be high" may be displayed.

[0254] FIG. 39 is an example of the playback window 130.

[0255] Alphabetical characters indicating the places and the order in which the driver of the vehicle should look when recording the image displayed in the image display area 141 are superimposed on the image. The example shown in FIG. 39 is, for example, an image when changing lanes, and it is preferable to look at the places indicated by the alphabet in alphabetical order. When recording the image data, the driver's line of sight is detected as described above, and the CPU 90 detects whether the driver's line of sight moves to the places indicated by the alphabet in alphabetical order. If the driver's line of sight moves to the places indicated by the alphabet in alphabetical order, for example, a character string saying "Ideal driving" is displayed in the image display area 141. The user can confirm that the driving skill is not bad. If the driver's line of sight does not move to the places indicated by the alphabet, a character string saying "There are places that have not been checked" is displayed in the image display area 141 to inform the user that there are places that have not been checked. If the driver's line of sight does not move to the places indicated by the alphabet, a character string saying "The checking order is not ideal" is displayed in the image display area 141 to inform the user that the checking order is not good. Users can know whether their eye movements while driving are appropriate.

[0256] Furthermore, in the example shown in FIG. 19, a processing process is performed to blur the area 142 outside the field of view, but it is also possible to detect an area that the user is not looking at within a certain period of time and blur the area. For example, it is also possible to detect an area that the user is not looking at between the start of a right or left turn and the end of the right or left turn and blur the area. It goes without saying that the line of sight of the user, who is the driver, is detected when recording the video data, and the area that the user is not looking at is detected using the line of sight data. For example, when an area that is not being looked at is detected, the area that is not being looked at is blurred or made semi-transparent in the same way as the area 142 outside the field of view as shown in FIG. 19.

[0257] Furthermore, when playing back video data recorded by the drive recorder 1, the CPU 90 may perform correction (one example of processing) so that the color matches the driver's memory. In order to correct the color to match the memory color, the contrast, tone, and saturation of the obtained image are adjusted, and the degree to which each adjustment is made may be determined in advance, or several preset adjustment values ​​may be determined and selected during playback.

[0258] Furthermore, a communication robot may be used together with the personal computer 80 when playing back video.

[0259] FIG. 40 is an example of a communication robot 280 placed near a personal computer 80.

[0260] The communication robot 280 includes a body 281 and a head 282. An eye camera is provided inside the eye 283 of the head 282, and a speaker is provided in the mouth 284. The head 282 can rotate freely clockwise and counterclockwise with respect to the body 281. The communication robot 280 is placed on the right side of the personal computer 80. The user sits in front of the personal computer 80. In this manner, the positions of the personal computer 80, the communication robot 280, and the user are determined in advance. The communication robot 280 is controlled so that it faces the display device 92 when it displays the playback window 130 on the display screen of the display device 92 of the personal computer 80 and outputs a comment on the video displayed in the video display area 141 as audio, and faces the user when it outputs a comment to the user viewing the video as audio.

[0261] FIG. 41 is a block diagram showing the electrical configuration of the communication robot 280.

[0262] The communication robot 280 includes a CPU 291 that controls the overall operation, a memory 292 that stores data, etc., a wireless communication circuit 294 that communicates with the personal computer 80, the drive recorder 1, etc., and a wireless LAN (local area network) board 293 that is connected to the wireless communication circuit 294. The communication robot 280 also includes a head motor 296 that rotates the head 282, an eye camera 298, a speaker 300, and a microphone 301. The head motor 296 is controlled by a motor control board 295, and video data captured by the eye camera 298 is taken into the communication robot 280 by a sensor board 297. Furthermore, the audio output from the speaker 300 and the audio input from the microphone 301 are controlled by an audio input / output board 299.

[0263] 42 is a flow chart showing the processing procedure of the communication robot 280. It goes without saying that the operation program of the communication robot 280 is stored in the memory 292.

[0264] The above-mentioned playback window 130 is displayed on the display screen of the display device 92 of the personal computer 80, and the video recorded by the drive recorder 1 is displayed in the video display area of ​​the playback window 130. The personal computer 80 and the communication robot 280 communicate with each other, and acceleration data is transmitted from the personal computer 80 to the communication robot 280 (data other than acceleration data, data recorded in the header recording area 31 of the video file, event data, etc. may also be transmitted from the personal computer 80 to the communication robot 280. It is sufficient that data that enables the communication robot 280 to give advice to the user, etc. after viewing the video is transmitted from the personal computer 80 to the communication robot 280).

[0265] Based on the acceleration data transmitted to the communication robot 280, when the CPU 291 of the communication robot 280 determines that there has been a sudden change in acceleration in the forward and backward directions of the vehicle to which the drive recorder 1 that recorded the displayed image is attached (step 311), it is considered that the vehicle has suddenly braked. Then, as shown in FIG. 40, the head 282 of the communication robot 280, which was facing the user, is rotated 90 degrees clockwise (step 312) and faces the display device 92 of the personal computer 80. The image of the scene where the sudden brakes were applied is displayed on the display screen of the display device 92, and when the communication robot 280 faces the display device 92, a voice is output from the speaker 300 to the effect that the driving is dangerous (step 313). For example, a voice saying "This kind of driving is dangerous" is output. The voice data is stored in the memory 292.

[0266] After that, the head 282 of the communication robot 280 is moved 90 degrees counterclockwise (step 314), and the communication robot 280 faces the user. When the communication robot 280 faces the user, a voice is output from the speaker 300 (step 315) to warn the user not to drive dangerously like the scene in the video displayed on the display screen of the display device 92. For example, a voice saying "Please try to drive safely" is output. Since the warning comes from the communication robot 280, not from a human such as an instructor, the user tends to listen to the warning obediently.

[0267] FIG. 43 shows the above-mentioned communication robot 280 placed on the dashboard 311 inside a car.

[0268] As described above, the drive recorder 1 is attached to the top of the windshield 310. The drive recorder 1 and the communication robot 280 communicate with each other, and when the drive recorder 1 recognizes that the vehicle is being driven autonomously as described above, an operation start command is sent from the drive recorder 1 to the communication robot 280. The communication robot 280 starts operating and begins talking with the driver of the vehicle, etc.

[0269] Furthermore, when playing back the video data recorded in the drive recorder 1, the CPU 90 may analyze information about the road on which the vehicle to which the drive recorder 1 is attached is traveling (e.g., information such as the road narrowing or a dead-end road) from the video data, and change the display of the video displayed in the video display area 141 according to the analysis results. For example, a character string indicating the detected road information may be superimposed on the video, or some of the hue, brightness, and saturation of the entire video or part of the video may be adjusted. Since the video displayed in the video display area 141 changes, it is possible to know that there is a change in the road condition, etc.

[0270] Furthermore, trigger information for event recording may be recorded in the header recording area 31 of the video file. For example, information on the operation of the turn signal, the operation of the sudden brake, the operation of the windshield wipers, etc. may be recorded as trigger information for event recording. Furthermore, a user may sit in the passenger seat during recording by the drive recorder 1 and wear a glasses-type display device on the user's head. The display screen of the display device displays information to be careful about when driving, and the view ahead of the vehicle being driven (an example of the actual view seen from a moving vehicle) is visible through the display screen. The information to be careful about when driving is displayed superimposed on the view ahead of the vehicle. It goes without saying that the road to be traveled is determined in advance, and the information to be careful about on that road is displayed on the display device. Also, a third party other than the user (for example, a driving instructor) may ride in the vehicle and select the information to be careful about on the display device, and the selected information to be careful about may be displayed on the user's display device.

[0271] In the above embodiment, the drive recorder 1 is capable of recording in all directions, including the front, sides, and rear of the vehicle, but it is not necessarily required to be capable of recording in all directions.

[0272] Furthermore, the recording in the drive recorder 1 may be performed in addition to the normal continuous recording and event recording. For example, in the case of event recording, the acceleration sensor and trigger switch are used to record before and after. This is kept as an event record separate from the continuous recording. It is also possible to record 30 seconds (1 minute) before and after the acceleration sensor reacts. For example, it is also possible to record a case where a first event occurs, a second event occurs, and the car hits a guard rail the first time, which causes the car to swing and hit the guard rail a second time. If the recording times overlap, they may be treated as one event. If there is a gap between two events, If there is free time, you can record it as a separate event, and if there is no overlap, you can process it as two events.

[0273] In addition, the above-mentioned drive recorder 1 may be used as a camera for constructing, monitoring, and adjusting a line in a factory to ensure that the factory line runs smoothly. When assembling with a machine tool, a camera may be temporarily installed in a place where a problem is likely to occur in order to capture the assembly. When a new line is launched in a factory, a camera may be temporarily installed for about three months to verify defects. The camera may record the time before and after an alert is issued by a machine tool in the factory, or it may capture the time before and after a trigger such as a light (signal light) on the machine tool that is green, yellow, or red. Video data obtained from such a camera may be displayed in the above-mentioned playback window 130. It may be possible to capture only events, or it may be possible to capture images all the time.

[0274] Even with a spherical drive recorder (a drive recorder that shoots in all directions) or a drive recorder with a wide-angle camera, the range that a person can see is not the same as the imaging range of the camera, and the range that can be seen varies depending on the age of the person and individual differences. Since a narrow viewing angle is dangerous during senior driver's training or license renewal, and people often run over bumps, it is a good idea to record the video of the driver driving using a drive recorder 1 and show the driver running over a bump while giving training. By displaying the visible range of the bow seat, it is possible to conduct training that does not deny the driver, but asks them to look carefully if they cannot see, or to shake their head so that they can see. For example, it is a good idea to display a frame so that the visible range can be seen when playing back.

[0275] When the driving situation (scenery, scenery) is recorded by the drive recorder 1, the driver's eye movements may be recorded at the same time, and when the field of view is shifted, a vector or circle indicating where the driver is looking within the field of view may be displayed. It is recommended to record where the driver's eyes were looking while driving, and when displaying the display in the playback window 130, the display range or frame may be shifted in the direction of the driver's eyes. If the drive recorder 1 is capable of capturing a full celestial sphere (all directions), the front, sides, and rear of the vehicle are all recorded, so the direction the driver is looking can be known by image recognition. It is recommended to move the display frame accordingly. It is recommended to display on the playback window 130 not only the present, but also the extent to which the driver was checking in the present, past, and future, and that the driver was looking straight ahead now but not in the past. For example, it is recommended to fill in the range that the driver was not looking at. It is possible to see the range that the driver was not looking at at all. It is possible to see how far the driver is turning his head, his line of sight, and how far the driver is looking. It is also possible to connect the drive recorder 1 to a line of sight detection sensor and display the line of sight captured by the sensor. It is possible to know where the user is not looking. The imaging time changes depending on normal driving, high-speed driving, driving at an intersection, etc. Driving time at a driving school, before and after stopping at an intersection, etc. The range of viewing is predetermined at each point in the driving school, and the degree to which it matches is judged. It is fine if there is a standard, such as a certain percentage of viewing to pass. Even in normal driving, the amount of viewing can be judged according to the speed, such as how much the user looks when stopping at an intersection, the speed, 40km / h to 80km / h, 80km / h or more. When driving at high speed, you don't look to the side, but you look at the rearview mirror, and in the city, you have to be careful about the side as well. When driving slowly at 20km / h or less, you have to look 360 degrees, and you have to drive while checking the mirrors and blind spots. The range of viewing changes based on the turn signal, brake, and steering angle. If you are trying to turn right, you have to look to the right relatively accurately. By using the playback window 130, it is possible to know, for example, that the user is checking before turning the turn signal or before turning the steering wheel. Furthermore, during driving lessons for elderly people, it is a good idea to have an elderly person actually drive in the driving school, turn on a driving recorder, and later show in the playback window 130 how they drove.A simulator is fine, but seniors won't be convinced unless they actually drive the car, so it's best to show them an actual video. This has two effects: it helps the person understand, and it makes it easier for the instructor to understand. It's a good idea to provide information for the instructor as well. The areas that are not seen are blacked out, so think of this as information for the instructor. You can also check whether the driver is looking at the instrument (speedometer, etc.) by recording the driving situation when the driver puts on the drunk driving glasses with a drive recorder 1 and show the situation to the driver.

[0276] A bus has over a dozen mirrors, so in addition to the tachograph, it is possible to visually evaluate whether the driver is looking at them all. By displaying the driver's line of sight, it is possible to evaluate what percentage of the time the driver is looking at the mirror. It is possible to evaluate what the driver is looking at and what he is not looking at before and after turning on the blinker. Buses and commercial vehicles always have routines, and there are manuals that must be checked at bus stops and before departing. Even forklifts require checking in front, behind, left and right, so it is possible to evaluate whether the driver is following the normal routines for buses, trains, trucks and forklifts. The movements of passengers can also be shown.

[0277] If you record the history of what you are looking at and where you are looking while the video is being played back, you can use the moment when you turn on the blinker as a trigger, and analyze the time before and after that to see if you are looking at blind spots properly. In the case of driving, you need to look at the right place at the right time, so this is different from normal gaze analysis. In the video, the car is moving, the trigger, the blinker, and the place to look is fixed before turning the steering wheel. Things like conditions change over time. The timing of looking may be late. It is important that it corresponds to the time and is appropriate. There is a place to look at. It shows where you should look and shows a guide-like example of whether the user was looking at the bottom. If you are approaching an intersection, look here, if you are turning, look here. The type of trigger, what to use as the trigger, speed, blinker, the location of the intersection, red light, road sign, if it is a crosswalk, are you checking right and left, are you looking behind when you put it in reverse, do you look ahead and not back up. Check by looking at the mirror. Only looking at the rearview monitor. Looking left and right. Are you checking such places, backing up is strange even though your eyes are facing forward. Driving is strange even though you are looking behind. From the video of the driver, gaze analysis and direction can be understood. It is also possible to detect whether the driver's eyes are open, and display information on whether the eyes are open during playback. Scenes of poor driving may be displayed as thumbnails like in a movie film. For example, there are things that need to be done in a given action, such as not looking at the part that should be looked at, not looking at the mirror when changing lanes, not operating the turn signal or accelerator, etc., so the parts that do not match can be cut out and displayed as thumbnails.

[0278] When scoring driving, the times when the vehicle is stopped can be excluded from the scoring. The score can be based on whether the vehicle moved smoothly, whether it stopped, and whether there were any sudden turns or sudden braking. The points that are scored are when buildings are moving in the footage, when the car in front is not moving, and when each part is moving and to what extent. When the vehicle is stopped, it is not scored. The points where the vehicle is stopped can be used for scoring. The points where the vehicle is stopped can be skipped and the footage can be fast-forwarded to the next part. The footage can be advanced to the part where the light changes from red to green. This can save time when the scorer views the footage. Not only footage of the outside, but also footage of the inside can be displayed when and only when a passenger gets in. If the footage is played only when there is a change in the passenger, it can be a special feature on when a passenger gets in. This is because in the case of a taxi, it is sufficient to know the passenger's face.

[0279] Also, when searching, if you say it was definitely bright, you will find bright footage, and if there was a Shinkansen, you will find footage of the Shinkansen, and you can pick out the footage of the Shinkansen. In this way, it is a good idea to start from a general overview and then pick out footage that meets your criteria. When you have footage from a week, you can start looking for it when someone says something vague. You can also find out what was around there when you want to show it in a presentation. You can also search for footage of a supermarket parking lot with lighting conditions you want to show. It is good to use keys such as whether it is a specific object, the situation, the time, the condition of the car that was recorded, whether it was speeding, accelerating suddenly, turning on the blinker, etc. After a red car passes a blue car, Find the footage after the dynamic movement. It is a good idea to find the desired scene after this sequence in the video.

[0280] The actual image can be displayed at the top and the recognized image at the bottom, or the car model can be identified by the image or shown by its outline. From the ambiguity of such scenes, candidates for the desired video scene can be displayed and the user can choose. Each image can be tagged with what happened here, and for how many minutes it was there. If it stops for more than five minutes, a voice recognition function can be added to the software that plays the video.

[0281] During playback, it is good to display where the driver was looking. When it is determined that the driver is looking away, a mark may be added to the playback window 130, or the image may turn red the moment the driver looks away. This is because the image is captured by a spherical camera, and it is possible to know where the driver is looking. If the driver is looking away, an alarm may sound. If the driver has his eyes closed, an alarm may sound. It is good to skip that part and play it, or play only that part. If the camera is not a wearable camera such as a 360-degree camera, it is possible to know the direction in which the user was looking, so that direction can be displayed. A wearable camera is also good. Instead of a wearable camera, a camera may be attached to the user by being placed on the head or using something like a headband, like a wearable sensor. It is good to have a sensor that knows where the driver is looking. An acceleration sensor (camera) may be attached to the ear, or a sensor (camera) may be attached to a woman's hair clip. A sensor (camera) may be attached to the brim of a hat with a clip. These sensors may be combined with a spherical image to show which direction the vehicle is facing. Analysis can be done with a spherical image. It may be possible to show the opposite side and inform the driver that there is a frame area indicating the field of view on the back side. It may be possible to inform the driver that the frame is being displayed but is not included in what is currently being seen. The outer frame may be made green, or the visible range may be clearly seen and the rest may be gray. It may be possible for the clear area to expand as the driver checks. The range that is not being viewed can be seen at a glance. The range may be cleared during the time when the head is not moving. It may be possible to display the visible range and the invisible range only before and after the steering wheel is turned, or before and after the turn signal is turned on. The angle at which the camera is facing may be displayed on the map below. Both the angle of the line of sight (eye line) and the angle of the car may be displayed. The angle of the image itself may be displayed. It may be possible to show how much is visible by the direction of the car and the direction of the line of sight. It may be possible to display the angle of the current view, or the angle of the displayed area.

[0282] It may be possible to display VR (virtual reality) in a drive recorder with a spherical camera. The image of a spherical camera is taken in 360 degrees, so an image like VR on a plane can be obtained. In this case, the reference direction may be based on the front camera side, or the direction of shooting, or the north side. It is necessary to determine how much to map the reference direction to the range of the VR viewing angle with the direction of north as the reference direction. In the case of the drive recorder 1, the forward direction is the front. It may be a playback device that shows the direction in which the main body is moved. This range is the range that you can see, but it can be shown that someone 10 years younger can see more. It may be equipped with a function that allows comparison. It is possible to see that what you can see now will not be as visible in 10 years or 30 years. It may be equipped with a function that allows the instructor to switch the angle range by age. It may be possible to switch by age. VR may be displayed based on the direction the driver can see. It may be possible to provide playback software for the images recorded by the drive recorder 1 that can be displayed on a VR device. With a VR device, acceleration and the like can be displayed as an overlay, or on an OSD (on-screen display). Acceleration and the like can be displayed in 3D. The image itself cannot be 3D, but it can be displayed semi-transparently in front of the visible surface of the sphere. It is also possible to have a panel appear in an unnecessary area. You can tell just by moving your neck. The direction of your gaze should be in front, and the overlay-like thing should be in front. It can get darker depending on how long you are looking at it. Alternatively, it may be possible to have darker and lighter areas depending on how often the user looks at the screen. If the user turns sideways, the image to the side can be seen, and if the user turns upwards, the image to the top can be seen. The direction the user is facing can be detected using an acceleration sensor, geomagnetic sensor, or gyro. Straight ahead means that the camera is fixed, so the direction ahead can be determined from this. The center of the sphere can be considered to be forward. The camera always faces forward. Image data can be provided to the framework.

[0283] From the perspective of the administrator, there is no point in watching this footage when the image is not changing, such as when waiting at a traffic light or the vehicle is stopped, so it would be a good idea to have the playback software (program) automatically recognize this and give it the ability to skip over it.

[0284] The vehicle's trajectory can be displayed on a map, and when a point on the trajectory is specified, a function to jump to that point can be provided. The time of the image at a certain point on the map is written corresponding to the point on the map, and the time is displayed when the cursor is moved, so the image at that time can be found. Street view can be displayed. A function to switch between Street view images and images recorded by a dashcam can be provided. Street view has past and present images, so you can see what the scenery was like a year ago. Images from three years ago of the area you drove through can be viewed. Images before and after the buildings were built can be viewed. It is good to display them in a separate window. It is good to be able to switch to older maps as well. If the date of the image is included, you can display images of the desired season, such as autumn images or spring images. If you are driving through the same place on your commute, summer and winter images can be displayed. It may be possible to display the same image as the area you are currently driving through. It may be possible to display what the area looked like on a rainy day.

[0285] In the playlist display area 172 of the playback window 130, information may be displayed that this video is a video that moved from this area to this area. For example, information that the video went from Okazaki City to Toyota City may be displayed. This may be displayed in text or as a reduced map display. This becomes very easy to understand and is easy to use. If there is city, town, village and area data, it is possible to display from XX City to XX City. If you use a location API (Application Programming Interface) to send latitude and longitude to a search server, information that the location is XX City will be returned, so you can output that information. The destination of the video file can be automatically assigned. When videos are lined up, if you commute by car every day and don't care about the commuting video, if you only want to see the travel video, it is easier to select if you categorize from here to here as commuting. For the current XX City, it is a good idea to categorize the places where the starting point and the arrival point are the same and move the rare ones to the top. If you have only been to a place once, it is considered that it is likely to be a trip.

[0286] GPS data is stored periodically in the header recording area 31 of the video file, such as frame by frame, every second, or every 0.5 seconds. To name the video file a specific place name, for example "Higashi Okazaki," the coordinates can be sent to the API of the location search server, and the video file name can be created using the information returned from the server. GPS data can be sent to a server that stores map images, and the map can be retrieved and displayed from the server. If the personal computer 80 has map data and place name information, there is no need to send it to an external server. Event information may be displayed. Data containing only events can be displayed in the playback window 130, and information such as "when this road was passed, an event occurred in this scene, in this scene" can be displayed. Where the sudden braking, etc. was applied may be displayed on the map. If it is a road on the way to and from work, events such as where the event occurred and the change were displayed together may be displayed. The road may be displayed, and when this road was passed, this event occurred may be displayed on the map in an overlapping manner. For example, if you tap on a road on the map, such as Route 248, all the events that occurred while you were driving along Route 248 will be displayed on the map for a certain period of time, say five years, and you will be able to see the events at that location. It is clear that areas are dangerous in concentrated areas. All logs are displayed and clicking on them will display logs, heart rate, bicycles, and other categories. It is also possible to identify and compile and display areas where minor incidents occur, where people are stepping on the accelerator a lot, and other locations where accidents are likely to occur. It is also possible to categorize and display video files of commuting, and when clicked, all of them can be mapped on a map.

[0287] Also, some kind of driving evaluation may be performed for each certain section, for example from one intersection to another, and the results may be automatically saved in the server. When playing back, the evaluation data of others is also accumulated for each section from time to time, so that the evaluation of one's driving compared to others' driving, and whether one's driving is reckless or safe, may be displayed in the playback window 130 during playback. Depending on the evaluation, a flower circle or a crying face may be displayed near the bottom of the image, or a score may be displayed, or the average score of others may be displayed.

[0288] Furthermore, the video may be stored in the server for each certain section, and the same location may be compared. For example, it may be possible to compare a certain road, such as from here to here on National Route 1. Driving scores are given based on the acceleration sensor, speed sensor, gyro, and braking technique, but on the other hand, is the score really a failing grade or a passing grade for that road? On a winding road, the driver will naturally turn the steering wheel many times, so even if they say that the driver is zigzagging too much, the evaluation is not appropriate. This is because the road is winding. Instead of absolute evaluation, it may be possible to give a relative evaluation or to show images driven by other people. It may be possible to display the video of the top driver, ``You are 30th place, and here is an image of great driving.'' There is the question of whether it is okay to display the top video, but a function to allow this may be provided. It may be possible to display a message asking ``Are you OK to provide this?'', or ``Do you want to allow others to view it?'', or to add an ``upload'' button. The camera (drive recorder 1) itself may have a communication function and upload, or it may send score data. The confirmation may be made when the playback software is installed on the personal computer 80, or may be made to occur all at once during installation, or the user may be allowed to select which video files to upload one by one, or an approval button may be displayed the moment an upload is attempted. Video files may be uploaded after confirmation, but scores may be uploaded without confirmation. A rule may be established that users are not linked to information about where they were. When uploading, date data and the like may be erased. Day of the week and time may be included, but it is better to erase any information that could identify individuals. If there is only location and time, and no model or personal name, it is fine as long as it is not clear whose data it is.

[0289] Since there is a low possibility of accidents occurring on roads with nothing in them, such as farm roads, they should be photographed (recorded), but the resolution should be lowered, and since there are obstacles in town, it is better to be able to specify for each location that they should be photographed (recorded) in high resolution. It is better to be able to specify that the range from here to here should be recorded in low resolution, and this range should be recorded in high resolution. This will allow for long-term shooting. The location is specified using the playback window 130. The user specifies in advance the range from here to here that is acceptable. It is better to circle it on the map, and set the playback window 130 to shoot (record) in low resolution when this range is entered. The memory card 25 with the recording settings is set in the drive recorder 1. Then, it will move according to that. If the drive recorder 1 has an LCD display, it can also be set on the drive recorder 1. The resolution can be adjusted here even during parking monitoring. It is also better to change the resolution while driving. This is because there are times when nothing will definitely happen here. It is fine to take detailed photographs (records) using event recording. If the cloud (server) has all the data, it can tell where an event will not occur and where it will occur, so it can adjust the resolution on the cloud side. It records at low resolution (frame rate) where the probability is low, and at high resolution where the probability is high. Since it is an event switch, Sometimes it is good to use high resolution. You can record at high resolution when you are traveling and at low resolution on the way to your destination. You don't need to record when you are driving on the highway. You can record only when an event occurs (event recording). For example, if you are going to Kyoto, record in Kyoto city at high resolution and stop recording when you leave Kyoto. You can record at a low frame rate, such as one frame per second. You can set in advance the locations you want to upload to a video sharing site. For example, if you go to a circuit, you can record the circuit area in high quality and not record the journey there. You can specify areas to record at a high frame rate (high image quality) instead of areas to record at a low frame rate (low image quality). You can record at a low frame rate when you are parked using parking monitoring.

[0290] For example, if you go to Kyoto and want to record in high resolution, it is a good idea to record the journey there and back as well. If you don't record, it's meaningless as a drive recorder, and even if you enter Kyoto and try to set it, you can't set it because you're driving. For example, you can record in high resolution only near Kiyomizu-dera Temple, but you can prevent this from happening because you might forget to set it. Also, it's a good idea to record in high resolution while driving on mountain passes. You can record in high resolution on expressways and in low resolution on general roads. You can record in high resolution on the Shuto Expressway and not record on general expressways, or you can record when the speed exceeds 100 km, or when there is acceleration around a corner. You can set the area by providing a slide switch on the drive recorder 1 to switch between high resolution and low resolution.

[0291] A 760MHz ITS service system, Intelligent Road Information, has been available as an experimental service since around 2016, and it would be good if this information could be recorded together with the information from the drive recorder1. The information includes, for example, other cars in the vicinity of the vehicle (position, direction of movement, acceleration, etc.), traffic lights (how many meters ahead, time until the light turns red, etc.), road signs (how many meters ahead, speed limit, etc.), pedestrians (position, direction of movement, etc.), emergency vehicle approaching notification, sudden braking warning, traffic congestion warning, fault signal vehicle warning, sign notification, motorcycle approaching notification, traffic signal timing notification, construction information notification, weather warning, etc. ITS includes road-to-vehicle communication and vehicle-to-vehicle communication, road-to-vehicle communication is communication between the road and the vehicle, and vehicle-to-vehicle communication is communication between the vehicle and the vehicle. Road-to-vehicle communication involves communication of vehicle number information and information obtained from sensors of surrounding pedestrians. Vehicle-to-vehicle communication involves communication of information such as which direction the vehicle is going and information about the current accelerator. For example, if data between the road and the vehicle is received and saved together with the video, for example, if an LED traffic light is flashing, and the frame where the flashing stops is synchronized with the frame rate of the camera, the light of the traffic light may not be captured in the video at all, and in some cases, it may remain dark. Even if the light of the traffic light is not captured, the state of the traffic light can be recorded as data using ITS information. If the ITS information shows that the traffic light is green, the information can be displayed at the edge of the playback window 130 as the traffic light is green. Since information is beginning to be provided using a dedicated frequency, it can be received by a receiver, recorded together with the video from the drive recorder, and displayed in the playback window 130.

[0292] It is desirable to record information about ITS services in the drive recorder 1 and display that information. It is desirable for the drive recorder 1 to be able to record information such as a car approaching from nearby, a bicycle approaching, crossing the road when the light is red, crossing the road when the light is yellow, not braking in front of the yellow light, etc., in the memory card 25. Information about other cars in the vicinity of the vehicle (position, direction of movement, acceleration), how many meters away the traffic light is, how many meters away it will take to turn red, the type of road sign, and pedestrians are also detected and transmitted, so it is desirable for the drive recorder 1 to receive and record such information. When an emergency vehicle is approaching, that information is sent between vehicles in a bucket relay when the car in front brakes suddenly, so it is desirable for the drive recorder 1 to record such information in the memory card 25. It is also desirable to be able to record information about how many cars ahead brake suddenly, as it is known in advance. In the case of road-to-vehicle communication, such information is transmitted at the intersection. Information is transmitted from an antenna near the point. When the vehicle enters an area about 5 meters from the antenna, the drive recorder 1 automatically receives the information. Between vehicles, vehicles that can receive information from the road can also transmit their own information. When receiving information between the road and the vehicle, such as the current speed in kilometers per hour, or the next time to go left based on the car navigation system, or the moment the brakes were applied, the information is received by the drive recorder. The drive recorder also receives information between vehicles, and information transmitted by the neighboring car can be received because it uses the same frequency. It is best for the drive recorder to only receive, not transmit. The ITS device is a transmitter and receiver, and the drive recorder 1 records communication between a car with a drive recorder and another car. It just intercepts the information. It is also possible to install software in the drive recorder and transmit the received information to other cars.

[0293] This is useful in situations where there is a building at an intersection and a car is coming from the blind spot, and one car can communicate with the other car to see how the car is behaving and whether it is braking slightly, in order to prevent accidents. Also, cameras can be installed at intersections to inform the other car of the presence of pedestrians. Since this is a system that makes drivers be careful when turning, it is easier for the administrator to manage if they know that the driver is braking when there is a hidden car, and it would be good to record cases where the driver did not brake even though the car warned them. It would be good to be told that fuel efficiency is poor if the driver does not slow down, even though the information on the display installed in the car shows that the previous light is red and will not turn green for another 10 seconds.

[0294] In the playback window 130, information about the vehicle in which the drive recorder 1 is installed is displayed below. For example, the speed, the operation status of the turn signals, etc. In addition to such information, it would be good to display information obtained by the ITS system in a separate place.

[0295] For example, it is possible to know that there is a car a certain number of meters ahead to the right, so if there is a traffic light a certain number of meters ahead, indicating that there is a car there, a bubble can be displayed the moment the traffic light comes into view, showing the time to that light. This can be displayed as text. It can also be displayed as text or an icon. A bubble is an example of an icon.

[0296] Since the footage from the drive recorder is used as evidence that objectively shows the circumstances of the accident, it is desirable that it cannot be altered. If there is any tampering, it is desirable to be able to detect it. Also, the footage recorded by the drive recorder 1 is not a recording of the scenery that the driver remembers, but an objective recording of footage that is based on the use of the cameras (front camera 11, rear camera 12) included in the drive recorder 1 at the scene. This is because the location where the drive recorder 1 is installed is different from the position of the driver's eyes, and the recording system of the drive recorder 1 is different from the human visual system. The differences between the shooting system of a drive recorder and the imaging system of the human eye are, of course, the angle of view, sensitivity, head movement, eye movement, differences in the optical system, and differences in the image sensor and the structure of the eye, but in particular, since the cognitive structure of the brain is different, it would be a good idea to obtain an image that the driver remembers and give drive recorder 1 a function that can output an image as close to that as possible, and then educate the driver based on the differences with the objective image recorded by drive recorder 1, or to correct any distortions in the driver's cognition.

[0297] It would be good to be able to process the colors, etc., of the video recorded by the drive recorder 1 so that the video would look like what the driver had in mind. It would be good to process the video so that it would look like what the driver had in mind objectively, and display the video close to the impression the driver had.

[0298] Regarding the detection of tampering, a hash value of the data recorded when recording the video of the drive recorder 1 is calculated, and the hash value is stored in association with the video and also in a block chain on a P2P (peer-to-peer) network connected to the network. Up until now, one entity stored all the video data on a server, but with a distributed data database, everyone, the many people connected to the internet, shares the data, and each server has the same data. The blocks that are posted on the internet are then attached to the existing block chain, creating an ever-increasing chain. The data in that chain is the same on all servers, and the same data is recorded in the database. The unique feature is that since everyone has the same data, tampering can be detected unless the majority of people tamper with it.

[0299] The way data is created is that when blocks are connected, the hash value of the previous block is calculated and always included in the next block. It is not possible to tamper with one block, and in order to tamper with it, all the blocks after that must be tampered with, making tampering extremely difficult. The amount of calculation required to calculate the value of the blockchain is set to about 10 minutes for Bitcoin, and it would be impossible to recalculate all of the blocks in time. The entire flow of a contract can be recorded on the blockchain, or a contract for import and export procedures, such as the person who transports the goods in the other country, the person who makes the settlement there, the person who exchanges money, how to place an order, and various other bundles of contracts can be recorded on the chain. For example, information such as the date on which the goods passed through here and the date on which they were loaded onto the ship can be recorded on the chain, making it possible to verify all of it later. It can also be used in areas where there are conflicting interests and where there is a mechanism for verification. Also, when an advertiser buys an ad slot and contracts to air a certain number of 15-second commercials during the prime time slot between 7:00 and 8:00 in a month, it is not possible to verify whether the ad was actually aired, or if it is a transportation ad, it is not possible to verify where it was aired, and on the Internet, it is not possible to verify whether the ad was actually given that many impressions, or whether it was displayed on the other person's browser, but blockchain can be used in cases where an audit is conducted between the advertiser and the person who actually places the ad. For example, when an ad is aired on the blockchain, it is placed in the block, and the chain continues to link the block. In this way, it is possible to verify on the blockchain that a person who asked to air 1,000 times has now aired 800 times.

[0300] Video data can be recorded on the blockchain so that it cannot be tampered with. For example, if data is stored for each frame of a video based on Motion JPEG (Joint Photographic Experts Group), the hash value of that frame can be calculated, sent as a block, and then linked to the chain by the blockchain system, making it impossible to tamper with. The hash value of the entire video file, such as a certain range or one minute, can be calculated.

[0301] In MPEG-based video data, dividing the data just before an I-frame makes it easy to tamper with the data, so it is better to divide the data just before a P-frame.

[0302] In a public blockchain, anyone can join the blockchain, like Bitcoin, and can view all the data in the blocks that flow through the blockchain, but in a consortium or private blockchain, for example, it is closed within a bank association, and the blockchain can only be used among banks that are members of the bank association. For example, you can use a public blockchain system. If you send not only hash values, but also location information, such as the current location of a drive recorder, to the blockchain system, information about who is where on what date in that block will be sent. For example, when information such as "Drive recorder 901-1 is located here, in Okazaki City, Aichi Prefecture" is sent to the blockchain system, the servers that manage all the blockchains will broadcast it via P2P. Video data, etc. can be sent to the blockchain system. If you send it to someone, and there's a user who wants it, you can sell it to them.

[0303] Advertisements can be delivered in the same way. For example, if you want to send an advertisement for a certain store to a user in a specific location, Okazaki City, Aichi Prefecture, you can send the advertisement to only those who are in front of the store. Blockchain requires costs to maintain the chain, but when a transaction is made, a small amount is deducted as a fee and distributed to the user who manages the server and the user who first attached the block. The information on the currency paid can also be recorded on the blockchain. For example, if you take a photo and ask someone to take it for 100 yen, the money will be deducted from the account of the person who requested the photo for 100 yen, and about 80 yen will go to the user of the drive recorder, and 20 yen will be distributed to the person who maintains the server.

[0304] It would be good if the image could be enlarged or reduced using a remote control while it is being played. Furthermore, it would also be good if the frame could be moved using a cursor or the like, and the position of the moved frame could be stored in association with the time of the image. It would be good if the frame could be moved during playback, the position of the moved frame could be recorded, and later a video stream could be created by extracting only that part of the image based on the recorded information on the frame position.

[0305] It would be better if the visual field range could be moved manually by the user, without being automatically detected. For example, if the frame is moved using a joystick, mouse, keyboard, etc., the position where it was moved could be stored, and the moved frame could be displayed during playback. This would allow other people to recognize the range that a person could see, even though the range of vision varies depending on the level of concentration while driving, age, or eye disease. For example, a person could claim that they could see this range, or that they were looking at this range. It would also be better to make it possible to correct subjectively recognized brightness, color, and memory color. In particular, it would be better to display objective and subjective images, and to compare the image captured by the camera with the image the user thinks he or she actually saw. If two types of images of the same scene are displayed, it would be possible to explain in court that the camera actually recorded it like this, but that it looked like this to the user. In criminal liability, one is the occurrence of an objective accident or result, and the other is intent. It is important to prove whether the person hit the car intentionally or negligently. The objective image cannot be tampered with, but the subjective image shows something different from what you thought it would, and both can be seen together.

[0306] In addition, the hash values ​​of the location information and event information of the vehicle can be calculated and sent to the blockchain system. This makes it difficult to tamper with event information such as sudden deceleration or acceleration.

[0307] Furthermore, in order to adjust to subjectively perceived brightness, color, and memory color, a brightness adjustment slider may be provided on the drive recorder 1 and the brightness, color, and memory color may be adjusted by operating the slider, or a brightness adjustment button, a color adjustment button, and a memory color adjustment button may be provided on the playback window 130 and the brightness may be adjusted by operating the buttons.

[0308] Furthermore, in the case of automatic driving of a vehicle such as an automobile, data for identifying whether the drive recorder 1 is recording during manual driving in which the driver is driving the vehicle, or during automatic driving in which the vehicle is being driven automatically, may be recorded in the header recording area 31 of the video file as described above. In this case, it is preferable to also record the level of automatic driving in the header recording area 31 of the video file. When identification data indicating whether the vehicle is being driven automatically or manually and data indicating the level of automatic driving are transmitted in the CAN (Controller Area Network) of the automobile, the drive recorder 1 can determine whether the vehicle is being driven automatically or manually by receiving the data communicated in the CAN (for example, (The drive recorder 1 connects the drive recorder 1 to the CAN and receives data communicated within the CAN at the communication circuit 17), and the drive recorder 1 can record, in the header recording area 31 of the video file, identification data indicating whether the vehicle is in automatic or manual driving mode, and, if the vehicle is in automatic driving mode, data indicating the level of automatic driving. When playing back video data, the playback window 130 can display whether the video displayed in the video display area 141 is during automatic or manual driving, and, if the video is during automatic driving, the level of automatic driving can also be displayed.

[0309] The image quality may be switched between automatic driving and manual driving, and video data may be recorded in the drive recorder 1. In the case of automatic driving, the image may be recorded in relatively high image quality, and in the case of manual driving, the image may be recorded in relatively low image quality, or conversely, in the case of automatic driving, the image may be recorded in relatively low image quality, and in the case of manual driving, the image may be recorded in relatively high image quality. Furthermore, the image quality may be changed according to the level of automatic driving. The higher the level of automatic driving, the higher the image quality may be recorded, or the higher the level of automatic driving, the lower the image quality may be recorded.

[0310] Furthermore, in the case where multiple cameras (front camera 11, rear camera 12) are included like the drive recorder 1, the camera to be photographed may be switched depending on whether the vehicle is in automatic driving or manual driving. For example, during automatic driving, video data obtained only by the front camera 11 may be recorded, and during manual driving, video data obtained only by the rear camera 12 may be recorded, or vice versa. Furthermore, during playback, only the video of automatic driving may be played back and the video of manual driving may be skipped, or vice versa, the video of manual driving may be played back and the video of automatic driving may be skipped. Also, video data recorded at one of the automatic driving levels may be played back, and video data recorded at the other levels may be skipped.

[0311] Furthermore, when playing back video data, the communication robot may provide guidance on the driving technique of the played back video. If the instructor directly explains to the user receiving the guidance, the user may become angry, but if the communication robot says, "That was dangerous," the psychological burden on both the instructor and the user is reduced. The communication robot is placed near the instructor and the user who are looking at the display device 92 of the personal computer 80 that is playing back the video. For example, the communication robot should always be placed to the right of the display device 92 of the personal computer 80, and the positions where the instructor and the user sit should be determined in advance. The communication robot may be set to the right or left of the display device 92 of the personal computer 80 using the playback software (program) for the video data. Of course, the communication robot may be provided with a camera that can take 360-degree images of the communication robot's surroundings and recognize the positions of the display device 92, instructor, and user that are in the vicinity. It is sufficient to recognize people wearing uniforms as instructors and other people as users.

[0312] Furthermore, the road width included in the image captured by the drive recorder 1 or the image represented by the image data obtained by recording the drive recorder 1 may be detected, and the image processing may be changed based on the road width. For example, when the road is narrow, left and right images may be displayed, and the type of image during playback may be changed according to the road conditions (narrow, gravel road, one-way road, etc.). Also, at an intersection, if the drive recorder 1 is capable of displaying images during recording, the front image may be displayed, and the front image may also be displayed during playback. Also, for example, right or left images that show the road on the right or left side of the road may be displayed on the display screen 3 of the drive recorder 1.

[0313] Furthermore, when the video data is recorded by the drive recorder 1, The camera may analyze the image a little ahead of the vehicle to which the camera is attached, perform image analysis to determine whether a dangerous situation is occurring, and switch the image to a camera that makes it easier to see the danger several seconds before the danger occurs. The display mode may be switched so that the image currently being displayed shows that there will be a danger in two seconds. For example, if a car crosses in front of you in two seconds, the image may be displayed in celestial sphere mode (a mode that displays the front, back, left and right situations of the vehicle), but may be switched to a mode that displays only the front. This allows the dangerous scene to be seen two seconds later. The original celestial sphere mode may then be restored. In this way, the display form may be automatically changed when something strange may occur. Also, the entire video file may be automatically analyzed in advance to detect strange places and mark the image to be played. Whether there is a danger can be determined by whether there is a large change in the image or a large change in acceleration. The display can be changed without the user having to operate the switch button. For example, it is preferable to switch several seconds, for example 5 seconds, before an event such as a danger may occur.

[0314] Furthermore, communication data from the ENGSTA and car security may be received in the drive recorder 1, and the received data may be used as a trigger to record an event. The drive recorder 1 may be provided with a communication circuit based on LTE (Long Term Evolution), a communication circuit based on Wi-fi, and a communication circuit based on Bluetooth (registered trademark), and data representing each communication state may be recorded, and a process of changing the state of the image displayed according to each communication state may be performed. The display change process may be a change in the display form of the image displayed on the display screen 3 of the drive recorder 1, or a change in the display form of the image in the playback window 130 during playback. For example, when a command signal to turn the car security on or off is received, the image may be automatically switched to one that captures the left and right directions of the vehicle and the area around the door. Also, when a Wi-fi signal is received, it is considered that the vehicle has entered the home area, so the image may be stopped. When there are multiple communication means, it is a good idea to determine which image to display according to each communication means. The display mode may be switched according to the communication means or the content of the communication. In cases where the car doors are unlocked or the engine is started using Bluetooth (registered trademark), the display may be switched from a state in which the image around the vehicle is displayed to a state in which the image inside the vehicle is also displayed in response to receiving a Bluetooth signal, and not only the image around the vehicle but also the inside of the vehicle may be captured. This allows the state inside the car to be known at the moment the engine is started. When the engine is started, the display may be returned to the original state after about 30 seconds, and when the door is unlocked, the display may be maintained in a state in which both the image around the vehicle and the inside of the vehicle are captured until the door is locked, after which the display may be switched back to only the image around the vehicle or only the front of the vehicle.

[0315] Furthermore, roll calls are always conducted by drivers in buses, taxis, and logistics. For example, breathalyzers are also required to check whether drivers are in perfect health today. The video of the roll call is recorded. It may be possible to view the video of the roll call and the video from the drive recorder together in the playback window 130. For example, it would be good to be able to play back the video recorded in the drive recorder 1 of the driver who was called out by specifying a video file of the roll call. For example, if drivers are called out in order and get on in order, it is possible to associate the roll call video with the video from the drive recorder 1 based on the temporal relationship.

[0316] If a drive recorder 1 is attached to a forklift and also to a vehicle such as a truck, it is desirable to transmit the images recorded in both drive recorders 1 to a server, and to display the images of the vehicles (truck, forklift) driven by the same driver as a continuous day's movements in a centralized manner. For example, if a driver drives a truck and arrives at a certain location, drives a forklift at a certain location, and then gets in a car to go to a delivery destination, it is desirable to be able to view the series of images. By combining this with the roll call images, it is possible to see the entire daily routine of the driver.

[0317] Furthermore, driving instruction may be given using mixed reality. For example, a user drives once, and the video is recorded in a drive recorder. After that, the video is played back in the playback window 130, and the driving technique is automatically evaluated. During this automatic evaluation, information on points to be careful about is added to the video data. Next, the user puts on the MR glasses and drives or rides in the passenger seat in the same location, and the information is displayed on the MR glasses. The user can see the information on points to be careful about superimposed on the actual scenery and landscape. For example, information on where to look at the traffic lights and where to look at the stop lines is displayed on the lenses (glass) of the MR glasses, and the user can check the information while looking at the actual scenery.

[0318] In addition, the driving history data may be recorded in the memory 21 of the drive recorder 1, and when the vehicle approaches a place that has been visited in the past, it may be possible to display when and where the vehicle passed and to where it went. By referring to the driving history data, a display or voice output may be output indicating that the vehicle has been there before.

[0319] Furthermore, the rear camera 12 of the drive recorder 1 may capture an image of the vehicle's meter, recognize it, and use it as a trigger to start recording the video data. Also, when the vehicle's interior lamp is captured and it is detected that the interior lamp is on, event recording may be started and data indicating that the interior lamp is on may be sent to a server. Data indicating that the interior lamp is on may be sent from the server to the user of the vehicle, for example. The user may know that the interior lamp is on.

[0320] It is possible to attach a drive recorder (camera) to an automated guided vehicle in a factory. The automated guided vehicle is equipped with a LIDAR (Light Detection and Ranging) sensor, which can scan the surrounding area at high speed with a laser. When the approach of a surrounding object is detected, event recording can be started as a trigger (recording for a certain period of time, for example, from several tens of seconds to several minutes, depending on the trigger).

[0321] Event recording may be triggered by image recognition at the moment the forks of a forklift move. Images may be taken with the front camera 11, rear camera 12, etc. of the drive recorder 1, and when the forklift moves, this may be used as a trigger to record. Event recording may also be started by triggering when a person wearing a helmet gets on the forklift, or conversely, when a person without a helmet gets on the forklift. In this case, recording may start when a person gets on and stop when the person gets off. The start and end of recording playback may be determined by whether a person gets on or off. A characteristic of forklifts is that they are owned by a company, but are driven not only by employees of that company, but also by transporters. Since the driver changes, recording may start when such a person gets on, start when a person comes, or record until three minutes after the person gets off and then stop. Forklift accidents are a problem, but it will become possible to identify who broke something.

[0322] Furthermore, when a photo is taken, Exif information is attached, and text data is embedded in the JPEG data. The exposure and shutter speed at that time are embedded. Such information may be displayed in the playback window 130 by playback software. A still image file may be generated from video footage, and recorded as Exif information in the header recording area of ​​the still image file. For example, it is preferable to record the information displayed in the playback window 130 as Exif information in the header recording area. For example, a series of acceleration changes in the video may be recorded historically in the header recording area of ​​the still image file. GPS information and speed, maximum acceleration when moving within a certain range, the speed of the car, whether the brakes were applied, whether the turn signal was turned on, etc. may be recorded as Exif information in the header recording area. A certain range of speed may be recorded, or all values ​​regarding the types of events that occurred within plus or minus three seconds before or after a certain event occurred, or the values ​​regarding the types of events that occurred within a certain range of three seconds after the ... In such a case, information indicating which camera the image is from, how many cameras were used for recording, and information indicating where the camera is attached to the vehicle may be recorded as Exif information in the header recording area. Furthermore, when a drive recorder is installed in an autonomous vehicle in a factory, the cause of the trigger, whether the lamp error was red or yellow, the state of the forklift, the amount of rotation of the forklift, the state of the gyro, the battery, etc. may be recorded as Exif information in the header recording area. Information obtained in AWS (Automatic Guided Vehicle), CWS (Collision Warning System), stoppers, vehicle-to-vehicle communication, and road-to-vehicle communication may also be recorded as Exif information in the header recording area. Information such as whether there is a person, there is no person, there is a driver, there is no driver, there is a person in the passenger seat, there is no person, there is a person in the passenger seat, there is no person, there is a person in the back seat, there is no person, a business vehicle, a different vehicle, a taxi, a bus, there is a person in the bus, there is no person, etc. may also be used as a trigger, and this information may also be recorded as Exif information in the header recording area. Additionally, information recorded as Exif information may be sent to a blockchain system and recorded on the blockchain as a transaction.

[0323] Furthermore, points may be given to the user in accordance with the use of playback software (program) that displays the playback window 130. For example, by connecting to the Internet, the data imported may be uploaded in response to pressing an import button, and points may be added. If the drive recorder 1 is a communication type, video data can be automatically transmitted once permission has been given.

[0324] The scope of the present invention is not limited to the configurations expressly described in the specification, but includes combinations of various aspects of the present invention disclosed herein. The configurations of the present invention for which a patent is sought are specified in the appended claims, but it is the intention of the present inventors to include in the claims any configurations disclosed in this specification that are not currently specified in the claims.

[0325] The present invention is not limited to the configurations described in the above-mentioned embodiments. The components of each of the above-mentioned embodiments and variations may be arbitrarily selected and combined. Any component of each of the embodiments and variations may be arbitrarily combined with any component described in the means for solving the invention or any component that embodies any component described in the means for solving the invention. We intend to obtain rights to these as well through amendments to this application or divisional applications, etc.

[0326] The scope of the present invention is not limited to the configurations expressly described in the specification, but includes combinations of various aspects of the present invention disclosed herein. The configurations of the present invention for which a patent is sought are specified in the appended claims, but it is the intention of the present inventors to include in the claims any configurations disclosed in this specification that are not currently specified in the claims. The present invention is not limited to the configurations described in the above-mentioned embodiments. The components of each of the above-mentioned embodiments and variations may be arbitrarily selected and combined. Any component of each of the embodiments and variations may be arbitrarily combined with any component described in the means for solving the invention or any component that embodies any component described in the means for solving the invention. We intend to obtain rights to these as well through amendments to this application or divisional applications, etc. In addition, by filing a conversion application to a design application, it is intended to obtain rights to the overall design or partial design. The drawings depict the entire device in solid lines, but they include not only the overall design but also partial designs claimed for some parts of the device. For example, it is of course possible to make some components of the device into partial designs, and it is of course possible to include some parts of the device as partial designs regardless of the components. A part of the device may be a component of the device, or it may be a part of that component. Not only the overall design, but also the solid line parts of the drawings may be included as partial designs. The applicant intends to obtain a patent for a partial design in which any part of the design is designated as the dashed line part. [Explanation of symbols]

[0327] 1: Drive recorder, 2: Mounting part, 3: Display screen, 4: Camera part, 5: Front lens, 6: Rear lens, 7: Character string, 10: Controller, 11: Front camera, 12: Rear camera, 13: Display control device, 14: Display device, 15: Clock, 16: Road-to-vehicle / Vehicle-to-vehicle information receiving circuit, 17: Communication circuit, 18: Acceleration sensor, 20: Input device, 21: Memory, 22: GPS receiver, 25: Memory card, 31: Header recording area, 32: Video recording area, 80: Computer, 81: Display control device, 82: Display device, 90: CPU, 92: Display device, 93: Acceleration sensor, 95: Memory card reader / writer, 96: Input device, 97: Memory, 98: Communication circuit, 130: Playback window, 131: Menu bar, 132: Folder icon, 133: Camera icon, 134: Printer icon, 135: Reel icon, 136: FD icon, 137: Log icon, 138: Gear icon, 141: image display area, 141A: image display area, 141B: image display area, 142: area outside the field of view, 143: frame, 143A: frame, 144: mark, 144A: mark, 145: mark, 146: speech bubble, 146A: speech bubble, 146B: speech bubble, 147: speech bubble, 148: dashed line, 151: map image display area, 152: icon, 161: vehicle information display area, 162: vehicle status display area, 163: operation button, 166: time display area, 168: acceleration graph display area, 17 1: Play list display area, 172: Play list display area, 280: Communication robot, 281: Body, 282: Head, 283: Eyes, 284: Mouth, 291: CPU, 292: Memory, 293: Communication LAN board, 294: Wireless communication circuit, 295: Motor control board, 296: Head motor, 297: Sensor board, 298: Eye camera, 299: Audio input / output board, 300: Speaker, 301: Microphone, 310: Windshield, 311: Dashboard

Claims

1. A system having a function of correlating and recording first image data obtained by a first recording device installed in a truck and second image data obtained by a second recording device installed in a forklift.

2. The system described in claim 1 has a function of transmitting first image data obtained by a first recording device installed in the truck to a server, transmitting second image data obtained by a second recording device installed in the forklift to the server, and correlating the image data with each other and recording them on the server.

3. The system described in claim 1 or 2, further comprising a function of storing first image data obtained by a first recording device installed in the truck and second image data obtained by a second recording device installed in the forklift in the same image file.

4. A system as described in any one of claims 1 to 3, comprising a function of recording first video data obtained by a first recording device installed in the truck onto a recording medium loaded into the first recording device, the recording medium being removable from the first recording device installed in the truck, and when the recording medium is loaded into the second recording device installed on the forklift, storing second video data obtained by the second recording device installed on the forklift in the same video file as the first video data obtained by the first recording device installed in the truck.

5. 5. The system according to claim 1, further comprising a function of recording the second video data based on at least one of an operation of the forklift and a driver getting on the forklift.

6. The system according to any one of claims 1 to 5, further comprising a function of starting image capture by a second storage device when a second recording device installed on the forklift detects that a driver is approaching the forklift.

7. A system as described in any one of claims 1 to 6, comprising a function for performing processing to associate first video data obtained by a first recording device installed on the truck and transmitted from the same location as the location where the truck is stopped, with second video data that recording is started by a second recording device installed on the forklift within a predetermined time from the time the truck is stopped.

8. A system as described in any one of claims 1 to 7, which has a function of treating, when there are two pieces of video data, a first video data obtained by a first recording device installed in a truck and a second video data obtained by a second recording device installed in a forklift, where the recording end time and recording start time are approximately the same and the location at which recording ends and recording starts are approximately the same, as video data of a transfer from the truck to the forklift.

9. A system according to any one of claims 1 to 8, further comprising a function for when the truck and the forklift are reversed.

10. The system according to any one of claims 1 to 9, further comprising a function of receiving a video file of a driver's roll call from a server and displaying the video of the driver's roll call and the video of the driver driving in the video display area.

11. A program for causing a computer to realize the functions of the system according to any one of claims 1 to 10.

Images