Systems, control methods, programs, etc.

Abstract

To provide a system, a control method, and a program that reduce the amount of data to be transmitted and suppress data communication charges.SOLUTION: In a drive recorder, when an event occurs at time t3 when a vehicle is subjected to an impact or the like, video data with a relatively large amount of data is generated from time t2 to time t4, and video data with a relatively small amount of data is generated from time t1 to time t5, and the generated video data is transmitted to a server or the like.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to, for example, a system, a control method, a program, etc. [Background technology]

[0002] In prior art documents, there are those that play back video recorded by a drive recorder on a personal computer or a so-called smartphone and display it on the display screen of the personal computer or the smartphone (Patent Document 1). In addition, there are also those that display video on the display screen of the drive recorder if the drive recorder is equipped with a display screen. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-14310 Summary of the Invention [Problem to be solved by the invention]

[0004] The method described in Patent Document 1 is still not sufficient.

[0005] The aim is to provide something better than ever before.

[0006] The purpose of the present invention is not limited to this, and the applicant intends to obtain rights to configurations that aim to obtain effects from the components disclosed in the present specification and drawings, etc., through divisional applications, amendments, etc. For example, the present specification discloses a problem in which the phrase "can be" is read as "the problem is." Each problem is described as an independent entity, and the applicant intends to obtain rights to the configuration for solving this problem separately through divisional applications, amendments, etc. Even if the problem is implicitly understood from the description in the specification, the applicant intends to include part of the components described in the present specification in the claims by amendment or divisional application. [Means for solving the problem]

[0007] (1) It is preferable that the system be capable of displaying images captured in a vehicle and be equipped with a means for processing image data for the display.

[0008] By processing the video data for display, we can provide something better than ever before.

[0009] The displayable system may be, for example, a personal computer, a smartphone, a drive recorder, etc. The processing of the video data for display may be, for example, processing for playing the video, processing for transmitting the video data, processing for adjusting the brightness of the video, processing for changing the video playback format, processing for reducing the amount of video data, etc.

[0010] (2) It is preferable to process the image data for the above display so as to adjust the brightness of the image based on instructions from the user.

[0011] The brightness of the image can be adjusted based on user instructions.

[0012] The system may be provided with instruction input means such as hardware buttons or software buttons, and instructions from the user may be received using these instruction input means, or a menu may be displayed on the screen displaying the video, and instruction input means may be provided that allows the user to select an instruction from the menu. The brightness may be adjusted so that the entire video or a dark part of the video becomes brighter, or the brightness may be adjusted so that the entire video or a bright part of the video becomes darker.

[0013] (3) The above image contains bright and dark areas within one frame, and it is preferable to process the image data for the above display so that the brightness adjustment amount of the dark areas is relatively larger than the brightness adjustment amount of the bright areas.

[0014] The difference in brightness between dark and bright areas is reduced, making the image easier to see overall.

[0015] A bright area may be an area within one frame that has a brightness equal to or greater than a threshold value, and a dark area may be an area that has a brightness less than the threshold value. A detection means may be provided that detects bright areas that are equal to or greater than a threshold value, with the detected areas being considered bright areas, and the undetected areas being considered dark areas. Alternatively, a detection means may be provided that detects dark areas that are less than a threshold value, with the detected areas being considered dark areas, and the undetected areas being considered bright areas. The brightness adjustment amount may be adjusted by adjusting the gain of the video data. Brightness adjustment of bright areas The brightness may be adjusted so that both the bright and dark areas are brighter, or so that the amount of brightness adjustment in the dark areas is relatively larger than the amount of brightness adjustment in the bright areas. It is even better to provide a means for adjusting this threshold value. The means for adjusting the threshold value is preferably a means for adjusting the threshold value by inputting a user instruction.

[0016] (4) The above image includes an image portion inside the vehicle cabin and an image portion outside the vehicle cabin within one frame, and it is preferable to process the image data for the above display so as to adjust the brightness of the image portion inside the vehicle cabin.

[0017] Image data is obtained with brightness adjusted for the image portion of the vehicle interior.

[0018] A means for detecting the image portion inside the vehicle cabin may be provided, and the detected portion may be the image portion inside the vehicle cabin, and the undetected portion may be the image portion outside the vehicle cabin. Alternatively, a means for detecting the image portion outside the vehicle cabin may be provided, and the detected portion may be the image portion outside the vehicle cabin, and the undetected portion may be the image portion inside the vehicle cabin. Edge detection may be used to detect the image portion inside the vehicle cabin. Brightness may be adjusted by adjusting the gain of the image data. Brightness may be adjusted so that both the image portion inside the vehicle cabin and the image portion outside the vehicle cabin are bright, or brightness may be adjusted so that only the image portion inside the vehicle cabin is bright.

[0019] (5) It is preferable to process the image data for the above display so that the brightness of the captured image is adjusted so that the amount of brightness adjustment inside the vehicle is relatively greater than the amount of brightness adjustment of the image outside the vehicle.

[0020] Since it is believed that the brightness of the image portion outside the vehicle cabin is often brighter than the brightness of the image portion inside the vehicle cabin, by adjusting the brightness of the captured image so that the amount of brightness adjustment inside the vehicle cabin is relatively greater than the amount of brightness adjustment of the image portion outside the vehicle cabin, the difference in brightness between the image portion inside the vehicle cabin and the image portion outside the vehicle cabin is reduced, making the image easier to see as a whole.

[0021] A means for detecting the image portion inside the vehicle cabin may be provided, and the detected portion may be regarded as the image portion inside the vehicle cabin, and the undetected portion may be regarded as the image portion outside the vehicle cabin. Alternatively, a means for detecting the image portion outside the vehicle cabin may be provided, and the detected portion may be regarded as the image portion outside the vehicle cabin, and the undetected portion may be regarded as the image portion inside the vehicle cabin. The image portion inside the vehicle cabin may be detected using edge detection. The brightness adjustment amount may be adjusted by adjusting the gain of the image data. The brightness adjustment may be performed so that both the image portion inside the vehicle cabin and the image portion outside the vehicle cabin are brightened, so that the brightness adjustment amount inside the vehicle cabin is relatively larger than the brightness adjustment amount of the image portion outside the vehicle cabin, or the brightness adjustment may be performed so that only the image portion inside the vehicle cabin is brightened. The means for detecting the image portion inside the vehicle cabin may, for example, receive a selection of an area of ​​the vehicle cabin area in the image from a user and detect the selected area as the image portion inside the vehicle cabin, or may automatically detect the image portion inside the vehicle cabin. An example of an automatic detection process may be a process that detects the movement of the vehicle and detects areas where the image changes in accordance with the movement as the outside of the vehicle cabin, and areas where the image does not change in accordance with the movement as the inside of the vehicle cabin. In particular, it is preferable to detect areas in the video where there is movement corresponding to the speed of the vehicle as being outside the vehicle cabin, and areas where there is no movement corresponding to the speed of the vehicle as being inside the vehicle cabin.

[0022] (6) It is preferable to provide a first instruction input means for inputting an instruction from the user to adjust the brightness inside the vehicle cabin.

[0023] The brightness inside the vehicle can be adjusted based on user instructions.

[0024] The system has hardware buttons and software buttons. may be used as the first instruction input means, or a menu may be displayed on a screen displaying the video, and the menu may be used as the first instruction input means. If a menu is used as the first instruction input means, instructions from the user may be selected from the menu. Also, a keyboard of a personal computer may be used as the first instruction input means, or a keypad may be displayed on the display screen of a personal computer, a smartphone, a drive recorder, or the like, and the displayed keypad may be used as the first instruction input means. It is preferable to adjust the brightness inside the vehicle cabin based on instructions from the user given by the first instruction input means.

[0025] (7) It is preferable to provide a second instruction input means for inputting an instruction from the user that does not perform the brightness adjustment.

[0026] It is possible to prevent brightness adjustment based on an instruction from the user. By combining the first instruction input means and the second instruction input means, it is possible to switch between an instruction to adjust brightness and an instruction not to adjust brightness.

[0027] The second instruction input means may be a hardware button or a software button in the system, or a menu may be displayed on a screen displaying video, with the menu serving as the second instruction input means. If a menu is used as the second instruction input means, the user may be able to select instructions from within the menu. The second instruction input means may also be a keyboard of a personal computer, or a keypad may be displayed on the display screen of a personal computer, a smartphone, a drive recorder, or the like, with the displayed keypad serving as the second instruction input means. When an instruction is given from the second instruction input means, the instructions from the second instruction input means may be stopped, and when an instruction is given from the second instruction input means, the instructions from the first instruction input means may be stopped.

[0028] (8) It is preferable to process the image data for the above display so as to separate a first window that displays the image from a second window that includes an operating system for manipulating the displayed image.

[0029] The first window that displays the image can be separated from the second window that includes an operating system for manipulating the displayed image.

[0030] For example, by displaying the first window displaying the video over the entire display screen, the video is displayed larger and easier to see. The first window displaying the video and the second window including an operation system for manipulating the displayed video may be displayed on the same screen or on different screens, but preferably on different screens. A switching instruction input means may be provided for switching between an instruction to display the first window and the second window on the same screen and an instruction to display the first window and the second window on different screens. In response to a command from the switching instruction input means, the first window and the second window can be displayed on the same screen or on different screens. For example, a software switching button may be provided on the screen, or a hardware switching button may be provided in the system. The display can be switched in response to a command from the switching button. The operation system for manipulating the displayed video may be, for example, one that issues commands to control video playback, such as starting playback, stopping playback, pausing playback, fast forwarding, and rewinding.

[0031] (9) It is preferable to process the video data for the above display so that the display screen on which the first window for displaying the video is displayed is separate from the display screen on which the second window including the operating system for manipulating the video is displayed.

[0032] The display screen on which the first window for displaying the image is displayed can be separate from the display screen on which the second window including the operating system for manipulating the image is displayed.

[0033] For example, it is preferable to position the display screen displaying the first window for displaying the video in a location that is easy to see for the person viewing the video, and the display screen displaying the second window including the operating system for operating the video in a location that is easy to operate for the person operating the video. For example, when video is being obtained by a driver managed by an administrator, the administrator operates the video while looking at the display screen displaying the second window, and the driver or other person managed by the administrator views the video on the display screen displaying the first window, allowing the administrator to move the display screen displaying the second window to a location that is easy to see so that the video can be easily operated, and the driver or other person managed by the administrator can move the display screen displaying the first window to a location that is easy to see so that the video can be easily viewed. It is preferable that the display screen displaying the first window for displaying the video is larger than the display screen displaying the second window including the operating system for operating the video.

[0034] (10) It is preferable to process the image data for the display so as to generate an area for displaying the image in a second window including an operation system for operating the displayed image.

[0035] In this way, the image is also displayed in the second window, allowing the user to operate the device while viewing the image. When the second window containing the operating system for operating the displayed image is displayed, the first window may appear on the display screen when the user issues a display command for the first window for displaying the image. At that time, the first window may be displayed over the entire display screen.

[0036] (11) It is preferable to process the video data for display so that accompanying information of the video is displayed in the first window for displaying the video.

[0037] By doing this, it is possible to view the accompanying information along with the video.

[0038] The accompanying information for the video may be, for example, vehicle speed information, acceleration information, position information, etc. The accompanying information may be displayed on the video, or may be displayed in an area other than the area in the first window for displaying the video. For example, when video is being obtained by a driver supervised by an administrator, the administrator operates the video while looking at the second window, and the driver supervised by the administrator views the video displayed in the first window. The video viewed by the supervised driver displays accompanying information such as vehicle speed information and acceleration information when the driver was driving, so by looking at this accompanying information, the driver can more clearly understand how he or she was driving.

[0039] (12) It is advisable to process the video data for display so as to reduce the amount of data when the time is far from the time when the recording command is given.

[0040] The amount of data when the time is far from the time when the recording command is given can be reduced compared to the amount of data when the time is near the time when the recording command is given.

[0041] The recording command may be generated, for example, when some kind of trigger is given to the vehicle capturing the video or when some kind of event is applied to the vehicle. For example, an event may occur when the brakes of the vehicle are applied or when an impact is given to the vehicle, and a trigger may be given to the vehicle. Also, a software button or a hardware button may be provided in the system, and a recording command may be given by operating these buttons. This reduces the amount of data when the time from when the recording command is given to a later time point is far away. Alternatively, the amount of data may be reduced when the time from the time when the recording command is issued to the previous time is distant. A period including the time when the recording command is issued is defined as a first period, and at least one of the periods after the first period has elapsed or before the start of the first period is defined as a second period, and the amount of data in the second period may be reduced compared to the amount of data in the first period. The first period may start a certain period after the recording command is issued, and the second period may start after the end of the first period, and the amount of data in the second period may be reduced compared to the amount of data in the first period.

[0042] The amount of data can be reduced by increasing the compression rate of the video data, lowering the frame rate of the video data, or thinning out the number of pixels in the video. The amount of data can be reduced when video data is obtained by shooting in a vehicle. A rule for reducing the amount of data, which indicates the method by which the amount of data is reduced, can also be recorded.

[0043] (13) It is preferable to process the video data for display so that the amount of video data transmitted is reduced.

[0044] By doing so, it becomes possible to transmit reduced video data, and since the amount of video data to be transmitted is reduced, the transmission time is shortened.

[0045] The data volume should be the amount of data per unit time. Video data should be transmitted based on mobile phone communication standards such as LTE (Long Term Evolution) and 4G. When capturing video in a vehicle using a drive recorder, the volume of the captured video data should be reduced before transmitting it from the drive recorder to a server.

[0046] (14) It is preferable to process the video data for display so that the amount of video data to be transmitted is smaller than the amount of video data recorded on the recording medium in response to a recording command.

[0047] The amount of video data to be transmitted can be made smaller than the amount of video data to be recorded.

[0048] The recording medium may be a hard disk or semiconductor element memory built into the recording device, or may be a portable recording medium such as an SD card or an xD picture card.

[0049] (15) It is preferable to process the video data for the above display so that the playback speed of the captured video is changed based on the speed of the vehicle.

[0050] The playback speed of the captured video can be changed based on the vehicle speed. The vehicle speed should be the speed of the vehicle that captured the video. The vehicle speed should be information based on vehicle speed information, particularly vehicle speed information. The vehicle speed information should be recorded in association with the video data on a medium on which video data representing the captured video is recorded. The vehicle speed information should be obtained based on GPS (global positioning system) information, vehicle speed pulse information, etc., which are associated with the video data representing the continuously recorded video and recorded on a recording medium together with the video data. The playback speed of the captured video can be changed by changing the time required to capture the video and the time required to play it back. The playback speed can be changed by changing the frame rate of the video when it was captured and the frame rate when it was played back.

[0051] (16) It is preferable to process the video data for display so as to have a function of slowing down the video playback speed when the vehicle is traveling at a slow speed compared to the video playback speed when the vehicle is traveling at a fast speed.

[0052] In this way, the video captured when the vehicle is traveling slowly can be clearly seen, and the video captured when the vehicle is traveling quickly can be made difficult to see. The playback speed of video captured when the vehicle is traveling slowly can be slowed down, making it easier to check the video captured when the vehicle is traveling slowly. Since the playback speed of video captured when the vehicle is traveling quickly is not slow, the playback time of the entire video can be shortened. For example, a vehicle traveling slowly can be defined as being when the vehicle is stopped, and in this way, the video captured when the vehicle is stopped can be viewed. A vehicle's speed can be determined to be fast when its speed is above a threshold. When other vehicles are captured in the video, a vehicle's speed can be determined to be slow when it is slower than those other vehicles. In particular, in conjunction with the control in (16), control can be performed to slow the playback speed of the video when it is slower than other vehicles. The playback speed of the captured video can be slowed down by making the time required to play the video longer than the time required to shoot the video. The function for slowing down the vehicle speed should preferably be a function for increasing the number of frames per unit time of the video during playback compared to the number of frames per unit time of the video during shooting. The vehicle should be the vehicle that is shooting the video. For example, when the vehicle speed is slow so that information about stopped vehicles can be obtained, the playback speed should be normal speed. When the vehicle speed is fast, the playback speed should be n times faster than normal speed. n should be a number greater than 1.

[0053] (17) It is preferable to process the video data for display so as to provide a function for increasing the video playback speed when the vehicle speed is slow compared to the video playback speed when the vehicle speed is fast.

[0054] This allows for clearer visibility of images when the vehicle is traveling at a high speed, while making it difficult to see images when the vehicle is traveling at a low speed. Images captured when the vehicle is parked can be played back slowly, since the images are the same. Images captured when the vehicle is traveling at high speed can be played back slowly, since the vehicle is traveling at high speed and the images flow quickly. Images captured when the vehicle is stopped at a traffic light or other such location can be fast-forwarded for less important information, such as images captured when the vehicle is traveling at a low speed. This simplifies viewing. It is advisable to determine that a vehicle's speed is slow when its speed is below a threshold. When surrounding vehicles are captured in the video, it is advisable to determine that a vehicle's speed is slow when it is slower than the surrounding vehicles. In particular, with regard to the control of (17), it is advisable to speed up the playback speed of the video when it is slower than other vehicles. The function for speeding up the playback speed of the video should be a function for speeding up the playback speed of the video by shortening the time required to play the video compared to the time required to shoot the video. In particular, the function for increasing the video playback speed should be a function for reducing the number of frames per unit time of the video during playback compared to the number of frames per unit time of the video during shooting. For example, when the vehicle speed is fast, the playback speed should be normal speed or less than 1x. When the vehicle speed is slow, the playback speed should be n times faster than the normal speed. n should be a number greater than 1.

[0055] (18) It is preferable to process the video data for the display so as to increase the playback speed of the video of the front or rear of the vehicle.

[0056] The playback time can be shortened. In particular, since the playback speed of the video of the front or rear of the vehicle may be increased without causing any sense of incongruity, the playback time of the video can be shortened relatively smoothly. It is preferable to increase the playback speed of the video of the front or rear of the vehicle without receiving a command from the user to increase the playback speed. The playback speed is increased automatically. Whether the video is of the front or rear of the vehicle can be determined, for example, by image analysis.

[0057] (19) It is advisable to process the image data for the above display so as to slow down the playback speed of the image of the vehicle's lateral direction.

[0058] You can view the lateral video at a leisurely pace. In particular, if the playback speed of the lateral video of the vehicle is too fast, the video may blur and create an unnatural feeling. However, since the playback speed of the lateral video of the vehicle can be slowed down, this can be prevented from creating an unnatural feeling. To slow down the playback speed, for example, it is advisable to make the time required for playback longer than the time required for filming. Whether or not the video is a lateral video of the vehicle can be determined, for example, by analyzing the video.

[0059] The video captured in the vehicle may include at least one of a video of the front of the vehicle or a video of the rear of the vehicle, and a video of the side of the vehicle. Video data representing such video may be recorded on a recording medium. When video of the front or rear of the vehicle is captured by a first camera that captures video of the front or rear of the vehicle, the playback speed of the video obtained by such first camera may be increased. When video of the side of the vehicle is captured by a second camera that captures video of the side of the vehicle, the playback speed of the video obtained by such second camera may be decreased. When video is captured by a camera that can capture video of at least one of the front or rear of the vehicle and video of the side of the vehicle, the display area may be changed in response to a command to change the display area of ​​the video captured by such camera. When video of the front or rear of the vehicle is displayed, the playback speed may be increased. Furthermore, when video of the side of the vehicle is displayed, the playback speed may be decreased. When an image of at least one of the front or rear of the vehicle and an image of the side of the vehicle are displayed, the playback speed of the image should be determined based on the ratio of the display portion of the image of at least one of the front or rear of the vehicle to the display portion of the image of the side of the vehicle. For example, when the display portion of the image of at least one of the front or rear of the vehicle is larger than the display portion of the image of the side of the vehicle, the playback speed of the image should be increased. Conversely, when the display portion of the image of the side of the vehicle is larger than the display portion of the image of at least one of the front or rear of the vehicle, the playback speed of the image should be decreased.

[0060] (20) It is preferable to process the video data for the above display so that the captured video is played back at the normal playback speed based on a normal playback speed command given when the video is not being played back at the normal playback speed.

[0061] By doing this, the video can be viewed at the normal playback speed. The normal playback speed command can be issued by the user using a button or the like. It can also be generated based on a video scene in which some kind of impact has occurred to the vehicle. It can also be generated based on information recorded in association with the video (for example, so-called historical information such as acceleration information, speed information, information on whether the brakes have been applied, information on whether the turn signals have been turned on, etc.). The normal playback speed should be the same as the time required for shooting and the time required for playback. For example, if you are shooting at a rate of one frame every 1 / 30 seconds, it is recommended that you play back one frame every 1 / 30 seconds.

[0062] (21) It is preferable to process the image data for the above display so that the area for displaying information about the vehicle obtained while driving the vehicle is large.

[0063] The area for displaying information about the vehicle is larger, making it easier to see the information about the vehicle. Information about the vehicle obtained while driving the vehicle should be data other than video data representing the video captured by the vehicle. Information about the vehicle obtained while driving the vehicle should be, for example, information about the vehicle obtained while the vehicle capturing the video is driving. Information about the vehicle should be information obtained from the vehicle capturing the video (for example, map information showing the vehicle's position, vehicle acceleration information, speed information, information that the brakes have been applied, information that the blinker has been turned on, etc.). The area for displaying information about the vehicle should be divided into at least two levels, a small area and a large area. It is preferable that the size of the area is the same as that of a floor, and that the function be to make the area larger. It is preferable that the area displaying information about the vehicle be switched from a small area to a large area in response to a command given from a button or the like that gives a command such as a command to enlarge the area. It is preferable that a default size is determined for the area displaying information about the vehicle, and that the area be larger than that default size. It is preferable that the area related to the vehicle be enlarged to a predetermined size in response to a command from the user or a command from the system. It is preferable that the function is not a function for enlarging the window by dragging it. It is preferable that the area related to the vehicle be enlarged in response to a command from a button or the like such as a command to enlarge the area. When an area for displaying an image captured by the vehicle and an area for displaying information about the vehicle are defined in the window, it is preferable that the area for displaying information about the vehicle be larger than the default size. It is preferable that the size of the area displaying the image is not changed, but it is especially preferable that it be changed.

[0064] (22) It is preferable to process the image data for the above display so that the area for displaying information about the vehicle is larger than the area for displaying the image captured in the vehicle.

[0065] Since the area for displaying information about the vehicle is larger than the area for displaying the image, the information about the vehicle is easier to see than the image.

[0066] The information about the vehicle may be information obtained by the vehicle capturing the video (for example, so-called historical information such as map information showing the vehicle's position, vehicle acceleration information, speed information, information on whether the brakes have been applied, and information on whether the turn signals have been turned on). Normally, the area displaying information about the vehicle may be smaller than the area displaying the video captured by the vehicle, and upon receiving a command from the user or the system, the area displaying information about the vehicle may be larger than the area displaying the video captured by the vehicle. The video captured by the vehicle may be displayed in a relatively large area, and information about the vehicle may be displayed in a relatively small area, and upon receiving a command from the user or the system, the video captured by the vehicle that was displayed in a large area may be displayed in a smaller area, and information about the vehicle that was displayed in a small area may be displayed in a larger area. The video captured by the vehicle may be displayed in a relatively large area, and information about the vehicle may be displayed in a relatively small area, and upon receiving a command from the user or the system, the information about the vehicle may be displayed in an area even larger than the relatively large area displaying the video captured by the vehicle. It is preferable that the image captured by the vehicle is displayed in a relatively large area and information about the vehicle is displayed in a relatively small area, and that upon receiving a command from the user or the system, the image captured by the vehicle is displayed in an area that is even smaller than the relatively small area.It is preferable that the information about the vehicle is displayed in an area that is relatively larger than the image captured by the vehicle.

[0067] (23) It is preferable to process the image data for the above display so that the vehicle acceleration over time is displayed in three dimensions according to the time at which the image was taken.

[0068] By doing this, acceleration is easier to see. The vehicle acceleration should be the acceleration of the vehicle that captured the video. The vehicle acceleration can be displayed in three dimensions by displaying a three-dimensional space with degrees of freedom in each direction of width, depth, and height as a plane. Data representing acceleration should be recorded on a recording medium in association with video data representing one video. Video data representing one video should be included in one video file. The entire acceleration represented by data recorded on a recording medium in association with video information representing one video should be displayed in three dimensions. The origin representing the vehicle acceleration at the start of filming should be the same as the point in time representing the vehicle acceleration at the end of filming. A portion of the vehicle acceleration from the start to the end of filming should be displayed.

[0069] (24) The video data for the above display may be processed so that the acceleration in front of the vehicle is displayed on a first axis of the three-dimensional graph, the lateral acceleration of the vehicle is displayed on a second axis of the three-dimensional graph, and the vertical acceleration of the vehicle is displayed on a third axis of the three-dimensional graph.

[0070] This makes it easier to see the acceleration. It is preferable that the first, second, and third axes are perpendicular to each other. This makes it relatively easy to understand the vehicle's acceleration. It is preferable to define the first axis as from back to front or from front to back, the second axis as the width direction (right or left), and the third axis as the up or down direction. It is preferable to define the first axis as the X-axis, the second axis as the Y-axis, and the third axis as the Z-axis.

[0071] (25) It is preferable to process the image data for the above display so that information about the vehicle is superimposed on the image.

[0072] The information about the vehicle becomes easier to see. The information about the vehicle should be information obtained by the vehicle capturing the video (for example, so-called historical information such as map information showing the vehicle's position, vehicle acceleration information, speed information, information on whether the brakes have been applied, and information on whether the turn signals have been turned on). When displaying information about the vehicle superimposed on the video, it is advisable to erase the area in which the information about the vehicle is displayed. When displaying information about the vehicle superimposed on the video, it is advisable to display the video in a larger area than when the information about the vehicle is not superimposed on the video.

[0073] (26) It is preferable to process the image data for the above display so that an image of a car is displayed within a frame and information on the acceleration of the car is displayed within the frame in which the image of the car is displayed.

[0074] Vehicle acceleration information is displayed within a frame displaying the image of the car, making it relatively easy to understand the vehicle's acceleration. It is a good idea to display the acceleration in the car's forward direction on the front side of the image of the car. It is also a good idea to display the car's lateral acceleration on the side (left or right) of the image of the car. It is also a good idea to display the car's vertical acceleration above or below the image of the car. It is a good idea to display vehicle acceleration information within the frame displaying the image of the car so that the arrow points in the positive direction of acceleration. For example, when acceleration is occurring in the forward direction, the arrow should point in the forward direction; when acceleration is occurring in the backward direction, the arrow should point backward; when acceleration is occurring in the left direction, the arrow should point left; when acceleration is occurring in the right direction, the arrow should point right; when acceleration is occurring in the upward direction, the arrow should point upward; and when acceleration is occurring in the downward direction, the arrow should point downward.

[0075] (27) In the above system whose performance does not satisfy the predetermined conditions, the video data for the display may be processed so as to notify the user that the system does not have the capability to process the video data for the display.

[0076] It is clear that the system does not have the capability to process video data for display. In systems whose performance does not meet specified conditions, it can be notified that the system does not have the capability to process video data, and that there is no problem with the video or the device that captures the video. In systems whose system computer or CPU (Central Processing Unit) does not meet specified conditions, it is desirable to notify that the system does not have the capability to process video data for display. The specified condition is desirably a performance that causes frame drops when displaying video captured in a vehicle.

[0077] (28) It is preferable to process the video data for the above display so that the video is played back from a predetermined playback position in response to a command to move the playback position of the video.

[0078] This makes it possible, for example, to skip video up to a specified playback position and play it back. The desired video portion can be quickly played back. The specified playback position may be a position that satisfies certain conditions. The position that satisfies certain conditions may be a position that contains a specific scene or a predetermined position. For example, the position that contains a specific scene may be determined by analyzing video captured by a vehicle and determining the position of a scene that contains a specific subject, such as a specific person, object, or building. The position that contains a specific scene through video analysis may be determined in advance before video playback. In particular, the determined position may be stored in association with the video. The predetermined position may be determined based on information about the vehicle. For example, it may be the position of a scene where a specific vehicle situation occurs, such as sudden braking or a sudden start. The vehicle information may be information obtained from the vehicle capturing the video (e.g., so-called history information, such as map information indicating the vehicle's position, vehicle acceleration information, speed information, information on whether the brakes are applied, and information on whether the turn signals are turned on).

[0079] (29) It is preferable to process the video data for the above display so as to display the current playback position and the predetermined playback position.

[0080] You can see the current playback position and the specified playback position. When playing back video at a specified playback command in response to a video movement command, you can see which part of the entire video will be played. For example, it would be good to display a playback bar or seek bar that indicates the playback position of the video, and display a mark such as a line on such a bar that indicates the specified playback position.

[0081] (30) It is preferable to process the video data for the display so as to change the display mode of a predetermined playback position determined based on an analysis of the video and the display mode of a predetermined playback position determined based on information about the vehicle obtained when the video was captured.

[0082] By looking at the playback position display, it is possible to determine whether the predetermined playback position is determined based on an analysis of the video or based on information about the vehicle obtained when the video was shot. The predetermined playback position determined based on an analysis of the video may be, for example, a position obtained by analyzing a scene in the video that includes a scene of a predetermined person or building. The playback position determined based on information about the vehicle may be, for example, a position obtained from historical information such as map information showing the vehicle's location, vehicle acceleration information, speed information, information about whether the brakes have been applied, and information about whether the turn signals have been turned on. The information about the vehicle may be obtained separately from the video when it is shot. For example, it may be possible to change the display style of the mark displayed at the predetermined playback position determined based on an analysis of the video, using a different color (e.g., red) or shape (e.g., triangle) from the color (e.g., green) or shape (e.g., circle) of the mark displayed at the playback position determined based on information about the vehicle.

[0083] (31) The video data for display may be processed so as to limit transmission of the video data representing the video captured in the vehicle.

[0084] By doing this, when the system is equipped with a function for transmitting video data, it may be possible to relatively reduce the amount of video data transmitted. When the system is installed in a vehicle and transmits video data from the system installed in the vehicle, it is advisable to limit the transmission of video data. Also, when transmitting video data from the system installed in the vehicle via a telephone line, it is advisable to limit the transmission of video data by limiting the number of transmissions. It is advisable to detect whether the number of transmissions has been reached. Also, it is advisable to limit the transmission of video data by limiting the amount of video data transmitted. When the transmission of the video data is restricted, the video data should be transmitted by a transmission method that does not require transmission costs or that requires low costs.

[0085] (32) The video data for display may be processed to detect the amount of video data transmitted representing the video captured in the vehicle.

[0086] The amount of video data transmitted can be detected. When transmitting video data representing video captured in each of multiple vehicles from each vehicle, it is advisable to detect the amount of video data transmitted from each vehicle. It is advisable to have a function to display the detected amount of transmission. In particular, it is advisable to have a function to display a list of the amount of transmission from each of multiple vehicles. The list display can be displayed in descending order of transmission amount, either large or small. For example, a ranking of transmission amount can be obtained, and vehicles with large transmission amounts can be identified. In a company that owns multiple vehicles, it is advisable to detect the amount of video data transmitted from each vehicle representing video captured in each of multiple vehicles. It can be determined which vehicle in the company's fleet transmits the largest amount of video data.

[0087] (33) It is preferable to process the video data for the above display so that a communication speed test is performed before recording the video data representing the video captured in the vehicle, and the video data is transmitted under conditions obtained based on the communication speed test.

[0088] Video data can be transmitted under conditions obtained from a communication speed test. For example, it is a good idea to determine the video recording resolution, frame rate, bit rate, etc. based on the speed at which transmission was possible in a communication speed test and then transmit the video data. If a communication error occurs during communication, it is a good idea to conduct a communication speed test again and change the conditions to transmit the video data. When transmitting video data under conditions obtained from a communication speed test, it is a good idea to process the original video data so that the video data rate decreases as the communication speed becomes relatively slower.

[0089] (34) When the image captured in the vehicle differs from the image visible to humans, the image data for the display may be processed so as to at least either notify or record that the image differs from the image visible to humans.

[0090] This makes it possible to determine whether the captured image differs from what a human can actually see. For example, it is possible to determine whether the image the vehicle driver could see differs from the recorded captured image. Therefore, for example, when playing back video of an accident, it is possible to determine that the video may not have been visible to the driver. This allows the police, courts, insurance companies, etc. to use this notification or recorded information to determine whether the driver was at fault at the time of the accident. For example, if a polarizing filter or similar can be attached to the camera that captures the video, it is advisable to record or notify the driver so that it is clear whether the video was captured with or without a polarizing filter. For example, it is advisable to install a switch on the camera that captures the video and detect the state of the switch to determine whether the video was captured with a polarizing filter attached. It is also advisable to analyze the captured video to determine whether a polarizing filter was attached. For example, it is advisable to analyze the video to determine whether a polarizing filter was attached based on the reflection of the dashboard or other objects in the window glass. Furthermore, for example, when the image captured in a vehicle is not an image that can be seen by humans, the image data for the display may be processed to notify the user that the image is not obtained in a state that can be seen by humans.

[0091] (35) It is preferable to process the video data for the above display so as to communicate with a videophone device.

[0092] The system can be used as a videophone device. When the system is installed in a vehicle, the system installed in the vehicle can be used as a videophone device. The system should have a function for sending and receiving video data with the videophone, and a function for sending and receiving audio data with the videophone to make a call. The system should have a function for recording on a recording medium, and record audio data, video data, etc. showing the contents of a videophone call on the recording medium.

[0093] (36) It is preferable to process the image data for the display so that multiple images of the movements of a particular person are displayed in succession.

[0094] In this way, multiple images of a specific person's movements can be displayed in succession. For example, it is possible to display in succession images of a specific person's actions, such as images of the specific person before getting into the vehicle and driving, images of the specific person driving the vehicle, and images of the specific person after driving the vehicle. This makes it possible to confirm whether a specific person is behaving appropriately. For example, it is possible to display in succession the driver's daily movements, such as images of the driver before leaving the office, images of the driver driving the vehicle, and images of the driver returning to the office. For example, when multiple images are represented by multiple video data, the multiple video data can be identified by an ID that identifies a specific person, and the multiple data identified by that ID can be displayed in succession in chronological order of the recording of the video data. The function of continuously displaying multiple images of the movements of a specific person may, for example, identify the person through image recognition, such as whether a person in the captured video resembles an image of a person registered in advance, and continuously play back the portions of the video in which the specific person appears.It may also be possible to continuously play back different files of video in which the specific person appears in chronological order, or to extract and play portions of one file of video in which the specific person appears, or to play a combination of these, and in particular to continuously play back video files captured at different times throughout the day.

[0095] (37) It is preferable to process the video data for the display, representing the video captured in the vehicle, so that the video data recorded on the first recording medium and the video data recorded on the second recording medium are different video data.

[0096] By doing so, different images can be obtained. The video data recorded on the first recording medium and the video data recorded on the second recording medium should be shot at the same time. For example, video data representing a processed image of a celestial image should be recorded on the first recording medium, and video data representing an image closer to the raw image than the processed image should be recorded on the second recording medium. The processed image should be a partially cut-out image or a coordinate-transformed image. The image closer to the raw image should be a RAW image or a compressed image. The video data representing the forward image should be recorded on the first recording medium, and the video data representing the rearward image should be recorded on the second recording medium. The video data recorded on the first recording medium and the video data recorded on the second recording medium should be played back with their shooting times synchronized. The image represented by the video data recorded on the first recording medium should be displayed in a first window, and the image represented by the video data recorded on the second recording medium should be displayed in a second window. The image displayed in the first window and the image displayed in the second window should be displayed in synchronization. It is a good idea to record information about the processing that has been done on the processed video, such as the frame removal range applied to the RAW video, the details of the coordinate system transformation, etc., and display the corresponding range of the processed video in the RAW video with a dashed line, etc. If the system is a drive recorder installed in a vehicle, It is preferable to generate processed video and video that is closer to the raw video than the processed video. Also, when the system is a personal computer rather than a drive recorder installed in a vehicle, it is preferable to generate processed video and video that is closer to the raw video than the processed video on the personal computer.

[0097] (38) It is preferable to provide a button in the system that issues a command to upload to a server, and to process the video data for the above display so that video data representing the video taken in the vehicle is uploaded to the server in response to a command from the button.

[0098] In response to a command from the button, video data representing video captured in the vehicle can be uploaded to a server. The system is preferably installed in the vehicle. If the system has a function for recording video data, it is preferable to send the video data to the server before recording. It is also preferable to send the video data to the server while recording. It is also preferable to read out the recorded video data and send it to the server. It is preferable to record the position of the video data when a command from the button is received, and when the video data is played back, the position where the command from the button was received can be displayed together with a map using an icon or the like. When such an icon or the like is clicked, it is preferable to display the video represented by the uploaded video data. It is also preferable to display the video represented by the video data stored in a storage location identified by a uniform resource locator (URL).

[0099] (39) It is preferable to process the image data for the above display so that the image is captured in the vehicle based on a capture command from outside the vehicle capturing the image.

[0100] This system allows for shooting in a vehicle based on a shooting command from outside the vehicle. For example, when both the system and a mobile phone capable of communicating with the system are inside the vehicle, a shooting command can be sent from a server outside the vehicle to the mobile phone, which then sends the shooting command to a system installed inside the vehicle, and the system that receives the shooting command can begin shooting. For example, a smart speaker can be installed in the sales office of a company that owns vehicles, and trigger data for a shooting command can be sent using the smart speaker to a specific driver's mobile phone, which can then send the trigger data for a shooting command to a system installed in the vehicle driven by that specific driver. When recording video data obtained by shooting based on the trigger data, it is recommended to set a trigger name. The set trigger name can be displayed when playing back the video. For example, a trigger name such as "Shooting trigger by voice from Manager A to Driver X" can be displayed. For example, popular drivers or drivers who engage in dangerous driving may receive a lot of trigger data from various managers.

[0101] (40) The video data for the display may be processed to display the frame rate of the recording of the video captured in the vehicle and the frame rate of the display.

[0102] The frame rate of the video recording and the frame rate of the display can be known. A frame interpolation function may be provided. Data indicating the frame rate of the video should be recorded in association with the video data.

[0103] (41) It is preferable to process the image data for the above display so as to give the robot operating instructions based on the playback of the image taken in the vehicle.

[0104] The robot will move based on the playback of the video. For example, when playing a scene with an impact, you can give a robot, such as a communication robot, a command to make it look like it is about to fall over. The robot will move as if it is about to fall over. It would be good to give commands to the robot. It would also be good to give a motion command to make the robot get up if it has fallen over. It would also be good to give the robot a motion command to warn the driver about driving, for example, scolding the driver by saying, "Don't drive dangerously." It would be good to place a robot in a driving school and have it scold the driver instead of the driving school instructor. It would also be good to place a robot in a bus company or transportation company and have it scold the driver instead of the bus company's or transportation company's operations manager. If the operations manager scolds the driver, it will cause friction, but if the robot scolds the driver, it will have less of an impact on human relationships. When you want the robot to perform an action, the system can send the desired action command to the robot, and the robot will receive the action command and operate.

[0105] (42) It is preferable to process the video data for the display so as to output a file for presentation software relating to the video captured in the vehicle.

[0106] A presentation software file is obtained. There are presentation software files that can be output to evaluate the vehicle's driving based on the video and display a score for the vehicle's driving.

[0107] (43) It is preferable that the image captured in the vehicle is a spherical image, and the image data for the display is processed so that the spherical image is displayed in the virtual reality space.

[0108] It is possible to view spherical images in a virtual reality space. Spherical images should be captured using a camera that can capture images in all directions, including up, down, left, right, forward, and backward.

[0109] In the virtual reality space, it is a good idea to place the screen of the video playback app, PC viewer, etc. as part of a plane (board) or a gentle cylindrical surface (or gentle sphere) in three-dimensional space. When you press the dual-screen display button on the PC viewer, it is a good idea to display the PC viewer operation panel on one side and the playback image on the other side.

[0110] It is a good idea to provide a display button in the VR space that projects (texture-maps) a spherical image over the entire interior of the sphere, with the location of the person as the origin. Pressing this button will allow the viewer to see the image as if they were inside the vehicle. The sphere should be displayed behind the window. It is also a good idea to hide all windows, creating a display state that makes it appear as if the viewer is completely inside the vehicle. It is also a good idea to be able to switch between the two using a button on the controller. It is also a good idea to display the sphere in front of the window. By moving your position outside the sphere or moving forward, you can exit the sphere and operate the operation window, etc. For example, it is a good idea to project the image outside the sphere when you are outside the sphere. This way, the image can be viewed from outside the sphere. Instead of projecting onto a perfect sphere, it is a good idea to map onto a curved surface that is set to correct for distortions in the optical system, such as lenses. It is a good idea to provide a function that makes the spherical surface onto which the image is projected semi-transparent. It is a good idea to make it semi-transparent by pressing a button on the controller. The origin position and the radius of the sphere can be changed by moving the left and right sticks of the controller up and down. It is recommended that thumbnail displays, map displays, acceleration graphs, etc. be placed as independent panels in the three-dimensional space inside the sphere. It is also recommended that playback control functions such as play, stop, and next event can be performed using a controller.

[0111] (44) It is preferable that the system is installed in a vehicle and the video data for the display is processed so that the system operates based on audio data transmitted from outside the vehicle.

[0112] In this way, the system installed in the vehicle can be operated based on the voice data transmitted from outside the vehicle.

[0113] It is a good idea to operate a system like a dashcam from a smartphone or other device in the vehicle. If the vehicle is near your home and has Wi-Fi or other signals, you can operate it by sending voice data from a smartphone, smart speaker, communication robot, dedicated device, etc. in your home. By saying "Play the dashcam footage" to a voice recognition device (smartphone, smart speaker, communication robot, etc.), it will turn on the dashcam, stream the footage from "your" dashcam wirelessly via Wi-Fi or other means, and play that footage on a TV. If the vehicle's engine is off, it is a good idea to run it on battery power.

[0114] If you say "Play the dashcam footage from yesterday," it will search for yesterday's dashcam footage or the storage device (smartphone or PC) that stores data transferred from the dashcam, and if it is available, it will play it. If not, you can say "No dashcam footage from yesterday was found." It is also good to play the footage from the nearest dashcam instead, such as "I will play the dashcam footage from the day before yesterday instead." If you say "Play the dashcam footage from when I just went out," it will play the footage from the most recent dashcam.

[0115] When playing back dashcam footage, it is a good idea to play the audio recorded along with the dashcam footage, but it would also be good to have a function to play other music as background music. If you say "Set the background music to XX," it would change to the dashcam audio and play that song along with the video on a music streaming service such as Spotify. It would be good to play a digest of the part where an event is occurring. It would be good to be able to specify the playback time, such as "play the dashcam footage for about 5 minutes." In this case, it would be good to play the footage all at once at the specified time. It would be good to prioritize unplayed footage when playing back the footage.

[0116] When the dashcam recognizes the voice command "Keep this video" while the video is playing, the video file being played will be transferred from the dashcam to a storage device (smartphone, PC, etc.) via Wi-Fi or other means. You can also upload the video to a video sharing social networking site and share it.

[0117] When the dashcam recognizes a voice saying "Keep this video" while recording video, the dashcam can record the video using the voice command as a trigger, with the recorded video from a specified time before that point and the video data from that point to a specified time ahead being transferred to a storage device (such as a smartphone or PC) via WiFi or other means.

[0118] When the dashcam recognizes a voice saying "Send video" while recording video, it is recommended that the recorded video from a specified time before that point and the video data from that point onwards be sent to a server, etc. via LTE, etc. It is also recommended that this be uploaded to a video sharing SNS and shared. When the dashcam recognizes a voice saying "Stop video sending," it is recommended that the sending of video to a server, etc. be stopped. When the dashcam recognizes a voice saying "Send video up to this point" while recording video, it is recommended that the recorded video from a specified time before that point be uploaded to a video sharing SNS and shared. When the dashcam recognizes a voice saying "Start live streaming" while in front of the dashcam, it is recommended that live streaming of the dashcam video be started.

[0119] (45) When there are multiple images taken in a vehicle, it is preferable to process the image data for the above display so that the display mode of the thumbnail images changes for each image.

[0120] By doing this, when there are multiple videos, the display mode of the thumbnail images can be changed for each video.

[0121] It would be good to have a function to change the display method and display range of thumbnails for each individual thumbnail. For example, it would be good to have a pop-up menu that allows you to select how to display the clicked thumbnail when you right-click on an individual thumbnail. The types of display should include, for example, fisheye, panoramic, enlarged flat, ring, and dome displays. It is also recommended to provide original, twin cut, panoramic, and angle cut options. Left-clicking on an individual thumbnail allows you to change the display range of the image. It is recommended to provide "180 degree rotation" and "90 degree rotation" options in the right-click menu. For two cameras, it is recommended to provide a "camera switch" option.

[0122] It is recommended that the system has a function to save the changed thumbnail display method and display range, and that it is configured to maintain the state of any changes made. It is also recommended that the system has a function to undo and redo changes to the thumbnail display method and display range. It is also recommended that the system has a function to align the thumbnail display method and display range between thumbnails. It is recommended that left-clicking while holding down the shift key will draw a selection frame around the thumbnail. When there are multiple selection frames, right-clicking on any thumbnail will change the display method for all thumbnails with selection frames to the selected method in the pop-up menu that appears. Also, when there are multiple selection frames, left-dragging any thumbnail will change the state of all thumbnails with selection frames in real time within the range displayed by that thumbnail. The selection frame (selection state) should be released when the left-click on a part other than the thumbnail is released. It is also recommended that rows be provided to display thumbnails that are considered to be from the same time (for example, the same frame) but have different display methods or display ranges. For example, with two 360-degree cameras, a first row of thumbnails would display a 360-degree fisheye view of the front and a second row of thumbnails would display a 360-degree fisheye view of the rear, with thumbnails from the same time displayed in the same column. It would be ideal to provide a function that allows for arbitrary expansion of thumbnail rows. To expand thumbnails, create a space between the rows in the thumbnail display area, right-click that space, and select "Insert Row" from the pop-up menu that appears. This pop-up menu should also include options such as "Delete Upper Row" and "Delete Lower Row." A scroll bar should be provided to allow scrolling of rows or columns that exceed the display range of the screen. For example, with two 360-degree cameras, 10 rows of thumbnails could be created, and the following displays would be possible: front fisheye view, rear fisheye view, panoramic view centered on the front, panoramic view centered on the rear, enlarged flat view centered on the bottom, enlarged flat view with the top centered, ring view with the front as the most visible area, ring view with the rear as the most visible area, dome view with the front as the most visible area, and dome view with the rear as the most visible area. The same column will display these types of information at the same time.You can also scroll left and right to move the columns left and right and see all the thumbnails.

[0123] It is recommended that a function be provided to play the image of a thumbnail when the thumbnail is selected by double-clicking the left mouse button, and that playback begin in the display method and display range set for the thumbnail. A "Set this state as thumbnail" option should be provided in the pop-up menu that appears when you right-click in the playback window, and when this option is selected, the image in the display method and display range at that time should be set as the thumbnail for that image. Thumbnail display involves providing an area to display the original image corresponding to the right-clicked thumbnail, and a function to clearly indicate the display area of ​​the current thumbnail within that original image (by surrounding it with a line, filling it with semi-transparent paint, etc.). This makes it easy to see which part of the original image is set as the thumbnail.

[0124] In relation to "improving image quality and stitching accuracy," it would be good to have a function that clearly shows "the parts of the images that are joined together." For example, it would be good to draw a red line in the stitched parts. As stitching accuracy improves, the joins become less visible. Seams are not desirable in commercial photography, but if they become less visible, there is a risk of cropping across the seam. This could be a problem if a broken part in the join is discovered after delivery, such as after commercial printing, but this can prevent such problems from occurring.

[0125] In relation to "reducing reflections of the camera body," it is recommended to provide a "function to clearly indicate the reflected parts of the camera body." For example, it is recommended to use a shaded or semi-transparent fill in the area corresponding to the camera body. In relation to "automatic face recognition and blurred shooting," it is recommended to provide an area on the viewer screen to indicate whether the image is blurred or not. It is recommended to display an icon indicating that the video being played is a blurred image. It is also recommended to display an icon indicating that the image is blurred in the list of file names. In relation to "wireless live streaming," it is recommended to provide a function to record live streaming as well as video, store information that identifies which recorded video is being streamed, and display whether the recorded video is live streaming or not when playing it in the viewer. It is recommended to simply record live streaming and non-live streaming files separately and store information indicating whether the video is live streaming in the file properties. In particular, it is recommended to clearly record the live streaming area and non-live streaming files within a single file and display the two areas clearly on the timeline. In relation to "time-lapse photography," it is advisable to record the footage with information identifying it as time-lapse photography, and to provide a function that indicates whether it was time-lapse photography when playing it back in the viewer. It is also advisable to provide a function that handles a series of time-lapse photography videos as a single collection. Regarding "Smart Devices," it is advisable to provide a function that records the footage by adding information about which device the recording start, recording stop, and setting instructions came from, and to provide a function that displays this information in the viewer. Regarding "Smart Devices," it is also advisable to provide a function that uses voice recognition to operate the parts equivalent to the viewer's buttons, particularly a function that changes the display method and display range. It is also advisable to provide a function that changes the recording method and recording range using voice recognition.

[0126] (46) When communicating video data representing images captured in a vehicle using a device installed in the vehicle, it is advisable to process the video data for the display so that the communication is performed using a different communication method in addition to changing the communication speed depending on the type of communication contract.

[0127] This allows communication to be performed using different communication methods depending on the type of communication contract.

[0128] For example, a drive recorder for corporate use should have a function that allows it to accommodate various communication fee plans depending on how the company uses its vehicles.

[0129] Examples of desirable features for a low-speed, flat-rate, unlimited plan costing 300 to 500 yen per month include the following. For example, a SIM card with a data rate of around 200 kbps, which is not capable of high-speed communication, would be ideal. 200 kbps allows for audio transmission and still images (JPEG) to be sent at a speed faster than real time. Even mono 64 kbps provides sufficient sound quality to determine whether the sound is from an accident. Therefore, past data can be transmitted in a shorter time than real time. A function that compresses and transmits past data at a bitrate calculated from the current (most recent) communication bitrate or the SIM's contracted bitrate, and then transmits the data at that bitrate, would be ideal. Furthermore, a function that transmits past data as a compressed file and current data via streaming (IP phone is also acceptable). Furthermore, a SIM card with a data limit of up to a certain amount of data per period, such as 366 MB over three days, would be ideal. For example, a function that displays the remaining transfer capacity on the dashcam and viewer would be ideal. As an example, it would be good to display 10MB of 366MB of usage (356MB remaining until December 26th). When you are approaching your data limit, it is a good idea to limit the type of information you send. For example, you could change it so that no photo data is sent and only audio data is sent. For a low-speed SIM card that is capable of burst transfers (burst function: the first 1-2 seconds will produce a few Mbps), you could start sending when an event occurs by sending several photos (even just one photo) from different points in time related to the event, and then send the audio data. It is good practice to transfer photos in bursts.

[0130] The following features are recommended for nighttime plans. One example of a "nighttime plan" is one in which download and upload speeds are limited to a maximum of 200 kbps between 6:00 and 22:00, but high-speed communication is possible at night, with download speeds of up to 375 Mbps and upload speeds of up to 50 Mbps. It is recommended that the plan be equipped with a function that allows users to set times when high-speed communication is possible or when communication restrictions are imposed, and upload videos during times when high-speed communication is possible. In particular, it is recommended that the plan be equipped with a function that automatically turns on the power of dashcams in vehicles parked with the engine off during times when high-speed communication is possible, and begins transmitting video data.

[0131] The following functions are recommended for use with an upstream-only plan. One example of an "upstream-only plan" is one that limits the downstream data communication speed to a low maximum of 200 kbit / s, but ensures a wide upstream data communication speed of up to 50 Mbit / s for transmitting data. Since it is likely that downstream IP phone bandwidth can be secured, it is recommended that the server be equipped with a function that limits communication to instruction commands and downstream IP phone communication. In other words, even if a dashcam has a function for receiving video footage from other dashcams, it is recommended that this function be restricted.

[0132] The following features are recommended for corporate sharing plans: For corporate plans in particular, it would be good to have a data sharing plan where, for example, 50GB can be shared among 20 lines.

[0133] It is recommended that the system be equipped with a function to scrape or obtain information from the management screen provided by the SIM provider via API, count SIM usage on the dashcam, retrieve this information at the center, and display it in the viewer. It is also recommended that the system be equipped with a function to display comparative information. It is also recommended that the viewer display the total packet usage and remaining available capacity of dashcams that share packets. It is also recommended that the viewer display alerts when remaining capacity is low and display graphs of predicted capacity usage. It is also recommended that the viewer display rankings of individual packet usage and remaining available capacity of dashcams that share packets. For example, by looking at the event status of a vehicle with a top-ranked dashcam, it is easy to determine whether the driver is driving poorly or whether the dashcam settings are incorrect. For companies that have set up multiple sharing groups, it is recommended that the system be equipped with a function to compare usage status between sharing groups. This can be used as a reference for optimizing the allocation of SIMs for each dashcam to each group. Automatic analysis using AI or other methods can also be used to advise users to move specific SIMs to other groups. It is a good idea to display the file size in the selection field for what the viewer displays.

[0134] If you have an SMS-compatible SIM, it is recommended that you do the following: When a trigger occurs, it is recommended that you provide a function that generates a string that includes information about the level of maximum G and other information about the importance of the trigger factor, and sends it as an SMS to the center phone number.The center (e.g., a viewer) should decide whether or not to send a transmission request instruction to the dashcam based on the SMS data.The center should then decide information such as the type of data to be sent (such as video) and the transmission range, and send a transmission request instruction to the dashcam.

[0135] In the case of a SIM that supports voice calls, it is recommended to do the following. It is recommended that the dashcam be equipped with a voice call function, and that the center be equipped with a function to forward external calls that come in to the center (office). The center should obtain the driving status of the dashcam via LTE, etc., and have a function that does not forward external calls when set conditions are met, such as when the dashcam is in a driving state. It is also recommended that the dashcam be equipped with a function that calls the center using the appropriate method based on the driving status, such as the price comparison between IP and voice calls, the person being called, and past driving history. It is recommended that the dashcam be equipped with a function that calls by voice when a strong impact occurs, and by IP phone when a weak impact occurs, and that calls to the center be made by VoIP and external calls be connected by phone. It would be a good idea to provide a function for this.

[0136] When using both public Wi-Fi and LTE, it is recommended to do the following: Provide a connection function to the public Wi-Fi, send restricted data via LTE, and send unrestricted or loosely restricted video data via the public Wi-Fi. It is also recommended to provide a function to upload data from free Wi-Fi at convenience stores, etc.

[0137] (47) It is preferable to obtain information about the posture of the device held by the user's hand and process the image data for the display so as to change the display range of the image data based on the information about the posture. In this way, the user can change the display range of the captured video data by changing the orientation of the device held in their hand. In particular, the device may be a smartphone, the video may be from a celestial camera, and when the smartphone is tilted or rotated based on information from the smartphone's gyro or G sensor about the viewpoint of the celestial camera, the viewpoint of the video may be switched to that direction. (48) It is preferable that the program be a computer-readable program that controls the computer of the system described in any one of (1) to (47) above.

[0138] By processing the video data for display, we can provide something better than ever before.

[0139] The displayable system may be, for example, a personal computer, a smartphone, a drive recorder, etc. The processing of the video data for display may be, for example, processing for playing the video, processing for transmitting the video data, processing for adjusting the brightness of the video, processing for changing the video playback format, processing for reducing the amount of video data, etc.

[0140] (49) The present invention may also be a control method for a system capable of displaying images captured in a vehicle, the control method being a control method for a system equipped with means for processing image data for the display.

[0141] By processing the video data for display, we can provide something better than ever before.

[0142] The displayable system may be, for example, a personal computer, a smartphone, a drive recorder, etc. The processing of the video data for display may be, for example, processing for playing the video, processing for transmitting the video data, processing for adjusting the brightness of the video, processing for changing the video playback format, processing for reducing the amount of video data, etc.

[0143] The inventions shown in (1) to (47) above can be combined in any way. For example, it is possible to combine all or part of the configuration of the invention shown in (1) with at least part of the configuration of at least one invention from (2) onwards. In particular, it is preferable to combine the invention shown in (1), (48) or (49) with at least part of the configuration of at least one invention from (2) onwards.

[0144] In addition, any configuration may be extracted from the inventions set forth in (1) to (47) and the extracted configurations may be combined. The applicant of this application intends to obtain rights to inventions including these configurations. [Effects of the Invention]

[0145] According to this invention, by processing video data for display, it is possible to provide something better than before.

[0146] The effects of the present invention are not limited to this, and the effects achieved by the configurations disclosed in the present specification and drawings are also disclosed, and the applicant intends to obtain rights to the configurations that achieve these effects by filing divisional applications, amendments, etc. For example, in the present specification, it is stated that "can The above-mentioned parts are descriptions that clearly show the effect that can be achieved, and there are parts that show the effect even without the description "can...". Also, there are effects that can be understood from the configuration even without such a description. [Brief explanation of the drawings]

[0147] [Figure 1] This is an oblique view of the drive recorder from the rear. [Figure 2] This shows the interior of a vehicle equipped with a drive recorder. [Figure 3] FIG. 2 is a block diagram showing the electrical configuration of the drive recorder. [Figure 4] 10 is a time chart of the event recording video data and the transmission video data. [Figure 5] 10 is a flowchart showing a transmission processing procedure. [Figure 6] 10 is an example of a recording setting window. [Figure 7] FIG. 1 is a block diagram showing the electrical configuration of a personal computer. [Figure 8] 10 is a flowchart showing the processing procedure of a personal computer. [Figure 9] 10 is a flowchart showing a playback processing procedure. [Figure 10] 10 is a flowchart showing a brightness correction process procedure. [Figure 11] 10 is a flowchart showing a brightness stop process procedure. [Figure 12] 1 is an example of a playback window. [Figure 13] 1 is an example of a playback window. [Figure 14] 10 is an example of an image before brightness correction. [Figure 15] 10 is an example of an image after brightness correction. [Figure 16] 10 is an example of an image before brightness correction. [Figure 17] 10 is an example of an image after brightness correction. [Figure 18] 10 is an example of an image before brightness correction. [Figure 19] 10 is an example of an image after brightness correction. [Figure 20] 10 is an example of a brightness correction curve. [Figure 21] 10 is an example of a brightness correction curve. [Figure 22] 10 is an example of a processing procedure for displaying another window. [Figure 23] 1 is an example of a playback window. [Figure 24] 1 is an example of a playback window. [Figure 25] 10 is an example of a processing procedure for displaying a video on a separate display screen. [Figure 26] 10 is an example of a playback window and a video display window displayed on separate display screens. [Figure 27]The figure shows how to use a playback window and a video display window that are displayed on a separate display screen. [Figure 28] 10 is a flowchart illustrating an example of a playback process procedure. [Figure 29] The relationship between vehicle speed, playback speed, and video time is shown. [Figure 30] 10 is a flowchart illustrating an example of a playback process procedure. [Figure 31] 10 is an example of a flowchart illustrating a video analysis processing procedure. [Figure 32] 10 is an example of a flowchart illustrating a video analysis processing procedure. [Figure 33] 10 is a flowchart illustrating an example of a playback process procedure. [Figure 34] 10 is a flowchart illustrating an example of a playback process procedure. [Figure 35] 1 is an example of a flowchart showing a processing procedure of a personal computer. [Figure 36] 1 is an example of a playback window. [Figure 37] 1 is an example of a playback window. [Figure 38] 1 is an example of an acceleration graph. [Figure 39] 1 is an example of a playback window. [Figure 40] 1 is an example of a playback window. [Figure 41] 1 is an example of a flowchart showing a processing procedure of a personal computer. [Figure 42] 10 is an example of a flowchart illustrating a video analysis processing procedure. [Figure 43] 1 is an example of a flowchart showing a processing procedure of a personal computer. [Figure 44] 1 is an example of a playback window. [Figure 45] 10 is an example of a flowchart showing a first skip playback processing procedure. [Figure 46] 10 is an example of a flowchart showing a second skip playback processing procedure. [Figure 47]10 is a flowchart illustrating an example of a transmission process procedure. [Figure 48] 10 is an example of a flowchart illustrating a ranking display process procedure. [Figure 49] 10 is an example of a ranking display. [Figure 50] 10 is an example of a flowchart showing a recording process procedure. [Figure 51] 10 is an example of a flowchart showing a recording process procedure. [Figure 52] 10 is a flowchart illustrating an example of a playback process procedure. [Figure 53] 10 is an example of a flowchart showing a videophone recording process procedure. [Figure 54] 10 is a flowchart illustrating an example of a playback process procedure. [Figure 55] 10 is an example of a flowchart showing a recording process procedure. [Figure 56] 10 is a flowchart illustrating an example of a playback process procedure. [Figure 57] 10 is a flowchart illustrating an example of a transmission process procedure. [Figure 58] 10 is a flowchart illustrating an example of a playback process procedure. [Figure 59] 10 is a flowchart illustrating an example of a processing procedure of a smartphone. [Figure 60] 10 is a flowchart illustrating an example of a processing procedure between a playback device and a robot. [Figure 61] 1 is an example of a flowchart showing a processing procedure of a virtual reality device. [Figure 62] 10 is a flowchart illustrating an example of a processing procedure between a drive recorder, a smartphone, and a television device. [Figure 63] 10 is an example of a processing procedure for displaying thumbnail images. [Figure 64] 10 is an example of thumbnail image display. [Figure 65] (A) to (E) are examples of thumbnail images. DETAILED DESCRIPTION OF THE INVENTION

[0148] [First Example] Figures 1 to 6 show a first embodiment, which is a drive recorder.

[0149] FIG. 1 is a perspective view of a drive recorder 1 (an example of a system) seen diagonally from the rear. An SD card slot 10 is formed on one side of the housing of the drive recorder 1. A display 11 is formed over almost the entire back of the housing of the drive recorder 1, and multiple operation buttons 12 are provided on both the left and right sides of the display 11. A joint rail 13 is provided on the top surface of the drive recorder 1. Although not shown in FIG. 1, a camera lens is provided on the front of the drive recorder. A DC jack for the drive recorder 1 is formed on a side not shown in FIG. 1, and a speaker and HD (high definition) output terminal are formed on the bottom of the drive recorder 1.

[0150] The camera, including the lens, captures, for example, images of the area in front of the vehicle. It may also capture images of the interior and exterior of the vehicle (the area in front of the vehicle). The DC jack is a jack for connecting to a DC power source via a power cable. The SD card slot 10 is an insertion port for inserting an SD card (an example of a memory card; it may be an xD picture card or other memory card, or a USB (Universal Serial Bus) memory or other memory card). The speaker outputs sound and audio. The HD output terminal is a terminal for connecting to other information devices via a cable. The joint rail 13 is for attaching a joint for mounting the drive recorder 1 on a vehicle. The display 11 displays various images, including images captured by the camera of the drive recorder 1. The operation buttons 12 are operated by the user to input various commands to the drive recorder 1.

[0151] Figure 2 shows a view looking forward from inside the cabin of a vehicle equipped with a drive recorder 1.

[0152] A rearview mirror 4 is provided near the center of the upper part of the vehicle's windshield 3. A drive recorder 1 is fixed to the vehicle by a joint at a position adjacent to the rearview mirror 4 on the passenger side (on the left side as viewed from the front in Figure 2).

[0153] The DC jack of the drive recorder 1 is connected to the cigarette lighter socket 5 via the power cable 6. When the vehicle's accessory power is turned on, power is supplied to the drive recorder 1 from the cigarette lighter socket 5.

[0154] FIG. 3 is a block diagram showing the electrical configuration of the drive recorder 1. As shown in FIG.

[0155] The drive recorder 1 includes a controller 20. This controller 20 includes a CPU (Central Processing Unit) 20a that controls the overall operation of the drive recorder 1, a ROM (Read Only Memory) 20b that stores the operating program of the drive recorder 1, a RAM (Random Access Memory) 20c that temporarily stores data, etc., and a timer 20d. As will be described later, the controller 20 has functions such as a GPS information processing program, an image processing program, and a communication processing program.

[0156] The drive recorder 1 also includes a GPS (Global Positioning System) receiver 16, a camera 17, an SD card reader / writer 14, and an acceleration sensor 18. The GPS receiver 16 detects the position of the drive recorder 1, thereby determining the position of the vehicle in which the drive recorder 1 is installed. The camera 17, as described above, captures images of the area in front of the vehicle in which the drive recorder 1 is installed. When the SD card 23 is inserted into the SD card slot 10, the SD card reader / writer 14 reads data recorded on the inserted SD card 23 and writes the data to the SD card 23. The acceleration sensor 18 detects the up / down, left / right, and front / rear accelerations applied to the drive recorder 1, thereby determining the up / down, left / right, and front / rear accelerations of the vehicle in which the drive recorder 1 is installed.

[0157] The GPS data output from the GPS receiver 16 indicating the position of the drive recorder 1, the video data representing the video captured by the camera 17, and the signal representing the acceleration detected by the acceleration sensor 18 are each input to the controller 20. By executing a GPS information processing program, the controller 20 can record the GPS data output from the GPS receiver 16 on an SD card 23 inserted in the SD card slot 10. In addition, by executing a video processing program, the controller 20 can record the video data obtained from the camera 17 on the SD card 23 in association with the time.

[0158] Furthermore, the drive recorder 1 includes a speaker 15 for outputting the above-mentioned audio, a display 11 for displaying images, an operation button 12, a communication circuit 19, and an LTE module 21.

[0159] When the controller 20 outputs audio data etc. to the speaker 15, audio etc. is output from the speaker 15, and when the controller 20 outputs video data etc. to the display 11, video etc. is displayed on the display 11. Various commands from the operation buttons 12 are input to the controller 20.

[0160] The communication circuit 19 functions as a communication means for wirelessly communicating with external devices such as a server, a personal computer, a smartphone, a tablet terminal, etc. By executing the communication processing program, the device has the function of transmitting data such as video to an external device via the communication circuit 19. The communication circuit 19 may be one that complies with short-range wireless communication standards such as the WiFi standard or Bluetooth (registered trademark). The short-range wireless communication standard can be applied to communication between an external device and a work vehicle (such as a forklift) operating within a premises, for example. The mobile communication system standard can be applied to communication between an external device and a vehicle moving within a wider area, for example.

[0161] The LTE (Long Term Evolution) module 21 is a communication circuit that complies with the mobile phone communication standard. A communication circuit that complies with the standards of other mobile communication systems, such as 4G, may be used instead of the LTE module 21. The LTE module 21 can transmit video data captured by the camera 17 to an external server, etc.

[0162] As will be described later, the operating program of the drive recorder 1 may be pre-recorded on the SD card 23, or may be received by the LTE module 21 via the Internet or the like and recorded on the SD card 23, or the SD card 23 may be inserted into a personal computer or the like, connected to the Internet via the personal computer, downloaded from the personal computer, and recorded on the SD card 23.

[0163] Figure 4 is an example of a time chart when an event occurs in a vehicle equipped with the drive recorder 1.

[0164] In this embodiment, continuous recording and event recording are possible. Continuous recording involves, for example, continuing recording while the drive recorder 1 is powered on until the continuous recording area on the SD card 23 is full. If the setting allows overwriting of continuous recording video data, when the continuous recording area is full, the video data obtained by continuous recording is overwritten in the continuous recording area. Event recording involves temporarily storing video data obtained by shooting in RAM 20c or the like in sequence for a fixed period of time, such as 20 seconds, and repeatedly overwriting the data unless an event occurs. When an event occurs, the video data temporarily stored in RAM 20c or the like is read and recorded in the event recording area on the SD card 23. When it is determined that a change has occurred in the vehicle in which the drive recorder 1 is installed (for example, detection of acceleration above a threshold, detection of speed above a threshold, or detection of sudden braking), and when it is determined that an instruction to perform event recording has been given to the drive recorder 1 (for example, an instruction from the operation button 12), an event is generated from the controller 20 and event recording is performed.

[0165] In this embodiment, when an event occurs, video data obtained by capturing images with camera 17 during the period before and after the event is recorded on SD card 23 (a recording medium, which is an example of a portable recording medium), and is transmitted to a server, etc. by LTE module 21. In particular, in this embodiment, the amount of video data to be transmitted (amount of data per unit time) transmitted by LTE module 21 to a server, etc. is reduced as time passes from the time the event occurred.

[0166] When an event (which is an example of a recording command) occurs at time t3 (YES in step 41 in FIG. 5), video data temporarily stored in RAM 20c from time t1 (for example, 10 seconds before time t3) before time t3 when the event occurred is read and recorded in the event recording area of ​​SD card 23 as video data for event recording. In addition, video data obtained by shooting during the period from time t3 after time t3 to time t5 (for example, 10 seconds after time t3) is also recorded in the event recording area of ​​SD card 23 as video data for event recording. Video data shot during the period from time t1 to t5 (for example, 20 seconds) before and after time t3 when the event occurred is recorded in the event recording area of ​​SD card 23 as video data for event recording. The video data for event recording is, for example, video with a high frame rate and relatively high image quality, with a cycle of 30 frames per second, and each frame is recorded in high definition (HD) format (for example, with an effective vertical resolution of 1080 lines).

[0167] Furthermore, when an event occurs at time t3, the video data temporarily stored in RAM 20c from time t1, which is before time t3 when the event occurred, is read, and from time t1 to time t2, which is after time t1, the controller 20 generates video data with a reduced data volume compared to the video data temporarily stored in RAM 20c. The LTE module 21 transmits the reduced video data as video data for transmission to a server, etc. From time t2 to time t3 when the event occurred, the LTE module 21 transmits the same video data as the event recording video data as video data for transmission to a server, etc. From time t3 to time t4, the same video data as the event recording video data is generated from video data obtained by shooting with camera 17, and the LTE module 21 transmits the generated video data as video data for transmission to a server, etc. From time t4 to time t5, as from time t1 to time t2, the controller 20 generates video data with a reduced data volume compared to the event recording video data, and the LTE module 21 transmits the reduced video data as video data for transmission to a server, etc. The amount of video data for the period from time t2 to t4, which is before and after time t3 when the event occurred, will be relatively large, while the amount of video data representing the period from time t1 to t2 and the period from time t4 to t5 (e.g., one frame per second, where one frame is preferably represented by the number of pixels based on VGA (Video Graphics Array)) will be relatively small. From time t2 to t4, the video data to be transmitted does not necessarily have to have the same amount of data (amount of data per unit time) as the event recording video data, as long as the amount of video data to be transmitted is greater than the amount of video data to be transmitted from time t1 to t2 or from time t4 to t5. Furthermore, the amount of video data to be transmitted from time t1 to t2 may be different from the amount of video data to be transmitted from time t4 to t5. Furthermore, the amount of video data to be transmitted from time t2 to t3 may be different from the amount of video data to be transmitted from time t3 to t4.

[0168] Video footage before and after an event may show an accident or similar and is considered relatively important. Video footage from the period before and after an event has a large amount of data, so it is possible to transmit footage of important scenes and store the important footage on a server, etc. Video data from a certain period after the event and from a certain period before the event has a reduced data volume, so transmission time can be shortened and, if costs are incurred for data transmission, costs can be reduced.

[0169] In the example shown in Figure 4, data before and after the occurrence of an event is generated as video data to be transmitted, but data before or after the occurrence of an event may also be generated as video data to be transmitted. Furthermore, data a certain time after or a certain time before the occurrence of an event may also be generated as video data to be transmitted. Furthermore, although the video period represented by the data to be transmitted and the video period represented by the video data for recording an event are the same, they do not have to be the same. In any case, it is sufficient to reduce the amount of data for which time has passed since a recording command was issued, such as when an event occurred.

[0170] FIG. 5 is a flowchart showing the transmission process procedure of the drive recorder 1.

[0171] When an event occurs as described above (step 41), the controller 20 reads out the video data temporarily stored in the RAM 20c, and the controller 20 executes the event as described above. The controller 20 records the generated event recording video data in the event recording area of ​​the SD card 23 (step 42).

[0172] The event recording video data recorded on the SD card 23 is read, and from the read event recording video data, the controller 20 generates, as video data to be transmitted, low-data-volume video data representing video from the periods from time t1 to time t2 and from time t4 to time t5, and high-data-volume video data representing video from time t2 to time t4 (step 43) (this is an example of a process for reducing the amount of video data to be transmitted compared to the amount of video data recorded on the recording medium). The LTE module 21 transmits the generated video data to a server or the like (step 45).

[0173] 5, the transmission video data is generated from the event recording video data, but the event recording data and the transmission video data may be generated in parallel. For example, during the period from time t1 to time t3 when the event occurred, the event recording video data and the transmission video data may be generated simultaneously using video data temporarily stored in RAM 20c, and after time t3 when the event occurred, the event recording video data and the transmission data may be generated simultaneously using video data captured by camera 17.

[0174] FIG. 6 shows an example of a recording setting window for setting the amount of video data to be transmitted, etc.

[0175] As explained with reference to Figures 4 and 5, in addition to continuous recording, event recording and video data transmission are also performed. The user sets continuous recording, event recording, video data transmission, etc. using the recording setting window 30 shown in Figure 6.

[0176] The SD card 23 to be inserted into the drive recorder 1 is inserted into, for example, an SD card slot of a personal computer (not shown), and the recording setting program recorded on the SD card 23 is read out. Then, the recording setting window 30 shown in Figure 6 is displayed on the display screen of the personal computer.

[0177] The recording setting window 30 includes a recording method selection frame 30a, a recording resolution selection frame 30b, a continuous recording frame rate selection frame 30c, an event recording frame rate selection frame 30d, an event recording time selection frame 30e, a transmission setting frame 30f, a first transmission resolution setting frame 30g, a first transmission frame rate setting frame 30h, a second transmission resolution setting frame 30i, a second transmission frame rate setting frame 30j, and an SD card save instruction button 30k. The recording setting window 30 also includes other frames, buttons, etc., but their explanations will be omitted.

[0178] The recording method selection frame 30a is a frame for setting the type of recording. By pulling down, a menu appears that allows the user to set the type of recording, such as continuous recording only, or both continuous recording and event recording, and the user selects the desired type of recording. The recording resolution selection frame 30b is a frame for setting the resolution of one frame for continuous recording. By pulling down, a menu appears that allows the user to set various resolutions, and the user selects the desired resolution. When event recording is also set, the resolution of one frame for event recording is set to the same as the resolution of one frame for continuous recording. Of course, the resolution for event recording may be set to be higher than the resolution for continuous recording. The continuous recording frame rate selection frame 30c is a frame for setting the number of frames per second (frame rate) for continuous recording. By pulling down, a menu appears that allows the user to set various frame rates, and the user selects the desired resolution. The event recording frame rate selection frame 30d is a frame for setting the number of frames per second (frame rate) for event recording. By pulling down the menu, various frame settings are available, and the user can select the desired resolution.

[0179] The event recording time selection frame 30e is a frame for setting the time to record when an event occurs. By pulling down, a menu appears that allows the user to set various times before and after the event occurrence, and the user can set the desired time. It may be possible to set the time to record before or after the event occurrence, or to set the recording time from a certain time after the event occurrence, or to set the recording time going back a certain time before the event occurrence. The transmission setting frame 30f is a frame for setting whether to generate video data for transmission and transmit it to a server, etc., as described above. By pulling down, a menu of "enable" or "disable" appears. "enable" is selected when transmitting video data for transmission, and "disable" is selected when not transmitting video data for transmission. The first transmission resolution setting frame 30g is a frame for setting the resolution of video data for transmission for the period from t2 to t4 around the event occurrence, which sandwiches the event occurrence time t3, as described with reference to Figure 4. By pulling down, a menu of various resolutions appears, and the user can select the desired resolution. As explained with reference to FIG. 4, the first transmission frame rate setting frame 30h is a frame for setting the number of frames per second (frame rate) of video data to be transmitted during the period from t2 to t4 around the event occurrence time t3. By pulling down, a menu showing various frame rates appears, and the user can select the desired frame rate. The second transmission resolution setting frame 30i is a frame for setting the resolution of video data to be transmitted after (for example, the period from t4 to t5) or before (for example, the period from t1 to t2) a time distant from (before a point in time that has passed or gone back) the event occurrence time t3. By pulling down, a menu showing various resolutions appears, and the user can select the desired resolution. The second transmission frame rate setting box 30j is a box for setting the number of frames per second (frame rate) of video data to be transmitted after (for example, the period from time t4 to t5) or before (for example, the period from time t1 to t2) a time distant from (before a point in time that has passed or gone back) the event occurrence time t3. By pulling down, a menu showing various frame rates appears, and the user can select the desired frame rate.The save to SD card button 30k is clicked when the contents set as described above are to be saved to the SD card 23 inserted in the SD slot of the personal computer.

[0180] In this way, the settings made using the recording setting window 30 are saved on the SD card 23, and when the SD card 23 is inserted into the SD card slot 10 of the drive recorder 1, the event recording video data is recorded on the SD card 23 as described above, and the transmission data is generated and transmitted.

[0181] As the number of communication-oriented dash cams increases, sending 4K or 8K video data can increase communication charges. Therefore, as mentioned above, reducing the data volume can reduce communication charges. A dash cam only needs to send video data of accident scenes or dangerous driving scenes, and does not need to send video of normal scenes. When sending video data of an event, sending only the video data of the event itself results in a large amount of data, so further reduction in data volume is considered. To achieve this, it is effective to send high-volume video data to ensure clear video before and after the event, and low-volume, rough video data at other times. An event occurs when the vehicle's acceleration exceeds a threshold, and 20 seconds before and after the event are recorded on the SD card. To avoid the large amount of data required for immediate communication, only high-quality video before and after the event and VGA video for scenes where an accident likely occurred can be transmitted, significantly reducing the amount of data transmitted. Continuous recording is performed continuously, while event recording records 20 seconds before and after the event, for example. Continuous recording overwrites the oldest video data. There are three ways to reduce the amount of data sent over the network: reducing the frame rate, reducing the resolution, and increasing the compression rate.

[0182] Some drive recorders allow separate settings for event recording and continuous recording, while others do not. One type of video data with a large amount of data can have an effective vertical resolution of 1080 lines, while the other type of video data with a small amount of data can be set to VGA. Alternatively, two cameras can be used to shoot, with one camera obtaining large data with an effective vertical resolution of 1080 lines, and the other camera obtaining small data with VGA. The transmitted data can also be recorded on SD card. Video data for transmission can also be generated from video data recorded on the SD card. Continuously recorded data can also be transmitted. Basically, if data is transmitted via communication, it is recorded on the server, so there is no need to record it on the SD card 23 of the drive recorder 1.

[0183] A backup of the transmitted video data may be recorded on the SD card 23. Information (rules) on how the generated video data was transmitted as video data for transmission may be stored in the header of each frame of the video data for transmission or in the header of the SD card 23. Alternatively, data generated according to the rules may be recorded on the SD card 23. Information indicating that video data different from the video data recorded on the SD card 23 of the drive recorder 1 is being transmitted may be stored in the header of the video file storing the transmitted video data, the header of the SD card 23, or the like. It is preferable that video data (raw data) representing clear images be recorded on the SD card 23, or that processed video data be transmitted. It is preferable that information regarding the orientation of the device held by the user's hand be acquired, and the video data for display be processed so as to change the display range of the video data based on the information regarding the orientation. In this way, the user can change the display range of the captured video data by changing the orientation of the device held in their hand. In particular, it is advisable to use a smartphone as the device, the image as an image from a celestial camera, and when the smartphone is tilted or rotated based on information from the smartphone's gyro or G sensor regarding the viewpoint of the celestial camera, the viewpoint of the image should be switched in that direction.

[0184] [Second Example] 7 to 21 show a second embodiment, which relates to the playback of video captured by the drive recorder 1. In this embodiment, the brightness of the video captured by the drive recorder 1 is adjusted (for example, adjusted to be brighter) based on instructions from the user, so that when a video of a dark area (for example, a video of the inside of the vehicle cabin) and a video of a bright area (for example, a video of the outside of the vehicle cabin) are captured in one frame, the amount of brightness adjustment of the video of the dark area is made relatively larger than the amount of brightness adjustment of the video of the bright area based on instructions from the user.

[0185] FIG. 7 is a block diagram showing the electrical configuration of a personal computer (an example of a system) 50.

[0186] As described above, the video recorded by the drive recorder 1 can be played back using the personal computer 50. However, the video may also be played back in the drive recorder 1 without using the personal computer 50.

[0187] The overall operation of the personal computer 50 is controlled by a CPU (Central Processing Unit) 58 .

[0188] The personal computer 50 is provided with a display device 52. The display device 52 is controlled by a display control device 51 that is controlled by a CPU 58. The personal computer 50 also includes a solid state drive (SSD) 53 that is accessed by the CPU 58, a display control device 51 that reads data recorded on an SD card 23, and a display control device 51 that writes data to the SD card 23. Includes a built-in SD card reader / writer 54.

[0189] Furthermore, the personal computer 50 includes input devices 55 such as a keyboard and a mouse, memory 56 such as RAM, and a communication circuit 57 for connecting to a network such as the Internet.

[0190] The operating program, which will be described later, is received by the communication circuit 57 of the personal computer 50 via the Internet and installed in the personal computer 50. The CPU 58 reads and executes the installed program, thereby controlling each part. The operating program may be stored in a recording medium such as the SD card 23, and may be read from such a recording medium and installed in the personal computer 50.

[0191] When the SD card 23 removed from the drive recorder 1 is inserted into the SD card slot of the personal computer 50 and the detection program recorded on the SD card 23 is started, the CPU 58 reads the video data (e.g., continuous recording video data, event recording video data) recorded on the SD card 23 and detects the video portions inside the vehicle cabin and the video portions outside the vehicle cabin. The video portions inside the vehicle cabin and the video portions outside the vehicle cabin can be determined, for example, by edge detection between the windshield 3 and the dashboard, or by image analysis. The CPU 58 records data indicating the boundary between the detected video portions inside the vehicle cabin and the video portions outside the vehicle cabin, for example, in the header of the video file in which the video data is stored. The detection device (an example of a detection means) that detects the video portions inside the vehicle cabin may, for example, accept a selection of an area of ​​the vehicle cabin in the video from the user and detect the selected area as the video portion inside the vehicle cabin, or may, for example, automatically detect the video portion inside the vehicle cabin. The automatic detection process may be, for example, a process that detects the movement of the vehicle and detects a portion of the image where there is a change in the image according to the movement as the outside of the vehicle cabin, and a portion of the image where there is no change in the image according to the movement as the inside of the vehicle cabin. In particular, the process may detect a region in the image where there is movement according to the speed of the vehicle movement as the outside of the vehicle cabin, and a region where there is no movement according to the speed of the vehicle movement as the inside of the vehicle cabin.

[0192] When the playback program installed on the personal computer 50 is started by the user, the display control device 51, under the control of the CPU 58, displays the playback window 100 shown in Figure 12 on the display screen of the display device 52 of the personal computer 50.

[0193] 12, a menu bar 101 is formed at the top of the playback window 100, and a video display area 111 is formed below this menu bar 101. Furthermore, below the video display area 111, a map image display area 121, a vehicle information etc. display area 131, and a play list display area 141 are formed.

[0194] The menu bar 101 displays various items, and by selecting each item, a pull-down menu is displayed, allowing detailed instructions to be given. The menu bar 101 displays the strings "File," "View," "Play," "Tools," and "Information," as well as a folder icon 102, a camera icon 103, a printer icon 104, a reel icon 105, a floppy disk (FD) icon 106, a log icon 107, and a gear icon 108. By pulling down the strings "File," "View," "Play," "Tools," and "Information," a pull-down menu corresponding to each item appears, allowing detailed instructions to be given. Clicking the folder icon 102 specifies a folder, clicking the camera icon 103 converts to a still image, clicking the printer icon 104 prints, clicking the reel icon 105 converts to a video, and clicking the floppy disk (FD) icon 106 displays a digital image. The data is backed up and converted into log data by clicking the log icon 107, and the recording settings are changed by clicking the gear icon 108.

[0195] The video display area 111 is an area that displays video recorded by the drive recorder 1 or the like. The map image display area 121 is an area that displays an image of a map in the vicinity of the location where the video displayed in the video display area 111 was shot. Data representing the vehicle's position is sent from the personal computer 50 to a map server located on the Internet, and data representing a map in the vicinity of the vehicle's position is sent from the map server to the personal computer 50, thereby displaying the map image in the map image display area 121. The vehicle information, etc. display area 131 is an area that displays information such as the vehicle's speed at the time the video was recorded. The vehicle information display area 131 includes a vehicle status display area 132 that displays the status of the vehicle to which the drive recorder 1 that recorded the video displayed in the video display area 111 is attached, an operation button 133 that issues operation commands such as video playback and stop, an interior highlight button (an example of a first instruction input means) 134 that issues an instruction to brighten the video display of the vehicle interior, a normal display button (an example of a second instruction input means) 135 that issues an instruction to stop the bright display of the video display of the vehicle interior and switch to normal display, a time display area 136 that displays the recording time of the video displayed in the video display area 111, and an acceleration graph display area 138 that displays a graph of the vehicle's acceleration. The playlist display area 141 is an area that displays a list of the file names of video files recorded on the SD card 23. The brightness of the vehicle interior can be adjusted based on instructions from the user issued by the interior highlight button (first instruction input means) 134.

[0196] When the folder icon 102 is clicked and a folder containing video files is selected, the file names of the video files stored in that folder are displayed in a clickable manner in the playlist display area 141, as shown in Fig. 12. The video recording start time is used as the file name, but it goes without saying that the file name may be something other than the recording start time.

[0197] FIG. 8 is a flowchart showing the processing procedure of the personal computer 50.

[0198] As described above, when the folder icon 102 is clicked and the desired folder is selected, the file names of the video files are displayed in the play list display area 141. When one of the file names displayed in the play list display area 141 is clicked using the mouse included in the input device 96, the file is selected (step 61). The corresponding video file is read from the SD card 23. Playback of the video data stored in the read video file begins, and the video is displayed in the video display area 111 as shown in FIG. 13 (step 62). Various header data corresponding to the recording time of the video data is recorded in the header recording area of ​​the video file, and the recorded header data is also read when the video at the recording time is displayed, and various superimposition processes are performed on the video depending on the content of the read header data. Furthermore, the map image display area 121 displays an image of a map in the vicinity of the location where the video displayed in the video display area 111 was taken, and the vehicle status display area 132 displays the numerical values, status, etc. of the speed, acceleration (for example, forward / backward acceleration is expressed as an X value, left / right acceleration as a Y value, and up / down acceleration as a Z value) and position (for example, latitude and longitude) of the vehicle to which the drive recorder 1 that recorded the video displayed in the video display area 111 is attached. Also, the acceleration of the vehicle is displayed in a graph form in the acceleration graph display area 138.

[0199] FIG. 9 is a flowchart showing the playback processing procedure.

[0200] When a video file to be played is selected, the playback is performed according to the selected video file, as shown in Figure 13. The first frame of the video represented by the selected video file is displayed as a still image. In this state, when the user presses the play button (right-facing triangular button) included in the operation buttons 133 (YES in step 63), video playback of the video represented by the selected video file begins (step 64). The video is displayed in the video display area 111.

[0201] FIG. 10 is a flowchart showing the brightness correction processing procedure.

[0202] When the user presses the interior highlight button 134 (YES in step 71) while video is being played back in the video display area 111 or while video is being displayed as a still image in the video display area 111, the video data representing the video displayed in the video display area 111 undergoes brightness correction within the position (step 72), and the CPU 58 processes the video data so that the amount of brightness adjustment for the indoor (vehicle interior) video portion is relatively greater than the amount of brightness adjustment for the outdoor (outdoor) video portion. Generally, the interior of the vehicle is darker than the exterior, but adjusting the brightness so that both the indoor and outdoor video portions are uniformly bright can result in the bright outdoor video portion being blown out. In this embodiment, the amount of brightness adjustment for the indoor (vehicle interior) video portion is relatively greater than the amount of brightness adjustment for the outdoor (outdoor) video portion, thereby preventing blown-out highlights in brighter areas such as the outdoor video portion and brighter video can be brightened to make the indoor video portion easier to see.

[0203] Brightness correction can be achieved by applying digital gain processing to brighten the video data in the CPU 58. At that time, the amount of gain that brightens the detected indoor video portion should be greater than the amount of gain that brightens the detected outdoor video portion.

[0204] When the normal display button 135 is pressed (YES in step 73), the process proceeds to the process shown in FIG. 11, which will be described later.

[0205] 14 and 15 are examples of images displayed in the image display area 111, both of which are examples of images taken by the drive recorder 1 during the day.

[0206] FIG. 14 is an example of an image before the above-mentioned brightness correction is performed, and FIG. 15 is an example of an image after the above-mentioned brightness correction is performed.

[0207] 14, image portion 112a on the left side of image 112 is the image portion outside the room, and image portion 112b on the right side of image 112 is the image portion inside the room. Image portion 112a outside the room is relatively bright, while image portion 112b inside the room is relatively dark. There is a difference in brightness between the inside and outside of the room, making image 112 difficult to see.

[0208] Referring to Figure 15, image portion 112a on the left side of image 112 is the outdoor image portion after brightness correction, and image portion 112b on the right side of image 112 is the indoor image portion after brightness correction. Outdoor image portion 112a is slightly brighter than before brightness correction, and indoor image portion 112b is also brighter than before brightness correction. Because the amount of brightness adjustment for indoor image portion 112b is relatively greater than the amount of brightness adjustment for outdoor image portion 112a, the difference in brightness between outdoor image portion 112a and indoor image portion 112b is smaller than before brightness correction, making it easier to see. Furthermore, adjusting the brightness prevents the outdoor image portion from becoming too bright, preventing whiteout in bright areas.

[0209] 16 and 17 are examples of images displayed in the image display area 111, both of which are examples of images taken by the drive recorder 1 at night.

[0210] FIG. 16 is an example of an image before the above-mentioned brightness correction is performed, and FIG. 17 is an example of an image after the above-mentioned brightness correction is performed.

[0211] Referring to Figure 16, image portion 113a on the left side of image 113 is the image portion outside the room, and image portion 113b on the right side of image 113 is the image portion inside the room. Image portion 113a outside the room is slightly bright, but dark because it is nighttime. In contrast, image portion 113b inside the room is quite dark. Image 113 is quite dark overall and difficult to see.

[0212] 17, image portion 113a on the left side of image 113 is the outdoor image portion after brightness correction, and image portion 113b on the right side of image 112 is the indoor image portion after brightness correction. Outdoor image portion 113a is slightly brighter than before brightness correction, and indoor image portion 113b is also brighter than before brightness correction, providing a slight improvement in allowing the interior of the room to be seen more clearly.

[0213] 18 and 19 are examples of images displayed in the image display area 111, and both are examples of images captured by the drive recorder 1 at night.

[0214] FIG. 18 is an example of an image before the above-mentioned brightness correction is performed, and FIG. 19 is an example of an image after the above-mentioned brightness correction is performed.

[0215] Referring to Figure 18, image portion 114a on the left side of image 114 is the image portion outside the room, and image portion 114b on the right side of image 114 is the image portion inside the room. Image portion 114a outside the room is slightly bright, but dark because it is nighttime. In contrast, image portion 114b inside the room is quite dark. Image 114 is quite dark overall and difficult to see.

[0216] 19, image portion 114a on the left side of image 114 is the outdoor image portion after brightness correction, and image portion 114b on the right side of image 114 is the indoor image portion after brightness correction. Outdoor image portion 114a is brighter than before brightness correction, and indoor image portion 114b is also brighter than before brightness correction, making it relatively easy to see what is happening inside the room.

[0217] FIG. 11 is a flowchart showing the brightness correction stop processing procedure.

[0218] When the normal display button 135 is pressed by the user (YES in step 81), the CPU 58 stops the brightness correction (indoor brightness correction) process described above (step 82).

[0219] When the indoor emphasis button 134 is pressed, the process proceeds to the process shown in FIG. 10, where the brightness correction described above is performed.

[0220] In the above-described embodiment, the brightness of the exterior is adjusted, but the brightness of the exterior does not necessarily need to be adjusted if the amount of adjustment of the brightness of the interior (inside the vehicle) image portion is relatively larger than the amount of adjustment of the brightness of the exterior (outside the vehicle) image portion.

[0221] Furthermore, in the above-described embodiment, the brightness adjustment amount for the video portion inside the room (inside the vehicle) is relatively larger than the brightness adjustment amount for the video portion outside the room (outside the vehicle), but when a single frame of video contains dark and bright areas, rather than indoors and outdoors, the brightness adjustment amount for the dark areas may be relatively larger than the brightness adjustment amount for the bright areas to adjust the brightness. In this way, it is not necessary to necessarily detect the indoor and outdoor areas. Within one frame, it is preferable to define areas as having brightness above a threshold value, and dark areas as areas having brightness below the threshold value. A detection device (an example of a detection means) that detects bright areas above a threshold value may be provided, and the detected areas may be defined as bright areas, and the undetected areas may be defined as dark areas. Furthermore, a detection device (an example of a detection means) that detects dark areas below a threshold value may be provided, and the detected areas may be defined as bright areas, and the undetected areas may be defined as dark areas. The detected portion may be regarded as a dark portion, and the undetected portion may be regarded as a bright portion. The brightness adjustment amount may be adjusted by adjusting the gain of the video data. The brightness may be adjusted so that both the bright and dark regions are brightened, or so that the brightness adjustment amount of the dark region is relatively larger than the brightness adjustment amount of the bright region, or so that only the dark region is brightened. It is even more preferable to provide an adjustment device (an example of an adjustment means) that adjusts this threshold value. The adjustment device (an example of an adjustment means) that adjusts the threshold value may be an adjustment device that adjusts the threshold value by inputting a user's instruction.

[0222] 20 and 21 show conversion curves for adjusting the brightness of video data.

[0223] FIG. 20 shows an example of a brightness conversion curve in which, when a frame contains both dark and bright areas, the brightness adjustment amount for the dark areas is made relatively larger than the brightness adjustment amount for the bright areas to adjust the brightness.

[0224] In Figure 20, the horizontal axis indicates the brightness level (input level) before brightness conversion, with the left side indicating darker and the right side indicating brighter. The vertical axis indicates the brightness level (output level) after brightness conversion, with the bottom side indicating darker and the top side indicating brighter.

[0225] By performing brightness conversion based on the brightness conversion curve C1 shown in Figure 20, the percentage of brightness that becomes brighter after brightness conversion is greater when the brightness before brightness conversion is dark than when the brightness before brightness conversion is bright. In brightness conversion based on the brightness conversion curve C1 shown in Figure 20, if the brightness before conversion is extremely dark, the percentage of brightness that becomes brighter after brightness conversion is not very large. In this way, if the brightness before conversion is extremely dark, it is possible to prevent the percentage of brightness that becomes brighter after brightness conversion from becoming too large. For example, assuming a first threshold value and a second threshold value greater than the first threshold value, it is possible to set the adjustment amount of brightness that is equal to or greater than the first threshold value and equal to or less than the second threshold value to be brighter than the adjustment amount of brightness that is less than the first threshold value and brighter than the first threshold value.

[0226] When the indoor emphasis button 134 is pressed, brightness adjustment may be performed using the brightness conversion curve C1 shown in Figure 20, or a button such as a dark area emphasis button may be provided instead of the indoor emphasis button 134, and when that button is pressed, brightness adjustment may be performed using the brightness conversion curve C1 shown in Figure 20.

[0227] FIG. 21 is an example of a brightness conversion curve C2 that is used when the brightness adjustment described above is not performed.

[0228] In Figure 21, the horizontal axis also indicates brightness before brightness conversion, with the left side darker and the right side brighter. The vertical axis indicates brightness after brightness conversion, with the bottom side darker and the top side brighter.

[0229] The brightness conversion curve C2 shown in FIG. 21 has linearity, and by using the brightness conversion curve C2, brightness correction is performed at a constant rate regardless of the brightness before brightness conversion.

[0230] It is preferable that brightness adjustment be performed using the brightness conversion curve C2 shown in FIG. 21 when the normal display button 135 is pressed.

[0231] In the above, the brightness conversion curve C1 shown in Figure 20 is used when a single frame contains dark and bright areas without considering indoors and when brightness adjustment is performed by making the amount of brightness adjustment in the dark areas relatively larger than the amount of brightness adjustment in the bright areas. However, the indoor image portion and the outdoor image portion are detected within a single frame, and the brightness of the detected indoor image portion is corrected using the brightness conversion curve C1 shown in Figure 20, and the detected For the outdoor image portion, brightness correction may be performed using a brightness conversion curve that has linearity, for example, the brightness conversion curve C2 shown in FIG.

[0232] In the above-described embodiment, buttons 134 and 135 are used to instruct brightness correction and to instruct stopping brightness correction, but instead of providing these buttons 134 and 135 on the playback window 100, a menu may be displayed on the playback window 100 as described below, and the brightness correction instruction and the brightness correction stop instruction may be selected from the displayed menu.

[0233] Dashcams use lenses that capture not only the front but also both the front and rear, and 240-degree views of the front and the interior of the vehicle. Therefore, adjusting the video to match the outside brightness, whether it's daytime, evening, or night, results in dark interiors, such as the vehicle's interior. Nighttime video is easier to see inside the vehicle in places with streetlights than during the day, but it becomes harder to see in darker areas with fewer streetlights. Increasing the brightness causes the outside to be overexposed, so increasing the brightness when recording is a waste. Digital images tend to lose data when overexposed, while overexposed images retain data. By adjusting the brightness, you can see the driver even in pitch-black video. Adjusting the overall brightness to make the interior visible creates a bottleneck, as the exterior becomes too bright. However, adding buttons to enhance the visibility of the interior or exterior allows for the creation of video that emphasizes the interior or exterior. While the above example only discussed two options, with or without brightness adjustment, linear brightness adjustment is also possible. An intermediate level of brightness adjustment is also possible.

[0234] As mentioned above, if you move the cursor over the displayed image and right-click, a menu will appear, and you can use this menu to perform interior priority correction (interior emphasis correction, interior priority correction, interior priority correction).

[0235] Using the brightness conversion curve C1 shown in Figure 20, the degree of brightness is increased in the dark areas, and the bright areas are not corrected much so that they become brighter. The amount of brightness adjustment is different for bright and dark areas. Brightness is adjusted for both day and night.

[0236] Brightness correction is performed during playback, but it is also advisable to perform the same brightness correction when playing back video on the display 11 of the main body of the drive recorder 1. A menu may be displayed on the display 11 of the drive recorder 1, and a button to switch between instructing brightness correction and stopping it or a button to set it may be selected from the menu, or a touch panel may be formed on the display 11, and such a switch button or button to set it may be provided on the display. When playing back video data while it is being recorded on the drive recorder 1, the brightness correction described above may be performed on the video data to be played back.

[0237] [Third Example] 22 to 27 show a third embodiment, in which the video in the playback window 100 is displayed separately from the playback window 100. FIG.

[0238] As described above, display control device 51 displays playback window 100 as shown in Fig. 23 on the display screen of display device 52 of personal computer 50. The user moves cursor 162 onto the video displayed in video display area 111 and right-clicks (YES in step 151). In response, display control device 51 superimposes menu 161 on the video displayed in video display area 111 (step 152).

[0239] Menu 161 displays the following strings: "Image status: normal", "Return to standard", "Invert", "Zoom in / out", "In-car priority correction", and "Display in separate window".

[0240] The video display area 111 of the playback window 100 can display not only video captured in the forward direction of the vehicle, but also so-called spherical video captured in all directions, up, down, left, and right, with the drive recorder at the center. With such spherical video, it can be difficult to tell whether the image is in a normal or inverted state. Therefore, the "image state" indicates the state of the image, and "image state: normal" indicates that the video displayed in the video display area 111 is in a "normal" state. Since spherical video captures all directions, up, down, left, and right, with the drive recorder at the center, moving the video displayed in the video display area 111 can display not only video captured in front of the vehicle equipped with the drive recorder, but also video viewed in all directions, such as backward, left, right, upward, and downward. "Return to standard" is selected to change the video displayed in the video display area 111 to a standard view (e.g., from the front). When "invert" is selected, text strings such as "vertical inversion," "horizontal inversion," and "front-to-back inversion" are displayed. When "Flip vertically" is selected, the image is flipped upside down, when "Flip horizontally" is selected, the image is flipped left to right, and when "Flip front to back" is selected, the image is flipped from the front image to the rear image or from the rear image to the front image. "Zoom in / out" is selected when zooming in or out on the image. "Interior priority correction" is selected when adjusting the brightness by making the amount of brightness adjustment to brighten the interior of the vehicle greater than the amount of brightness adjustment to brighten the exterior, as described above. "Display in separate window" is selected when displaying the image displayed in the image display area 111 in a window separate from the playback window 100.

[0241] 22, when "display in separate window" is selected (YES in step 153), CPU 58 generates a window for displaying video separate from playback window 100. As shown in Fig. 24, a window 160 for displaying video separate from playback window 100 appears on the display screen of display device 52 (step 154).

[0242] 24, the video that was displayed in video display area 111 of playback window 100 (which is an example of a window including an operation system for operating the displayed video) is displayed in video display window 160. Video display window 160 may be displayed over the entire display screen of display device 52 so as to hide playback window 100, or video display window 160 may be displayed mainly in the left or right half of the display screen, and playback window 100 may be displayed mainly in the right or left half of the display screen. Also, video display window 160 may be displayed mainly in the upper or lower half of the display screen, and playback window 100 may be displayed mainly in the lower or right half of the display screen. Furthermore, the size of video display window 160 and the size of playback window 100 may be the same or different.

[0243] Furthermore, when a video display window 160 is displayed separately from the playback window 100, the video may be displayed in the video display area 111 of the playback window 100. However, at that time, the video does not have to be displayed in the video display area 111 of the playback window 100. When the playback window 100 and the video display window 160 are displayed on the same display screen, it is preferable not to display the video in the video display area 111 of the playback window 100.

[0244] When a cursor 162 is moved onto the image displayed in the image display window 160 and the right mouse button is clicked, a menu 161 is displayed.

[0245] The menu 161 displayed on the video displayed in the video display window 160 differs from the menu 161 displayed on the video in the playback window 100 in that it contains the string "Close window" instead of the string "Display in a separate window." When the string "Close window" is selected, the video display window 160 disappears, and the video display area of ​​the playback window 100 is closed. The image displayed in the image display window 160 is displayed in 111.

[0246] The information displayed in vehicle status display area 132 (an example of accompanying information) may be displayed on the image in image display window 160. While viewing the image displayed in image display window 160, it is possible to check the speed, acceleration, etc. of the vehicle equipped with the drive recorder that captured the image. When displaying playback window 100 and image display window 160 on the same display screen, it is preferable to display the information displayed in vehicle status display area 132 on top of the image displayed in image display window 160 when vehicle status display area 132 is hidden by image display window 160.

[0247] 25 to 27 show modified examples.

[0248] 22 to 24, the playback window 100 and the video display window 160 may be displayed on the same display screen or on different display screens, but in the modified examples shown in Figures 25 to 27, they are displayed on different display screens. In this modified example, an external display device other than the display device 52 is connected to the personal computer 50.

[0249] FIG. 25 is a flowchart showing the processing procedure for displaying the playback window 100 and the video display window 160 on separate display screens.

[0250] As shown in Fig. 23, when the video displayed in the video display area 111 is right-clicked (YES in step 171) while the video display window 160 has not yet been generated, a menu 161 is displayed on the video (step 172). In this modified example, the character string "Display on a separate display screen" appears in the menu 161 instead of "Display in a separate window." When the user selects the character string "Display on a separate display screen" that appears in the menu 161 (YES in step 173), the CPU 58 forms the video display window 160. Data representing the formed video display window 160 (data for displaying the video on a separate display screen) is output to the external display device connected to the personal computer 50 (step 74).

[0251] FIG. 26 shows how a playback window 100 and a video display window 160 are displayed on different display screens.

[0252] A video display window 160 (an example of a first window) is displayed on a display screen 190 of an external display device different from the display device 52 of the personal computer 50, and a playback window 100 (an example of a second window) is displayed on a display screen 180 of the display device 52.

[0253] Although no video is displayed in the video display area 111 of the playback window 100, it is preferable to display one. When the video display window 160 is displayed on a display screen 190 different from the display screen 180 of the display device 52 of the personal computer 50, if the cursor 162 is moved over the video display area 111 and the right click is clicked, a menu 161 appears in the video display area 111. The menu 161 contains the character string "Close separate display screen." When the character string "Close separate display screen" is selected, the output of data to the external display device stops, and the video display window 160 displayed on the display screen 190 of the external display device disappears. If no video is displayed in the video display area 111 of the playback window 100, the video is made to be displayed in the video display area 111.

[0254] FIG. 27 shows an example of a method of use when the playback window 100 and the video display window 160 are displayed on separate display screens.

[0255] For example, the data obtained by the drive recorder installed in the vehicle driven by the driver 211 The video recorded on the external display device 210 is played back, and the above-mentioned playback window 100 is displayed on the display screen of a personal computer 200 of a manager 201 of a driver 211, etc. The personal computer 200 is connected to a relatively large external display device 210, and data for displaying the above-mentioned video display window 160 is output to the external display device 210. The video display window 160 is displayed on the display screen of the external display device 210.

[0256] An administrator 201 or the like operates a playback window 100 displayed on a personal computer 200 at hand to control the video displayed on the display screen of an external display device 210. The driver 211 can receive guidance, warnings, etc. from the administrator 201 while watching the video displayed on the display screen of the external display device 210.

[0257] Furthermore, in the above-described embodiment, the playback window 100 is displayed on the display screen of the personal computer 50, but the playback window 100 may also be displayed on the display 11 of the drive recorder 1 or the like instead of on the display screen of the personal computer 50.

[0258] For example, if a driving school has a driving recorder in its training car and is showing driving footage of students at the school during a lesson, it would be difficult to see the footage if it was shown on a small screen for everyone to see, but by displaying the footage on the display screen of external display device 210 as described above, it becomes easier to see. A manager (for example, a driving school instructor) can operate personal computer 200 at hand to display the driving footage of the students on the display screen of external display device 210 and teach driving precautions to the students at the driving school. In a similar manner, an operations manager may instruct truck drivers.

[0259] In addition to the video, the map image displayed in the map image display area 121, the information displayed in the vehicle status display area 132, etc. may be displayed in a separate window or separate display screen. Other accompanying information may also be displayed in a separate window or separate display screen as needed. To display a separate window or separate display screen, right-click to display a menu and select from the menu, or a button may be provided and operated to display a separate window or separate display screen.

[0260] It is preferable to operate the video using the playback window 100, which is the window of the operation unit, and not to operate the video in the video display window 160. It is preferable that the video display window 160 does not include an operation system or operation buttons. It is preferable that the video display window 160 does not include an operation unit or operation system and cannot be operated, and the video is displayed in such a window. It is preferable that the video in the video display window 160 is controlled by operating the operation button 133 of the playback window 100, etc., as described above.

[0261] The operation playback window 100 may be displayed on the screen of the drive recorder, and the video display window 160 may be displayed on a separate monitor to display the video. The drive recorder 1 may perform the same functions as programs and software for personal computers.

[0262] [Fourth Example] 28 to 34 show a fourth embodiment.

[0263] In the fourth embodiment, the playback speed of the video can be adjusted. A system capable of adjusting the playback speed of the video can use, for example, the personal computer 50 shown in Fig. 7, but it can also use the drive recorder 1 shown in Fig. 3, etc.

[0264] FIG. 28 is a flowchart showing an example of a video playback processing procedure.

[0265] When a video file to be played is specified, the video stored in the specified video file is played. The data is read by the CPU 58, and as shown in FIG. 13, a still image representing a portion of the video is displayed in the video display area 111 of the playback window 100 displayed on the display screen of the display device 52 (step 221). The CPU 58 also reads the vehicle speed pulse data that accompanies the video data as additional information (step 221). When the user presses the playback button (right-facing triangular button) included in the operation buttons 133 (YES in step 222), playback of the video begins. The CPU 58 determines whether the vehicle speed represented by the read vehicle speed pulse data is slow (step 223). For example, if the speed of the vehicle in which the drive recorder 1 that captured the video is installed is slower than a speed threshold, the vehicle speed is determined to be slow, and if the speed is equal to or greater than the speed threshold, the vehicle speed is determined to be not slow. The speed threshold may be predetermined or may be specified by the user. Video in which the vehicle is stopped is also determined to be slow.

[0266] If it is determined that the vehicle speed is slow (YES in step 223), the CPU 58 slows down the playback speed of the video displayed in the playback window 100 (step 224). Slowing down the video playback speed means that the video playback speed is slower than the normal playback speed. The normal playback speed is a speed at which the shooting time during shooting is the same as the playback time during playback. For example, by making the number of video frames per unit time the same during shooting and playback, the shooting time during shooting and the playback time during playback will be the same speed. For example, if the shooting rate is one frame every 1 / 30 seconds, then the playback will also be one frame every 1 / 30 seconds without any dropped frames. If it is determined that the vehicle speed is not slow (NO in step 223), the playback speed of the video displayed in the playback window 100 is set to the normal speed by the CPU 58 (step 225). In this way, the playback speed of the video is slowed down or set to the normal speed, and the video is played back (step 226) (this is an example of changing the playback speed of the captured video based on the vehicle speed).

[0267] FIG. 29 shows the relationship between vehicle speed and video playback speed.

[0268] In the video from time t0 to time t3, it is assumed that the vehicle's speed was slow from time t1 to time t2. Also, it is assumed that the vehicle's speed was not slow (normal speed or slow speed) from time t0 to time t1 and from time t2 to time t3. Then, the playback speed of the video from time t1 to time t2 is slowed down. The playback speed of the video from time t0 to time t1 and the video from time t2 to time t3 is set to normal speed. They may also be played back at a faster speed. Also, the video when the speed is slow may be played back at the normal playback speed, and the video when the speed is not slow may be played back at a speed faster than normal speed (for example, n times the normal playback speed).

[0269] Fig. 30 is a flowchart showing an example of a video playback processing procedure. In Fig. 30, the same processes as those shown in Fig. 28 are denoted by the same reference numerals and their explanations will be omitted.

[0270] In the process shown in FIG. 28, when the vehicle speed is slow (YES in step 223), the video playback speed is slowed down, but in the process shown in FIG. 30, when the vehicle speed is slow (YES in step 223), the video playback speed is increased (for example, n times the normal playback speed) (step 224A). When the vehicle speed is not slow (NO in step 223), the video playback speed is set to the normal speed (or a playback speed less than 1x) (step 225). When the vehicle speed is slow, such as when the vehicle is stopped, it is assumed that important scenes are not captured in the video, and scenes where the vehicle is moving slowly can be fast-forwarded, thereby shortening the video playback time. For example, when a manager who manages the driver's driving status views the video, he or she can fast-forward video when the vehicle is moving slowly and the information is not very important, such as when stopped at a traffic light, thereby simplifying viewing.

[0271] In the above embodiment, it is determined whether the vehicle is slow based on the vehicle speed pulse data. However, other information such as GPS information may also be used to determine whether the vehicle is moving slowly. Also, in the above embodiment, a mode may be used in which the footage before and after the occurrence of an event obtained based on the vehicle's history information is played back at normal speed. Furthermore, since the footage while parking is the same image, it may be played back in a skipped manner, and footage while driving at high speed may be played back slowly because the footage flows quickly. In particular, footage obtained with a camera that shoots from the side flows quickly, so the footage appears to be fast. For this reason, it may be played back at a slow speed.

[0272] FIG. 31 is a flowchart showing the processing procedure for video analysis.

[0273] In this embodiment, a video analysis button (not shown) is formed in the playback window 100. However, instead of forming a video analysis button, a menu may be displayed by, for example, right-clicking on the video display area 111, and video analysis processing may be specified in the menu. Also, video analysis as shown in Fig. 31 may be performed before playing the video in response to pressing the play button.

[0274] When a video to be played is designated and the video analysis button is pressed (YES in step 231), video data representing the designated video is read (step 232). If the video data is stored in the semiconductor memory of the personal computer 50, it is read by the SSD 53, and if the video data is stored on the SD card 23, it is read by the SD card reader / writer 54.

[0275] The image represented by the read video data is analyzed by CPU 58 (step 233). The portion of the image that includes other vehicles is detected (YES in step 234), and it is determined whether the host vehicle (the vehicle capturing the video) is slower than the other vehicles (YES in step 235). If it is slower, data representing the time period when it is slower is associated with the video data and recorded (step 236). For example, the data representing the time period when it is slower is recorded in the header of the video file in which the video data is stored. If the host vehicle is not slower than the other vehicles (NO in step 235), the processing of step 236 is skipped. The processing of steps 234 to 236 is repeated until the video analysis is completed (step 237).

[0276] In this way, by identifying the time periods in the video where the vehicle is traveling slowly, when playing back the video, the playback speed of the video can be slowed down (step 224 in Figure 28) or increased (step 224A in Figure 29), as shown in Figure 28 or Figure 29.

[0277] FIG. 32 is a flowchart showing another example of the video analysis processing procedure.

[0278] As in the process of FIG. 31, when the video to be played is designated and the video analysis button is pressed (YES in step 241), the video data is read (step 242) and the video is analyzed (step 243).

[0279] If the result of the video analysis shows that only the front or rear of the vehicle is displayed (YES in step 244), identification data indicating that the video portion is a front or rear image is recorded (step 247). For example, the identification data is recorded in the header of the video file in which the video data is stored. If the result of the video analysis shows that the video portion is not a front or rear image of the vehicle only (NO in step 244) but is a lateral image of the vehicle only (YES in step 245), identification data indicating that the video portion is a lateral image is recorded (step 248). In this case, too, for example, the identification data is recorded in the header of the video file in which the video data is stored. If the result of the video analysis shows that the video portion is not a lateral image of the vehicle only (NO in step 245), it is determined whether the proportion of the front or rear image of the vehicle (the proportion of the front and rear images of the vehicle) is greater than the lateral image (step 249). If the proportion of the image in the front or rear of the vehicle is greater than the proportion of the image in the lateral direction (YES in step 246), identification data indicating that the image portion is a front or rear image is recorded (step 247). If the proportion of the image in the lateral direction of the vehicle is greater than the proportion of the image in the front or rear of the vehicle (proportion of the image in the front and rear of the vehicle) (NO in step 246), identification data indicating that the image portion is a lateral image is recorded (step 248).

[0280] The processing from steps 244 to 249 is repeated until the analysis of the video is completed (step 248).

[0281] FIG. 33 is a flowchart showing an example of a playback processing procedure for video that has been subjected to the video analysis processing of FIG.

[0282] When a video to be played is designated, the video data and video identification data are read (step 251). When the play button included in operation buttons 133 is pressed (YES in step 252), it is determined from the read identification data whether the video represented by the read video data is a front video, a rear video, or a side video (step 253).

[0283] If the video is a forward or backward video, the video is played back at a faster than normal playback speed (step 254). When the video is a forward or backward video, slowing down the playback speed often does not create a sense of incongruity. The playback time can be shortened while still providing a natural playback experience. When the video switches (YES in step 255), if playback has not ended (NO in step 258), the process is repeated from step 253.

[0284] If the video is landscape, the video is played back at a slower playback speed than normal (step 256). When the video is landscape, the video may appear to flow too quickly, creating a sense of incongruity. Since the playback speed is slowed down when the video is landscape, this often does not create a sense of incongruity. When the video switches (YES in step 257), if playback has not ended (NO in step 258), the process is repeated from step 253.

[0285] The processing shown in Figure 33 is performed on video that has undergone the video analysis processing shown in Figure 32, but the processing shown in Figure 33 is not necessarily performed on video that has undergone the video analysis processing shown in Figure 32, and processing similar to the processing shown in Figure 33 may be performed on video that has not undergone the video analysis processing shown in Figure 32.

[0286] For example, if a first camera captures the front or rear of a vehicle and a second camera captures the side of the vehicle, it can be seen that the image captured by the first camera represents the front or rear of the vehicle, and the image captured by the second camera represents the side of the vehicle. In such a case, the image captured by the first camera may be played back at a faster playback speed, and the image captured by the second camera may be played back at a slower playback speed. Data indicating whether the image was captured by the first camera or the second camera would be recorded in the header of the video file in which the video data representing the image is stored.

[0287] In addition, when playing back video captured by a camera capable of capturing images in all directions, including up, down, left, right, front, and back, it is possible to display the front or rear of the vehicle, or the side view. In such video, it is advisable to detect whether the front or rear of the vehicle is displayed more frequently during playback, and to speed up the playback speed when the front or rear of the vehicle is displayed more frequently, and to slow down the playback speed when the side view of the vehicle is displayed more frequently.

[0288] FIG. 34 is a flowchart showing an example of a playback processing procedure.

[0289] In this embodiment, it is preferable that a normal playback command button is provided in the playback window 100, but it is also possible to give a normal playback command to the personal computer 50 even without the normal playback command button, by clicking on the video display area 111 to display a menu and then giving a normal playback command from that menu.

[0290] When video is being played back, it is determined whether it is not being played back at the normal playback speed (step 261). If the video is being played back at a speed faster or slower than the normal playback speed, it is determined that the video is not being played back at the normal playback speed. If the video is not being played back at the normal playback speed (YES in step 261), and a normal playback command is given to the personal computer 50 by pressing a normal playback button or the like (YES in step 262), the video that is not being played back at the normal playback speed will be played back at the normal playback speed (step 263). The video that is not being played back at the normal playback speed will now be able to be played back at the normal playback speed.

[0291] While the frontal view of a vehicle looks natural while driving, the side view is difficult to see when it moves, so the video display speed can be slowed down. Since the video appears slow when viewed from the front, the playback time can be shortened by speeding up the playback. When the vehicle is stopped, playback is performed at normal speed, but the video moves when driving at high speeds. Therefore, it is possible to play the video slowly when the vehicle is moving at high speeds and automatically play it at double the speed when the vehicle is stopped. Normal playback should be the playback performed during the actual recording. For example, when waiting at a traffic light, the user would normally have to press the fast-forward button, but the video will automatically speed up. It is best to display the speed numerically in the PC viewer. The speed information stored in the drive recorder includes GPS information and vehicle speed pulses, so these can be used. GPS information and vehicle speed pulses are included in the header of the video file containing the video data, so they can be used. The vehicle speed pulse is the raw information from the vehicle's speedometer and is output based on the signal output in response to the tire rotation speed.

[0292] [Fifth Example] 35 to 40 show a fifth embodiment, which relates to the display form of the playback window.

[0293] Figures 35 and 36 show larger versions of the map image.

[0294] FIG. 35 is a flowchart showing the processing procedure of the personal computer 50, and FIG. 36 is an example of the playback window 100.

[0295] 35 starts when the display software for the playback window 100 is activated by the user. Then, as shown in FIG. 12, the playback window 100, with the video display area 111, map image display area 121, etc., blank, is displayed on the display screen of the display device 52 of the personal computer 50 (step 271). In the list display area 141, tabs are displayed (not shown in FIG. 36, etc.) that can be used to issue commands to play back the captured video as well as to issue commands to play back the history information of the vehicle that captured the video (vehicle position information, speed information, acceleration information, etc.). By selecting this tab (an example of an enlargement command button) and selecting a history information file, a command to play back the history record data represented by the selected history information file is issued to the personal computer 50. When a command to play back the history record data is issued to the personal computer 50 (YES in step 272), a command to swap the video display area 111 and the map image display area 121 is issued to the display control device 51 of the personal computer 50, and the video The area of ​​display area 111 is swapped with the area of ​​map image display area 121 (step 273). As shown in Fig. 36, the video display area 111 and the map image display area 121 are swapped (this is an example of a two-stage area consisting of a large area and a small area), and the map image and video are displayed (step 274). Because the map image is displayed large, it is relatively easy to check the shooting location, the position of the vehicle at the time of shooting, etc.

[0296] In the examples shown in Figures 35 and 36, the map image is enlarged, but the area for displaying information about the vehicle obtained while the vehicle capturing the video is operating may be enlarged. Information such as vehicle acceleration information, vehicle speed information, brake information, and turn signal information may also be displayed enlarged. For example, speed graphs and acceleration graphs may be displayed enlarged. The area for displaying information about the vehicle may be switched from a small area to a large area in response to a command given from a button that gives a command such as an area enlargement command.

[0297] In the examples shown in FIGS. 35 and 36, the area of ​​the video display area 111 and the area of ​​the map image display area 121 are swapped, but this is not necessarily required. It is sufficient that the size of the map image displayed when an instruction to play history record data is given is larger than the size of the map image displayed when an instruction to play history record data is not given (an example of the default size). Instead of increasing the size in response to dragging, for example, an enlargement command button may be provided in the playback window 100, and information about the vehicle, such as a map image of a predetermined size, may be displayed in an area larger than the predetermined size in response to an enlargement command button from the enlargement command button. Information about the vehicle, such as a map image of a predetermined size, may be displayed in an area larger than the size of the video, or in an area smaller than the size of the video. It is not necessary to change the size of the video display area 111, but it is particularly preferable to change the size of the video display area 111 as shown in FIG. 36.

[0298] The history record data and the video data are linked, and the video taken at the same time and the information about the vehicle are synchronized and displayed in the video display area 111, and it goes without saying that, for example, a map image is displayed in the map image display area.

[0299] 37 and 38 show three-dimensional displays of the acceleration of the vehicle whose video was captured. The processing shown in Fig. 37 and 38 may be performed in response to either an instruction to play back the video recording or an instruction to play back the history recording data.

[0300] Referring to FIG. 37, the playback window 100 has an acceleration graph display area 142 to the right of the play list display area 141. The acceleration of the vehicle that captured the video is displayed in three dimensions in the acceleration graph display area 142. In FIGS. 37 and 38, the X axis is displayed in the depth direction, the Y axis is displayed in the horizontal direction, and the Z axis is displayed in the vertical direction. The X axis represents the acceleration in the front and rear of the vehicle, the Y axis represents the acceleration in the lateral direction of the vehicle, and the Z axis represents the acceleration in the vertical direction of the vehicle. Acceleration in the front of the vehicle (positive acceleration) is displayed as an upward graph on the X axis, and acceleration in the rear of the vehicle (negative acceleration) is displayed as a downward graph on the X axis. Acceleration to the right as viewed from the vehicle is displayed as an upward graph on the Y axis, and acceleration to the left as viewed from the vehicle is displayed as a downward graph on the Y axis. Acceleration in the upward direction of the vehicle is displayed as a leftward graph on the Z axis, and acceleration in the downward direction of the vehicle is displayed as a rightward graph on the Z axis. Of course, if the accelerations are known, they do not have to be specified in this way.

[0301] When the acceleration graph display area 142 is formed, it is preferable to display a speed graph in the acceleration graph display area 138.

[0302] FIG. 38 shows an enlarged view of the acceleration graph displayed in the acceleration graph display area 142 compared to FIG.

[0303] The longitudinal acceleration of the vehicle is represented by graph 281, the lateral acceleration of the vehicle is represented by graph 282, and the vertical acceleration of the vehicle is represented by graph 283. In acceleration graph display area 142, the acceleration of the vehicle (the vehicle that captured the video) over time is displayed in three dimensions according to the time at which the video was captured. The vehicle acceleration displayed in acceleration graph display area 142 may be the acceleration of a portion of the vehicle in the captured video, and may particularly be the acceleration over a certain period of time (e.g., several seconds to several minutes before and after) before and after the video displayed in video display area 111. For example, if the current video displayed in video display area 111 is video from time t12, it is preferable that a graph representing the acceleration of the vehicle from time t11, which is before time t12, to time t13, which is after time t12, be displayed in acceleration graph display area 142. Data representing the acceleration is recorded in a header accompanying each frame of the video represented by the video data, and the graph representing the acceleration is displayed by reading that data. In addition, when the data is recorded in the header of a video file in which the video data is stored, or in a file that stores information about a vehicle other than the video file, it is desirable to read the data from these files and display a graph showing the acceleration.

[0304] 37 and 38 show graphs representing acceleration in a coordinate space consisting of the X-axis, Y-axis, and Z-axis, but the X-axis, Y-axis, and Z-axis do not necessarily have to be used as long as the first axis (an example of the X-axis) represents the acceleration in the longitudinal direction of the vehicle, the second axis (an example of the Y-axis) represents the acceleration in the lateral direction (width direction) of the vehicle, and the third axis (an example of the Z-axis) represents the acceleration in the vertical direction of the vehicle. The first axis, second axis, and third axis do not have to be perpendicular to each other. Furthermore, it is also possible to display a graph of acceleration that changes in the longitudinal direction in response to changes in the acceleration in the longitudinal direction of the vehicle, changes in the lateral direction in response to changes in the acceleration in the lateral direction of the vehicle, and changes in the vertical direction in response to changes in the acceleration in the vertical direction of the vehicle.

[0305] FIG. 39 shows a modified example, which is an example of a playback window 100.

[0306] The acceleration graph display area 142 in the playback window 100 shown in Fig. 37 is formed to the right of the playlist display area 141, but in the playback window 100 shown in Fig. 38, it is displayed superimposed on the video displayed in the video display area 111. This makes it relatively easy to check the acceleration while watching the video.

[0307] The information superimposed on the video is not limited to an acceleration graph, and may be other information related to the vehicle. This makes it easier to check other information related to the vehicle while viewing the video. In particular, as described with reference to FIGS. 26 and 27, when the playback window 100 and the video display window 160 are displayed on different display screens, it is preferable to display information related to the vehicle on the video. Furthermore, when the playback window 100 and the video display window 160 are displayed on different display screens, it is preferable to display information related to the vehicle on the video, and also display information related to the vehicle displayed on the video in the playback window 100. When the playback window 100 and the video display window 160 are displayed on different display screens, it is preferable to display an image in which an acceleration graph display area 142 is formed on the video in the video display window 160 as shown in FIG. 39, and to form an acceleration graph display area 142 in the playback window 100, which is displayed on a different display screen, as shown in FIG. 37.

[0308] FIG. 40 shows another modified example, which is an example of the playback window 100.

[0309] An acceleration display area 143 is formed on the right side of the playlist display area 141 formed in the playback window 100. The acceleration display area 143 displays an image of a car. In front of the image of the car, there is a forward-pointing arrow and the letters "+X", and next to the image of the car, there is a horizontal arrow. The letters "+Y" are displayed with an arrow pointing downwards, and the letters "+Z" are displayed with an arrow pointing upwards above the image of the car. The acceleration of the car in the forward or backward direction is displayed in front of the image of the car, the acceleration of the car in the lateral direction is displayed to the side of the image of the car, and the acceleration of the car in the vertical direction is displayed above the image of the car. Since the acceleration of the car is displayed along with the appearance of the car, it is relatively easy to see in which direction the acceleration is acting on the car. The image of the car will be stored in the setting file of the program that displays the playback window 100.

[0310] The direction of the arrow may also be changed depending on the acceleration. For example, if acceleration is applied to the front of the car, the arrow will point forward, and if acceleration is applied to the rear of the car, the arrow will point backward. Also, if acceleration is applied to the right side of the car, the arrow will point right, and if acceleration is applied to the left side of the car, the arrow will point left. Furthermore, if acceleration is applied to the top of the car, the arrow will point upward, and if acceleration is applied to the bottom of the car, the arrow will point downward.

[0311] An image of an automobile of a type corresponding to the type of vehicle from which the video was taken may be displayed in acceleration display area 143. For example, if the vehicle is a forklift and the video was taken by the forklift, an image of the forklift may be displayed in acceleration display area 143, and if the vehicle is a truck and the video was taken by the truck, an image of the truck may be displayed in acceleration display area 143. In this way, an image of a vehicle corresponding to the type from which the video was taken may be displayed in acceleration display area 143. Furthermore, an image of a vehicle of a manufacturer corresponding to the captured vehicle may be displayed in acceleration display area 143. For example, if the video was taken in a vehicle made by automobile manufacturer A, an automobile made by that automobile manufacturer B may be displayed in acceleration display area 143.

[0312] There's a concept called history. Until now, we've focused on video recordings, but with GPS information, we're capturing a whole lot of information on a whole new level. History alone is text, so the data volume is smaller than video. Information like when and where you passed is also part of history information. While the map is displayed small, as mentioned above, it's best to display the map larger on the history recording screen. Loading a history record displays the map in a larger format. The recording area on the SD card is also divided into an event recording area, a continuous recording area, and a history recording area. Continuous recording can only record about an hour, but history recording can record 24 hours, or 500 hours. History includes location information, speed information, acceleration information, braking information, and turn signal information. History recording is different from event recording and continuous recording. History recording is simply continuous recording without video data. You can set it to record up to 500 hours. History is also recorded in video data such as continuous recording and event recording. History information with video can be linked and displayed.

[0313] [Sixth Example] FIG. 41 shows a flowchart illustrating an example of the processing procedure of the personal computer 50 according to the sixth embodiment.

[0314] When the playback software that displays the playback window 100 is started, the processing shown in Figure 41 begins. Then, data indicating the clock frequency of the CPU 58 of the personal computer 50 is read (step 291). Based on the read clock frequency of the CPU 58, it is determined whether or not frames will be dropped in the playback of the specified video (step 292). If it is determined that frames will be dropped (YES in step 292), a message is displayed on the display screen of the display device 52 indicating that frames may be dropped in the video due to the low processing capacity of the personal computer 50 (step 293). If frames will not be dropped (NO in step 292), the processing of step 293 continues. You will be skipped.

[0315] Thereafter, the playback window 100 is displayed on the display screen of the display device 52 (step 294), and the video is displayed in the video display area 111 (step 295). The user realizes that the dropped frames in the video are not due to a problem with the video that was shot or the device that shot the video, but rather due to the capabilities of the personal computer 50 that plays back the video. Although the playback window 100 is displayed after a message indicating that dropped frames will occur, it is also possible to display a message indicating that dropped frames will occur along with the playback window 100. In particular, it is advisable to display a message indicating that dropped frames will occur on the playback window 100.

[0316] In the example shown in FIG. 41, whether or not frames will be dropped is determined based on the clock frequency of the CPU 58. However, the display of frame dropping may also be based on the capabilities and performance of a system such as the personal computer 50, such as the capacity of the RAM (random access memory) and VRAM (video random access memory) included in the personal computer 50, the read speed of the SD card reader when the video data is recorded on the SD card 23, or the read speed of the hard disk drive when the video data is recorded on a hard disk. Furthermore, when the capabilities and performance of the personal computer 50 are low (for example, when the capabilities and performance are insufficient to play back the video under the same conditions as when it was shot), slow playback may be performed, reducing the number of frames per unit time, rather than dropping frames. In this case, too, it is advisable to display (as an example of a notification) that slow playback will be performed due to the capabilities and performance of the personal computer 50. It is sufficient to be able to notify that the video is not capable of processing the video.

[0317] Moving video or displaying the video in a separate window requires a significant amount of CPU power from the PC. Some PCs cannot play video in a separate window, or change the display range of the video while double-speeding. For PCs that cannot do this, it is a good idea to impose restrictions such as only allowing it to be moved when paused. When creating a separate window button and making the playback window 100 and the video window separate windows, make the separate window button clickable, but rather than graying it out, a pop-up message will appear when pressed saying, "This computer does not have enough power to play in a separate window. Please prepare a computer with these specifications."

[0318] [Seventh Example] 42 to 46 show a seventh embodiment, in which playback is performed from a predetermined playback position.

[0319] Fig. 42 is a flowchart showing the processing procedure for video analysis. The processing shown in Fig. 42 starts when the video to be analyzed is specified and a video analysis command is given to the personal computer 50. For example, the processing shown in Fig. 42 starts when a playback window 100 with a video analysis button formed therein is displayed, a video is specified, and the video analysis button is pressed.

[0320] Video data representing the video to be analyzed is read (step 301), and the user inputs the conditions for the scene they want to find in the video (step 302). For example, a menu can be displayed by right-clicking on the video display area 111 of the playback window 100, and the desired scene can be selected from the menu, or a word describing the desired scene can be input in the menu. Once the scene conditions are input, the video data is analyzed, and video portions that match the input scene conditions are detected (step 303). The time of the video portions detected for each scene condition is stored in the header of the video file in which the video data representing the analyzed video is stored (step 304).

[0321] Fig. 43 is a flowchart showing the processing procedure of the personal computer 50. When the software of the playback window 100 is started, the processing shown in Fig. 43 begins.

[0322] The playback window 100 is displayed on the display screen of the display device 52 of the personal computer 50 (step 311). Data representing the time of the video portion that matches the scene conditions found in the video analysis is read from the header of the video file (step 312). Data representing the history information associated with the video file is also read (step 313). A first mark is added to the position indicating the time of the video portion that matches the scene conditions (step 314), and a second mark is added to the position indicating the time of the video portion based on the history information (step 315).

[0323] FIG. 44 is an example of the playback window 100.

[0324] In the playback window 100 shown in FIG. 44, a playback bar (seek bar) 300 is formed below the video display area 111. This playback bar 300 indicates the position of the video currently displayed in the video display area 111 within the entire video. The playback bar 300 displays a circular circle mark 351, a first cross mark 361, and a second square mark 362. The circle mark 351 indicates the position of the video currently displayed in the video display area 111. The first mark 361 indicates the video position of a scene that matches the scene condition. For example, if a "scene showing a child" is specified as a scene condition, the video corresponding to the position of the first mark 361 will be a video showing a child. Multiple scene conditions may be specified. While it is not necessary to change the mark depending on the type of scene condition, it is preferable to change the mark depending on the type of scene condition. The second mark 362 indicates the video position of a scene based on history information (information about the vehicle). For example, if information on sudden braking is specified as history information, the image corresponding to the position of the second mark 362 will be an image of sudden braking. Multiple pieces of history information may be specified. It is not necessary to change the mark depending on the type of history information, but it is preferable to change the mark depending on the type of history information.

[0325] Below the playback bar 350, a first skip playback button 341 and a second skip playback button 342 are formed. The first skip playback button 341 displays the character string "Skip 1." The second skip playback button 342 displays the character string "Skip 2."

[0326] FIG. 45 is a flowchart showing the first skip playback processing procedure.

[0327] When the first skip playback button 341 is pressed (YES in step 321), the circle mark 351 is moved to the position of the first mark 361 that is closest to the circle mark 351 (step 322). If the video is currently being played, the playback of the video is skipped up to the position of the first mark 361 to which the circle mark 351 has moved, and playback resumes from the position of the first mark 361 to which the circle mark 351 has moved. If the video is not currently being played, a still image of the video corresponding to the position of the first mark 361 to which the circle mark 351 has moved is displayed in the video display area 111. When the first skip playback button 341 is pressed again, the circle mark 351 moves to the position of the next first mark 361.

[0328] By using the first skip playback button 341, it is possible to view the video in a relatively short time while finding the video portion that matches the scene conditions.

[0329] FIG. 46 is a flowchart showing the second skip playback processing procedure.

[0330] When the second skip playback button 342 is pressed (YES in step 331), the circle mark 351 is moved to the position of the second mark 362 that is closest to the circle mark 351 (step 332). If the video is currently being played, the playback of the video is skipped up to the position of the second mark 362 to which the circle mark 351 has moved, and playback is resumed from the position of the second mark 362 to which the circle mark 351 has moved. If the video is not currently being played, a still image of the video corresponding to the position of the second mark 362 to which the circle mark 351 has moved is displayed. is displayed in the video display area 111. When the second skip playback button 342 is pressed again, the circle mark 351 moves to the position of the next second mark 362.

[0331] By using the second skip playback button 342, it is possible to view the video in a relatively short time while finding the video portion of the scene based on the history information. The first mark 361 and the second mark 362 only need to be displayed in different ways, and may have different shapes or colors.

[0332] When triggering an event search, you can either play back each video individually or remember the time and play it back. However, as mentioned above, by entering scene conditions such as "There was a person in the video at that time. The traffic light was red. It was where it was. What time was it?", events can be found based on that input. Continuous recordings and event recordings are recorded separately, but they are not related to each other. When playing back continuous recordings, you can identify the points where an event is recorded by looking at the date of the event recording file. A line indicating the location of the event is displayed in the slide bar of the continuous recording file. You can jump directly to the location of such a line. It would be convenient to have a button to jump directly to that line. Instead of pressing a button to jump directly to the event location, it would be better to use image analysis to jump directly to the location where the person was previously in the video. It would be useful to display images of intersections, people crossing the street, or cute girls one after another. In addition to event recordings, image analysis of continuous recording video can be used to extract triggers such as traffic lights, people crossing the street, and road signs. These locations can then be displayed one after another. It is a good idea to change the line color for each type of trigger and repeat playback for each type. The playback command can be a button or a menu. It is good to have a "jump" option in the menu with various types of options.

[0333] [Eighth Example] 47 to 49 show the eighth embodiment.

[0334] FIG. 47 is a flowchart showing the transmission process procedure of the drive recorder 1.

[0335] The processing procedure shown in FIG. 47 restricts the transmission of video data from the drive recorder 1.

[0336] When an event or the like occurs as described above and the CPU 20a of the drive recorder 1 issues a transmission command, it is checked whether the number of times video data has been transmitted from the drive recorder 1 has reached the limit (step 341). The limit is the number of times within a certain period of time, and may be any of a number of times per hour, day, week, month, etc. If the limit has not been reached (NO in step 341), it is checked whether the total amount of data to be transmitted has reached the limit (step 342). The total amount of data to be transmitted is also the total amount of data to be transmitted within a certain period of time, and may be any of a number of times per hour, day, week, month, etc. If the total amount of data to be transmitted has not reached the limit (NO in step 342), the video data to be transmitted is transmitted to a predetermined server or the like using the LTE module 21 of the drive recorder 1 (step 343), as described above.

[0337] When the transmission is completed, the number of transmissions is incremented by one (step 344). Data indicating the number of transmissions is stored in the RAM 20c of the drive recorder 1. The amount of data transmitted is also stored in the RAM 20c of the drive recorder 1 (step 345).

[0338] If the number of transmissions has reached the limit (YES in step 341), or if the total amount of data to be transmitted has reached the limit (YES in step 342), data indicating that data transmission is not possible due to the transmission limit is transmitted using the LTE module 21 of the drive recorder 1. The data is then transmitted to a server or the like (step 346). As a result, data that has not been received can be transmitted to a server or the like using Wi-Fi or the like without using the LTE module 21. Also, video data may be read from a memory card.

[0339] Although communication using the LTE module 21 is costly, the cost can be reduced because the data to be transmitted is limited. Also, if there is data that has not been received, the existence of that data is notified, so that data can be received by a method other than the costly method using the LTE module 21.

[0340] FIG. 48 is a flowchart showing the procedure for displaying rankings.

[0341] The server detects the amount of communication data for each user (step 351). For example, if a SIM (Subscriber Identity Module) card is loaded into the LTE module 21 of the drive recorder 1 and LTE-based communication is performed using that SIM card, the amount of communication data for each user can be detected by accessing the server of the contracted company for that SIM card. When a display command is given to the server (YES in step 352), the amount of communication data for each user is displayed on the display screen of a display device connected to the server (step 353).

[0342] FIG. 49 is an example of a graph showing the amount of communication data.

[0343] Figure 49 shows the communication data volumes of users 1 to 6. Of the communication data volumes of users 1 to 6, it can be seen that user 4's communication data volume is larger than that of the other users. For example, if the communication data volume of user 4 is compared with the amount of video data of user 4 sent to the server and there is a discrepancy, it can be seen that user 4 is not sending video data from drive recorder 1, but rather, for example, has removed the communication SIM card installed in drive recorder 1 and is misusing it for personal use. If the communication data volume of user 4 is compared with the amount of video data of user 4 sent to the server and there is a match, it can be seen that if user 4 is driving recklessly, many events will occur and the amount of video data sent will be large. This indicates that user 4 is driving recklessly. It can also be seen whether the video data volume is large due to poor video data transmission settings on drive recorder 1.

[0344] [Ninth Example] 50 to 52 show a ninth embodiment.

[0345] 50 is a flowchart showing the processing procedure of the drive recorder 1. For example, the process starts when the engine of the vehicle in which the drive recorder 1 is installed is turned on and the drive recorder 1 is powered on.

[0346] A test of the communication speed of video data is performed between the drive recorder 1 and the server (step 361). If the communication speed test is successful (YES in step 362), the resolution, frame rate, bit rate, etc. of the video data to be transmitted are determined by the CPU 20a of the drive recorder 1 based on the test results (step 363). For example, if the communication speed is fast, the resolution, frame rate, bit rate, etc. are determined so that the video represented by the video data to be transmitted has relatively high image quality, and if the communication speed is slow, the resolution, frame rate, bit rate, etc. are determined so that the video represented by the video data to be transmitted has relatively low image quality.

[0347] When an event occurs, the CPU 20a of the drive recorder 1 starts recording frames. When a command occurs (YES in step 364), the video data to be transmitted is recorded according to the determined resolution and the like (step 365).

[0348] If the communication speed test is not successful (NO in step 362), it is checked whether the number of test failures has reached a predetermined number (step 366), and if the predetermined number of failures has not been reached (NO in step 366), the communication speed test is retried (step 361). It goes without saying that the number of communication speed test failures is counted in the controller 20 of the drive recorder 1. If necessary, a counter may be provided in the drive recorder 1. If the communication speed test has failed a predetermined number of times (YES in step 366), when a recording command is generated (YES in step 367), video data is recorded at a predetermined resolution, etc. (step 368).

[0349] 51 and 52 show the processing procedure when a polarizing filter is attached to the drive recorder 1.

[0350] FIG. 51 is a flowchart showing the processing procedure of the drive recorder 1.

[0351] When a recording command is issued, the process shown in Figure 51 begins. The CPU 20a of the drive recorder 1 checks whether a polarizing filter is attached to the camera 17 of the drive recorder 1 (step 371). For example, if the drive recorder 1 is provided with a switch indicating that a polarizing filter is attached, the attachment of the polarizing filter is detected by detecting the state of the switch. If the drive recorder 1 is provided with a sensor indicating that a polarizing filter is attached, the attachment of the polarizing filter is detected by the sensor. The attachment of a polarizing filter may be detected by the CPU 20a analyzing (image recognition) the image represented by the video data captured by the camera 17 (for example, analyzing whether or not a polarizing filter of the interior of the vehicle, the dashboard, etc. is reflected in the window glass).

[0352] The video data is recorded on the SD card 23 (step 372), and identification data indicating that the video data was recorded with a polarizing filter attached is also recorded in the header of the file in which the video data is stored (step 374). A display device such as a light-emitting diode may be provided in the drive recorder 1, and the light-emitting diode may be illuminated when a polarizing filter is attached.

[0353] If a polarizing filter is not attached to the drive recorder 1 (NO in step 371), the video data is recorded on the SD card 23 without recording the above-mentioned identification data (step 374). However, it is preferable to record identification data indicating that the video data was recorded without a polarizing filter attached in the header of the file in which the video data is stored.

[0354] FIG. 52 is a flowchart showing the playback processing procedure in the playback window 100.

[0355] The process shown in FIG. 52 starts when a file in which video data to be played back is stored is specified and a play command is given using a play button or the like.

[0356] The header of the file in which the video data is stored is searched to determine whether identification data representing the video data recorded when a polarizing filter was attached to the drive recorder 1 is recorded in the header (step 381).

[0357] Next, the video data is played back and the video is displayed (step 382). During playback, the user is notified (by display, audio output, etc.) that the video data was shot with a polarizing filter attached (step 383).

[0358] Even if no identification data is recorded, the video data is played back and the video is displayed (step 384). When the video data is played back, the user is not notified (by display, audio output, etc.) that the video data was shot with a polarizing filter attached. However, it is preferable to notify the user that the video data was shot without a polarizing filter attached.

[0359] As dashcams become increasingly high-resolution, they may record things that were not actually visible. If you are a driver involved in an accident, you may want to argue in court that the footage is different from what you actually saw, but if it is presented as evidence in court, there is a possibility that clearer footage will be presented as evidence that will be unfavorable to the driver. In such cases, if information such as the presence or absence of a polarizing filter is included, you can argue that "this footage was taken when a polarizing filter was installed, so it is different from what I saw." This can serve as evidence for such a claim.

[0360] [Tenth Example] 53 to 56 show a tenth embodiment.

[0361] FIG. 53 shows the processing procedure of the drive recorder 1.

[0362] The drive recorder 1 has a function that allows it to make calls (communicate) with the smartphone of a user other than the driver of the vehicle in which the drive recorder 1 is installed. Of course, it can also make calls (communicate) with the smartphone of the user who is the driver of the vehicle. When the drive recorder 1 is connected to a predetermined communication device, such as a smartphone held by a user outside (or inside) the vehicle (YES in step 391), it is confirmed whether the videophone recording mode is set in the drive recorder 1 (step 392). Needless to say, the drive recorder 1 is provided with a mode switch or the like that can set the videophone recording mode. When the videophone recording mode is set (YES in step 392), data representing the content of the call with the communication device is recorded on the SD card 23. For example, a call recording area is defined on the SD card 23, and data representing the content of the call is recorded in that call recording area. The drive recorder 1 is provided with a microphone, and audio is input to the drive recorder 1 using the microphone.

[0363] The drive recorder 1 and the communication device may communicate (call) by receiving a call from the communication device, or by making a call from the drive recorder 1 to the communication device. It becomes possible to make calls using the drive recorder 1 even while driving. By equipping the drive recorder 1 with a camera that can record images inside the car, the drive recorder 1 can be used as a video phone.

[0364] FIG. 53 is a flowchart showing the playback processing procedure by the playback window 100.

[0365] An SD card 23 containing numerous files containing video data is loaded into a playback personal computer 50. The SD card contains not only files containing video data captured by the drive recorder 1, but also numerous files containing video data captured by surveillance cameras inside and outside the user's office, company, etc. The headers of these files also contain user IDs that identify the users recorded in the video using image recognition or other methods, as well as data indicating the date the video was taken. Therefore, it is possible to determine who appears in which file for each date the video was taken.

[0366] For example, the date on which the video data to be played back was taken and the user ID of the user whose movements are to be checked are input, and the CPU 58 of the personal computer 50 finds, from among the video files storing the video data taken on that date, the video data that shows the user identified by the input user ID (step 401).The multiple video data found are played back consecutively in chronological order (step 402).

[0367] It becomes possible to view the user's daily movements on a continuous timeline, such as the camera footage when leaving the company or sales office, the camera footage from the drive recorder 1, and the camera footage when returning to the company or sales office. It becomes easy to understand whether the user (employee) was working appropriately for a day, for example.

[0368] It will be possible to seamlessly play back dashcam footage and office footage. While dashcams only provide information about the driver's condition before getting into the car, what their facial expression looked like when they arrived at the office, and whether they've been drinking. This will allow for a clearer picture of a driver's daily activities. In addition to dashcam footage, you can also obtain footage of the driver arriving at the office and reporting home. Recordings with the same driver ID on the same day can be played back consecutively. If a driver drives to multiple locations and turns off the engine at each location, separate files will be created for the day, allowing for seamless playback. Footage from office surveillance cameras can also be combined to play back a day's events. Footage from the office, the first visit, the second visit, and so on, all can be played back consecutively. For example, truck, bus, and taxi operation managers are required to conduct roll calls. Recording this footage allows the company and management to check the drivers in the morning, on the way home, etc.

[0369] FIG. 55 is a flowchart showing the processing procedure of the drive recorder 1.

[0370] The process shown in Fig. 55 starts when a recording command is generated in the CPU 20a of the drive recorder 1. The process shown in Fig. 55 is performed in a drive recorder 1 that is configured to allow two SD cards, a first SD card and a second SD card, to be installed.

[0371] Video data captured by camera 17 is temporarily stored in RAM 20c. The video data is read from RAM 20c and processed in CPU 20a (step 411). For example, processing such as cropping a portion of the video (especially a celestial video) or coordinate conversion is performed. The processed video data is recorded on the first SD card (step 412). Data representing the processing content is recorded in the header of the file in which the video data is stored (step 413). Furthermore, unprocessed video data is read from RAM 250c and recorded on the second SD card (step 414). The video data recorded on the second SD card is video data representing video closer to the original video than processed video data, and is, for example, RAW video data, video data compressed based on H246, etc.

[0372] It is preferable that the unprocessed video data is recorded on a second SD card, the second unprocessed video data is read from the second SD card and processed, and the processed video data and data indicating the processed video data are recorded on a first SD card.

[0373] Furthermore, it is preferable that the video data representing the forward video from the dashcam (especially the spherical dashcam) is recorded on a first SD card (one of the SD cards) and the video data representing the rearward video is recorded on a second SD card (the other SD card).It is preferable that the video represented by the video data recorded on the first SD card and the video represented by the video data recorded on the second SD card are configured so that at least a portion of the video from the same time overlaps.

[0374] FIG. 56 is a flowchart showing the playback processing procedure for video data recorded on two SD cards as explained using FIG.

[0375] The first and second SD cards are loaded into a personal computer 50 or the like for playback.

[0376] The processed video data (first video data) and data indicating the processing details recorded on the first SD card are read from the first SD card (step 421), and the unprocessed video data (second video data) recorded on the second SD card is read from the second SD card (step 422).

[0377] The image represented by the second image data and the image represented by the first image data are displayed. At this time, a processing range indicating the processed portion is displayed in the unprocessed image (step 423). This allows the user to see which portion of the entire image has been processed and is being displayed in the processed image. In particular, it is advisable to synchronize two images at the same time and display each image in two windows (display areas).

[0378] It is also possible to not process the video data in the drive recorder 1, but to process the video data in a playback device such as a personal computer that plays back the video, and display both the processed video and the unprocessed video (for example, RAW video).

[0379] In playback devices such as personal computers, the two videos are recorded on separate SD cards, so the correspondence between the files must be determined, but this can be determined by time.

[0380] [Eleventh Example] 57 to 59 show an eleventh embodiment.

[0381] FIG. 57 is a flowchart showing the processing procedure of the drive recorder 1.

[0382] In the processing procedure shown in Fig. 57, a share button is provided on the drive recorder 1. A signal indicating that the share button has been pressed is input to the controller 20 of the drive recorder 1. The share button does not necessarily have to be a physically existing button, but may be a software button displayed on the display screen of the display 11.

[0383] When the user presses the share button (YES in step 431), the video data recorded continuously, representing the video captured within a certain period before and after the user pressed the share button, is sent to a predetermined cloud server (step 432). The video data sent to the cloud server is stored in a location identified by a predetermined uniform resource locator (URL).

[0384] In addition, data indicating the vehicle's location when the share button was pressed, the time when the share button was pressed, and a URL specifying the storage location on the cloud server are recorded in the header of the file in which the continuously recorded video data is stored (step 433).

[0385] It is desirable to have a function that not only sends video data before and after an event occurs when the share button is pressed, but also sends the video data (in this case, it may be video data for transmission, as described above) before and after the event occurs to the cloud server, just as it does when the share button is pressed.

[0386] FIG. 58 is a flowchart showing the playback processing procedure in the personal computer 50. do.

[0387] When the SD card 23 on which video data is recorded in the drive recorder 1 is loaded into the personal computer 50 and the playback button in the playback window 100 is pressed, the processing shown in FIG. 58 begins.

[0388] A map showing the vicinity of the location where the video being played was filmed is displayed in the map image display area 121 of the playback window 100 (step 441). If the location represented by the map displayed in the map image display area 121 includes the location where the Share button was pressed, an icon representing that location is displayed on the map (step 442). Data indicating the location where the Share button was pressed is recorded in the header of the file in which the video data is stored, so the icon is read from that header and displayed on the map. The icon is also clickable, and a URL indicating the location where the video data is saved is linked to it. This URL is also recorded in the header of the file in which the video data is stored, so it is read from the header and linked to the icon.

[0389] When the user clicks on an icon displayed on the map (YES in step 445), the request data for the URL linked to the clicked icon is sent. Then, the video data stored in the storage location specified by the URL is sent to the personal computer 50 that is playing the map. The user can view the video in the vicinity of the location where the share icon was clicked.

[0390] For example, by publishing a map on the Internet that displays the above-mentioned icon, a third party can view the map and click on the icon to see footage of the area near the icon, allowing the captured footage to be shared.

[0391] FIG. 59 is a flowchart showing the processing procedure of a smartphone capable of communicating with the drive recorder 1.

[0392] The smartphone is placed within a range where it can communicate with the drive recorder 1 installed in the vehicle via Wi-Fi, Bluetooth (registered trademark), etc. For example, if the drive recorder 1 is installed in a car, the smartphone is placed inside the car. The smartphone also has the function of connecting to the Internet via a wireless local area network (wireless LAN) with smart speakers and other devices installed in homes, offices, business establishments, etc. that have a wireless LAN environment.

[0393] Using the smart speaker, a command (trigger) to send, for example, video data is sent to the smartphone, and when the command is received by the smartphone (YES in step 451), the received command is sent by the smartphone to the drive recorder 1 (step 452). For example, if a command such as "Give me a video of Mr. A" is spoken to the smart speaker, the command is sent to Mr. A's smartphone, and a command to send video data is sent from the smartphone to the drive recorder 1 installed in the car that Mr. A is driving.

[0394] When the drive recorder 1 receives a transmission command from the smartphone, the video data recorded in the drive recorder 1 is transmitted from the drive recorder 1 to the smartphone in accordance with the command. The video data transmitted to the smartphone may be video data recorded immediately before the command was received, video data from a certain period before, after, or before the timing of the command being received, or video data recorded on the day the command was received.

[0395] When the smartphone receives the video data transmitted from the drive recorder 1 (YES in step 453), the received video data is transmitted from the smartphone to the server (step 454).

[0396] In this way, smartphones can connect to the cloud, and triggers can be pushed from the cloud to the smartphone, which then sends them to a dashcam connected to the smartphone via Wi-Fi or Bluetooth. It would be extremely convenient if it could be made to work in conjunction with IFTTT or similar. For example, if Manager X says to a smart speaker installed in the office, "Give me some footage of Driver A," a notification would be sent to Driver A's smartphone, and a trigger would be sent from the smartphone to the dashcam, which would then record the footage. The trigger name can be set by the user. The set trigger name will be displayed in the viewer. For example, it could be displayed as "Voice recording trigger from Manager X to Driver A" (for example, a popular driver (a dangerous one) could receive many triggers from various managers).

[0397] The smartphone is connected to the cloud. Since the smartphone is connected to the Internet, it can connect to the Internet even if the dashcam does not have the ability to connect directly to LTE. The smartphone and dashcam are paired and connected via Bluetooth. The server sends commands to the smartphone, which then sends commands to the dashcam. If you ask a smart speaker installed in your office to send video footage of a certain driver, the smart speaker sends a command to the smartphone, which then sends a command to the dashcam. The trigger name is important. Currently, trigger names are fixed—for example, acceleration triggers or switch press triggers—but it would be better if triggers could be named. It would be nice to be able to set a trigger, such as when Person X instructs Person Y to start recording. It would also be nice if users could change the trigger name themselves in the viewer. To display the trigger name in the viewer, a green line appears on the slide bar during an event. When you hover your mouse over the green line, the trigger name will pop up. It would be good if the trigger name popped up, such as "smartphone trigger" for the yellow line, or "smartphone trigger is a trigger from Administrator A."

[0398] Additionally, since gamers are concerned about FPS (frames per second), it's a good idea to equip the PC viewer with a function that displays the recording frame rate and the display frame rate. The viewer can also be equipped with a frame interpolation function. The recording frame rate is fixed, but the playback frame rate may not be able to reproduce the recording frame rate depending on the PC's capabilities. Both the recording frame rate and the playback (display) frame rate are displayed on the display screen. They are displayed as a superimposition in the upper left corner of the video being played. The playback frame rate will change, showing 30 fps for recording and 25 fps for playback, for example. This shows that the video is not displayed smoothly due to poor PC specs, not the dashcam's specs.

[0399] [Twelfth Example] 60 to 62 show a twelfth embodiment.

[0400] Figure 60 shows the processing procedure when playing back video data, and includes a flowchart showing the processing procedure of the playback device of the personal computer 50 and a flowchart showing the processing procedure of a robot that can communicate with the personal computer 50.

[0401] The robot is, for example, a humanoid robot with a head, arms, and legs attached to a torso. The robot has a communication function with a personal computer 50, such as Wi-Fi. The robot's movements are controlled by a computer built into the robot, and it has the function of performing movements according to the contents of communication with the personal computer 50. The robot is not limited to a humanoid robot, but may also be an animal-shaped robot such as a dog or cat.

[0402] When the SD card 23 on which the video data is recorded is loaded into the personal computer 50 and the playback program is started and the playback button is clicked, the video data is played back (step 461). If an impact point such as sudden braking is played back during playback of the video data (YES in step 462), a command to perform an impact action is sent from the personal computer 50 to the robot (step 469).

[0403] When the robot receives the command sent from the personal computer 50 (YES in step 471), the robot performs a falling motion in response to the command (step 472). The robot then gets up and warns the user about dangerous driving (step 473). For example, it outputs a voice message to warn the user about dangerous driving.

[0404] If the playback of the video data has not finished (NO in step 464, NO in step 474), the playback device repeats the process from step 461, and the robot repeats the process from step 471.

[0405] Commands for movement are sent from the PC viewer to the communication robot. For example, when playing back a section where an impact occurred, the communication robot will make a motion as if it is about to fall over. Then it will get up and make a motion as if scolding the driver, saying that dangerous driving is not okay. This is extremely effective when placed in a driving school. Being scolded by the operations manager of a bus or transportation vehicle can be irritating, but being scolded by a robot does not affect human relationships. Normally, when a operations manager instructs a driver, the operations manager is the one to instruct, but instead, the robot will give an objective response. The trigger content and what the robot will say are associated, and the robot will speak when the trigger is played back. Movements are also associated and stored. It moves and adjusts the volume to match the magnitude of acceleration from a G-sensor or other device. The correspondence is recorded, and the robot will give instructions when it is time to play back that information.

[0406] Also, for personal computers, it is a good idea to give up on reports and provide a function to export presentation files. Commercial drive recorders have the ability to print business reports on paper. These include verbal warnings such as, "How many sudden turns did you make?" Reports can also be output using the print function of a PC viewer. Instead of printing reports, they can produce presentation files, something like PowerPoint templates, that show your driving score.

[0407] FIG. 61 is a flowchart showing the processing procedure of the virtual reality device.

[0408] A video data playback device such as a personal computer 50 is connected to a virtual reality device. The virtual reality device includes a VR controller and a head-mounted display, and the user wears the head-mounted display and operates it using the VR controller. When the user wears the head-mounted display, a virtual playback window 100 is displayed (step 481).

[0409] When the dual-screen display button formed on the VR controller of the virtual reality device is pressed (YES in step 482), the user wearing the head-mounted display sees a first window and a second window. The first window corresponds to the playback window 100 shown in Fig. 26 and displays operation buttons, while the second window corresponds to the video display window 160 shown in Fig. 26 and displays a video (step 483).

[0410] When the display button for deployment formed on the VR controller of the virtual reality device is pressed (YES in step 484), the user wearing the head-mounted display Then, the spherical image is displayed (step 485).

[0411] Windows application windows can be positioned in three-dimensional space as a plane (board) or as part of a gentle cylindrical surface (or sphere). When the PC viewer (playback software that controls the playback window 100) is launched, the application screen can be similarly positioned in three-dimensional space as a plane (board) or as part of a gentle cylindrical surface (or sphere). Therefore, when the dual-screen display button on the PC viewer is pressed, one side is displayed as the PC viewer operation panel, and the other side is displayed as the playback image.

[0412] As mentioned above, a display button is provided in the VR space that projects (texture-maps) a spherical image over the entire interior of the sphere, with the location of the person as the origin. Pressing this button allows you to see an image that makes it seem as if you are inside the car. The sphere can be displayed behind the Windows window, but it is also possible to hide all windows and display the image as if you are completely inside the car. It is a good idea to be able to switch between the two using a button on the controller. The sphere can also be displayed in front of the window. By changing your position outside the sphere (moving forward), you can go outside the sphere and operate the operation window, etc. For example, it is a good idea to project the image onto the outside of the sphere while you are outside the sphere. In this way, the image can be viewed from outside the sphere. Instead of projecting it onto a perfect sphere, it is also possible to map it onto a curved surface that is set to correct for distortions in optical systems such as lenses. It is a good idea to have a function that makes the spherical surface onto which the image is projected semi-transparent. It is a good idea to make it semi-transparent by pressing a button on the controller. The origin position and the radius of the sphere can be changed by moving the left and right sticks up and down on the controller. Thumbnail displays, map displays, acceleration graphs, etc. should each be placed as independent panels in the three-dimensional space inside the sphere. It is a good idea to configure it so that playback control functions such as play, stop, and next event can be performed with the controller.

[0413] There is something called a VR headset, and when you start up a PC viewer, the VR image is displayed like a board. When you play it, the spherical image is played on that board. When trying to display two screens, it is best to configure it so that the viewer's controls are on the left and the boards of the image being played are on the right. A display button is provided that expands the spherical image inside the sphere, with the person's location in the VR space as the origin, and pressing this button will enter VR mode. It is best to configure it so that it feels like you are inside the sphere. It gives the impression of sitting in the driver's seat. In reality, it feels like you are in the installed position.

[0414] It is a good idea to configure the sphere to be displayed at the back of the Windows window. You can view the image of the sphere while watching the image on your computer (while operating your computer). You can also hide the window operation window so that you are inside the image. Use the control button to switch between the two. Use the stick on the controller to adjust the radius of the sphere. The position of the origin can also be adjusted with the stick. The sphere can be in front of the window. When you look behind in this state, you will see the sphere, but if the user moves and breaks the sphere, you will see the image. When you look behind in this state, the sphere will be there, so if you attach an image to the back of the sphere, the image can be seen from outside the sphere as well.

[0415] In reality, the image is projected in the shape of a rugby ball or egg due to lens distortion. You can view the image with the lens distortion corrected. The window contains a map display, acceleration graph, and operation buttons, but these buttons and objects are assigned to the VR screen. The operation panel-like thing is displayed as a panel in 3D space.

[0416] FIG. 62 is a flowchart showing the processing procedures of the drive recorder 1, the smartphone, and the television device.

[0417] The process shown in FIG. 62 involves viewing the video data recorded on the SD card 23 loaded in the drive recorder 1 on a television device located at home, in the office, at a business establishment, etc.

[0418] A user at home, in an office, or at a business uses a smartphone to send a playback command to the drive recorder 1 installed in the vehicle (step 501). When the vehicle engine is off, the drive recorder 1 is powered by a mobile battery or the like and receives the playback command.

[0419] When the drive recorder 1 receives the playback command sent from the smartphone (YES in step 491), the video data recorded on the SD card 23 is read, and the read video data is sent from the drive recorder 1 to the smartphone (step 492).

[0420] When the video data transmitted from the drive recorder 1 is received by the smartphone (YES in step 502), the received video data is transmitted from the smartphone to the television device (step 503).

[0421] When the video data transmitted from the smartphone is received by the television device (YES in step 511), the video is played back and the video captured by the drive recorder 1 is displayed on the display screen of the television device (step 512). The television device has a function for connecting to the smartphone via Wi-Fi, and receives and plays back the video data transmitted from the smartphone using Wi-Fi. The television device may also be configured to be connected to a LAN at home, in an office, or at a business premises, and to be connected to the Internet. The video data is transmitted from the smartphone to the television device via the Internet.

[0422] If you say something like "Play a panda video on a video playback site" to the smart speaker, the TV connected to the Wi-Fi device will turn on and start playing a panda video. It's extremely convenient to be able to give general voice instructions and have the TV play a video without specific instructions, such as when you want a video to be played on the TV as background music. Many people don't play dashcam footage unless an accident occurs, meaning the footage they've taken is hardly ever used. Furthermore, if an accident occurs while using a dashcam that wasn't working properly before the accident, it's conceivable that crucial accident footage may not have been recorded.

[0423] This system allows you to play dashcam footage via voice command. While it's possible to operate it from a smartphone in the car, if the car is near your home and has Wi-Fi or other signal reception, it's better to operate it from a smartphone, smart speaker, communication robot, or dedicated device at home. Saying "Play dashcam footage" to a voice-recognition device (smartphone, smart speaker, communication robot, etc.) will turn on the dashcam, stream your dashcam footage wirelessly via Wi-Fi, and play that footage on your TV (or run on battery power if the engine is off). Saying "Play yesterday's dashcam footage" will search for yesterday's dashcam footage on the dashcam or on a storage device (smartphone or PC) that stores data transferred from the dashcam, and play it if available. If not, it will say "Yesterday's dashcam footage was not found." It's also a good idea to play footage from the nearest dashcam instead, such as "I'll play the dashcam footage from the day before yesterday instead." If you say, "Play the dashcam footage from when I went out just now," the most recent dashcam footage will be played. When playing back dashcam footage, it would be good to play the audio recorded along with the dashcam footage, but it would also be good to have a function to play other music as background music. If you say, "Set the background music to XX," the dashcam will Convert the audio to a 3D audio file and play the song along with the video on a music streaming service such as Spotify. It is recommended that playback be a digest of the part where the event occurred. It is also recommended that you can specify the playback time, such as "play about 5 minutes of dashcam footage." In this case, for example, the video can be played all at once at the specified time. It is also recommended that unplayed video be played first.

[0424] When the dashcam recognizes a voice saying "Keep this video" while the dashcam is "playing back" video, the video file being played will be transferred from the dashcam to a storage device (such as a smartphone or PC) via Wi-Fi or other means. It may also be uploaded to a video-sharing SNS and shared. When the dashcam recognizes a voice saying "Keep this video" while the dashcam is "recording" video, the dashcam will use Wi-Fi or other means to transfer the recorded video from a predetermined time before that point to a storage device (such as a smartphone or PC) and the video data from that point to a predetermined time ahead will be recorded in the dashcam as a voice instruction trigger. When the dashcam recognizes a voice saying "Send video" while the dashcam is "recording" video, the dashcam will send the recorded video from a predetermined time before that point to a server or other means via LTE or other means. It may also be uploaded to a video-sharing SNS and shared. When the dashcam recognizes a voice saying "Stop sending video," it will stop sending the video to a server or other means. When the dashcam recognizes the voice command "Send me the footage up to this point" while recording video, it will upload the recorded footage from a specified time before that point to a video server via LTE or other means. It can also be uploaded to a video sharing social networking site for sharing. If you say "Start live streaming" in front of the dashcam, it will start live streaming of the dashcam footage.

[0425] There are AI speakers, speakers connected to the cloud, and devices that can stream smartphone footage to a TV. While smartphones require you to select each video individually, saying "play a video of a panda" will automatically play related footage. Dashcam footage won't play until an accident occurs. Sometimes, when an accident does occur, the dashcam isn't working properly. Play dashcam footage with voice commands. Control the dashcam with voice commands from your smartphone while in the car. If your home and car are close to each other and your home Wi-Fi signal can reach your car, you can play the car's dashcam footage from home via Wi-Fi and play the dashcam footage on your TV. Using voice recognition technology, a smartphone, smart speaker, or robot can automatically turn on a nearby dashcam, transfer your dashcam footage via Wi-Fi, and play it on your TV. Instead of giving a playback command to an AI speaker to play a YouTube video, you can give a playback command to the AI ​​speaker and play pre-registered dashcam footage on a smartphone, TV, etc. If the car's engine is off, it runs on battery power. The dashcam itself only needs to function to receive commands. When the command comes via Wi-Fi, it turns on the entire system, runs on battery power, and transfers the footage. The cigarette lighter socket is turned off when the engine is turned off. The cigarette lighter socket is connected to a mobile battery, which then supplies power to the dashcam. Until now, you had to remove the SD card and insert it into a computer to play the footage, but now you can easily view it.

[0426] If you ask to play yesterday's dashcam footage, it will play yesterday's dashcam footage. You can specify the date of the footage to be played. If it's not available, you can just say "Not Found," or you can choose to play the footage from the day before yesterday instead of yesterday's footage. If you ask for footage from when you just went out, it will play the "most recent dashcam footage." There is a function to specify "when." Dashcams also record audio. Continuous recordings and event recordings also contain audio. You can play it as a stream, or if you ask for "do something" for background music, it will replace the dashcam audio and play background music from a music streaming service. You can watch dashcam footage with music as background music. Playback will play a digest focusing on the part where an event occurred. There's no point in playing a long video for too long. Skip around 30 seconds of the event part. You can specify the playback time of dashcam footage. You can play a digest of an event for about 5 minutes. Adjust the playback time so that it plays back as soon as possible. Unplayed footage is given priority. Played footage is recorded on the dashcam, smartphone, etc. While playing dashcam footage, if there is footage you want to watch later, you can say "keep this footage" and the file will be transferred from the dashcam to your smartphone via Wi-Fi, etc.

[0427] While driving and recording video on the dashcam, if you say "Keep this video on my smartphone," the dashcam will transfer it to your smartphone if you have one in the car. If you say "Keep this video on the dashcam," it will be recorded on the dashcam as an event record. You can also say "Start recording this video" or "Stop recording this video." It can also be recorded in a separate area as voice-command recorded data. If you say "Send this video" while recording, it will be sent to the server via LTE. If you say "Stop video transmission," it will stop sending to the server. If you say "Send video up to this point" while recording, the data recorded continuously up to the previous set time will be sent to the server via LTE. If you say "Start live streaming" on the dashcam, it will start live streaming. It will keep uploading via stream.

[0428] [13th Example] Figures 63 to 65(A), 65(B), 65(C), 65(D) and 65(E) show a thirteenth embodiment.

[0429] The thirteenth embodiment relates to thumbnail image display.

[0430] Fig. 63 is a flowchart showing the processing steps for displaying thumbnail images in a playback device such as the personal computer 50. Fig. 64 is an example of thumbnail image display. Figs. 65(A)-(E) are examples of thumbnail image display formats.

[0431] This embodiment is suitable for displaying thumbnail images of circularly recorded video captured by a semi-spherical camera or a spherical camera, but can also be applied to displaying thumbnail images of other video.

[0432] 64, when SD card 23 on which video data representing a plurality of videos is recorded is loaded into personal computer 50, playback software is started, and playback window 100 is displayed on the display screen of display device 52, thumbnail images Ith1-Ithn of the plurality of videos are displayed, for example, in play list display area 141 of playback window 100. When cursor 531 is placed on and clicked on any of thumbnail images Ith1-Ithn of the plurality of videos (YES in step 521 of FIG. 63), pop-up menu 530 for selecting the display format of the thumbnail image is displayed on the clicked thumbnail image (step 522 of FIG. 63).

[0433] Referring to FIG. 64, the pop-up menu includes thumbnail image display menus of "Fisheye View", "Enlarged Flat View", "Panorama View", "Ring View" and "Dome View".

[0434] "Fisheye display" displays still images of video captured using a fisheye lens as thumbnail images, as shown in Figure 65(A). "Enlarged flat display" displays areas of interest to the user in a larger scale than other areas as thumbnail images, as shown in Figure 65(B). "Panoramic display" displays a single horizontally long image covering 360 degrees as thumbnail images, as shown in Figure 65(C). "Ring display" displays thumbnail images in a ring shape, as shown in Figure 65(D). "Dome display" displays thumbnail images in a dome shape, as shown in Figure 65(E).

[0435] A pop-up menu 530 is used to specify how any thumbnail images are displayed. If a display method is specified (YES in step 523), the clicked thumbnail image is displayed in the specified display method (step 524). Even if there are multiple thumbnail images, each thumbnail image can be displayed in a different display method.

[0436] In software for managing the playback of video files, thumbnail displays are known that display a reduced version of the first image of a video, allowing users to view the video for purposes such as selecting a desired video. Such thumbnail displays typically display a reduced version of the first image of a file or a still image of a specified frame. However, because video from a hemispherical camera is recorded in a circular format, thumbnail displays that allow users to select the display state are desirable, but generally, it is not possible to change the thumbnail display format or the display range. On the other hand, some smartphone apps allow users to switch between thumbnail displays by tapping the icon, change the thumbnail display format, and move the video display range by flicking the thumbnail, but the thumbnail display method is switched all at once.

[0437] For this reason, as mentioned above, a function is provided to change the thumbnail display method and display range for each individual thumbnail. Right-clicking on an individual thumbnail displays a pop-up menu from which the user can select the display method for the clicked thumbnail. The types of thumbnail display methods mentioned above are recommended. As mentioned above, fisheye, panoramic, enlarged flat, ring, and dome display methods are recommended. Original, twin cut, panoramic cut, and angle cut options may also be provided. Left-clicking on an individual thumbnail changes the image display range. It is recommended that the display range be changed. A "180 degree rotation" or "90 degree rotation" menu should be provided in the right-click menu. In the case of two cameras, a "camera switch" option is recommended. A function is provided to save the changed thumbnail display method and display range.

[0438] As a general rule, it is best to configure it so that the state of any changes made is maintained. It has a function to undo / redo changes to the thumbnail display method and display range. It has a function to align the thumbnail display method and display range between thumbnails. A selection frame will be drawn around the thumbnail when left-clicking while holding down the shift key. This is displayed by right-clicking on any thumbnail when there are multiple selection frames. The display method selected in the pop-up menu will change for all thumbnails with selection frames. Also, when there are multiple selection frames, left-dragging any thumbnail will change the state of all thumbnails with selection frames in the range displayed by that thumbnail in real time. Note that the selection frame should be released (the selection state will be canceled) when the left-click on an area other than the thumbnail is released.

[0439] It is recommended to provide rows of thumbnails that are considered to be from the same time (for example, the same frame) but have different display methods or display ranges. For example, with two 360-degree cameras, you can provide a row of thumbnails with a 360-degree fisheye view of the front and a row of thumbnails with a 360-degree fisheye view of the rear, and draw thumbnails from the same time in the same column. It is recommended to provide a function that allows you to add thumbnail rows as desired. To add thumbnails, create a space between the rows in the thumbnail display area, right-click that space, and select "Insert Row" from the pop-up menu that appears. This pop-up menu should also have options such as "Delete Row Above" and "Delete Row Below." It is recommended that you use a scroll bar to scroll through rows or columns that exceed the display range of the screen. For example, create 10 rows of thumbnails using two 360-degree cameras, and from the top, you can choose from front fisheye view, rear fisheye view, front-centered panoramic view, rear-centered panoramic view, bottom-centered enlarged flat view, top-centered enlarged flat view, ring view with the front as the most visible area, ring view with the rear as the most visible area, and dome view with the front as the most visible area. It is recommended to display it like a dome display with the back being the most visible part. The same column will display these types of images at the same time. You can also scroll left and right to move the columns left and right and see the entire thumbnail.

[0440] It is a good idea to provide a function that plays the image of a thumbnail when the thumbnail is selected by double-clicking, etc., and it is especially good to start playback using the display method and display range set for the thumbnail. It is also a good idea to provide a "Set this state as thumbnail" option in the pop-up menu that appears when you right-click in the playback window, and when selected, set the image with the display method and display range at that time as the thumbnail for that image. Thumbnail display involves creating an area to display the original image corresponding to the right-clicked thumbnail, and providing a function to clearly indicate the display area of ​​the current thumbnail within that original image (by surrounding it with a line, filling it with semi-transparent paint, etc.). This makes it easy to see which part of the original image is set as the thumbnail.

[0441] It would be a good idea to have a function that clearly shows the "jointed parts of the image." For example, you could draw a red line over the stitched parts. As the stitching precision improves, the joints become less visible. You don't want to use joints in commercial photography, but if they become less visible, you might end up trimming across the joints. This could be a problem if you discover broken joints after delivery, such as after commercial printing. Having such a function would help.

[0442] It should have a "function to clearly indicate the reflected portion of the main body." For example, it would be good to have a function to fill the area corresponding to the main body with a shaded or semi-transparent fill. The viewer screen should have an "area to indicate whether the image is blurred or not." An icon can be displayed to indicate that the video being played is a blurred image, or an icon indicating a blurred image can be displayed in the file name list. In relation to "wireless live streaming," it should have a function to record live streaming along with the recorded video, store information that identifies which recorded video is being streamed, and indicate whether the recorded video is live streaming or not when played in the viewer. It is fine to simply record live streaming and non-live streaming content separately and store information about whether it is live streaming in the file properties. However, it is especially good to distinguish between live streaming and non-live streaming content within a single file and display the two areas clearly on the timeline.

[0443] It is recommended that the system be equipped with a function that adds and records information that identifies the video as time-lapse footage, and that indicates whether or not it is time-lapse footage when playing it back in the viewer. It is also recommended that the system be equipped with a function that handles a series of time-lapse footage as a single collection. It is also recommended that the system be equipped with a function that adds and records information about which device the recording start, recording stop, and setting instructions came from, and that displays this information in the viewer. It is also recommended that the system be equipped with a function that uses voice recognition to operate the parts equivalent to the viewer's buttons, especially a function that changes the display method and display range. It is also recommended that the system be equipped with a function that uses voice recognition to change the recording method and recording range.

[0444] A semi-spherical camera records in a circle. Some allow you to select the state of the thumbnail. There are long thumbnails, ones that can switch between two, and ones that spin around when you flick. Switching thumbnails switches multiple thumbnails at once. This file does not allow you to change between spherical and long thumbnails, so it would be good to have a function that allows you to do this. It would be good to have a function that allows you to right-click on a thumbnail and have a pop-up menu that allows you to select the type of display. There are various ways to display thumbnails in a forklift drive recorder, such as fisheye display, panoramic display, and flat display, and it is important to be able to select these displays for each thumbnail. It would be good to be able to specify the display method. It would be good to have a function that changes the way the image is displayed by left-clicking. It would be good to have a function that moves by dragging within the thumbnail. A right-click menu could be set up to rotate 90 degrees or 180 degrees. In the case of two cameras, a camera switching function could be set up. It would be good to have a function that saves the changed thumbnail display method and display range. It would be good to be able to save the state of the thumbnail. If you rotate it, the rotated state will be saved. This can be used when you only want to see the driver.

[0445] Add a button to undo changes to the thumbnail display method and display range. Reset the thumbnail to its original state after making an operation. Undo a small movement and return to its original state. It would be nice to have a function to align thumbnails. To align all thumbnails within the driver's area, aligning them one by one is tedious. For example, hold down the control key and left-click to create an outer frame, then right-click to display a pop-up menu with display options. Selecting a thumbnail changes the display method for all thumbnails at once. Dragging the contents while the selection frame is open should move everything. Clicking an unrelated area will clear the selection frame. It would be nice to have rows that display thumbnails from the same time but with different display methods or display ranges. I want to create thumbnail rows with different perspectives. While there is only one thumbnail row per camera, it would be nice to have a function to display thumbnails from different perspectives. For example, with a 360-degree camera, it would be nice to display thumbnails from the front and rear cameras. Adding rows would be useful, but the columns should be aligned at the same time. It would be good to be able to add more rows of thumbnails as desired. It would also be good to be able to view them in a variety of display methods and angles. It would be good to have space between the rows of thumbnails, and then right-clicking in that space would insert and display a new row of thumbnails. When a row is inserted, it would be good to have the copied image displayed as the default. You can change them individually, or you can display a selection frame and change them all at once.

[0446] Left double-clicking a thumbnail to select it will play the thumbnail video. Playback will be in the display method and display range set for the thumbnail when playback begins. If a circular thumbnail is displayed, double-clicking will play it in circular form. If a long, thin thumbnail is displayed, the video will be played in the same way. The video being played will be made into a thumbnail. Right-clicking on the playback window will display a menu for making it into a thumbnail, and the displayed video will be made into a thumbnail. It is recommended that the default be the first frame of the file.

[0447] It would be good to have a function that shows where it is in the whole picture. It shows where the thumbnail is in the displayed image.

[0448] With a spherical image, it is necessary to connect the front and back images, but the connection is noticeable, so it would be beneficial to have a function that clearly displays the connection. The line appears when a button is pressed on the PC viewer. Reflection reduction. While parts of the image, such as hands and the camera body, are processed and removed, it would be beneficial to provide a function that clearly indicates these parts by using a shaded area or other means. For images with automatic face blurring, it would be beneficial to have the PC viewer indicate this. Faces can also be automatically blurred on the website where dashcam footage is uploaded. A recording function can be provided along with live streaming, and commands to start and stop streaming can be recorded in the continuous recording. This allows you to see which parts of the recorded video were streamed. The live streaming section can be displayed in pink on a slide bar. Time-lapse shooting can be set up to take still images every hour or 10 seconds. It would be beneficial to include a mark indicating that the video was recorded in a continuous recording. Time-lapse footage can also be combined into a single video.

[0449] It starts taking pictures based on commands from some device. For example, when a door opens, it will take pictures of your finger. A message will appear. Record which device started and ended recording from. Receive and record device IDs, etc. Name each button on the viewer. Change settings by voice. Once set, the dashcam will respond with the settings and begin recording with those settings.

[0450] Furthermore, a SIM card contract is required to upload video data captured from a drive recorder to the cloud using a mobile phone network such as LTE. Until now, drive recorder video viewers have lacked functionality specifically related to fees and data capacity. This has created the problem of not being able to tell from the viewer how much data is being transmitted or whether the pricing plan is appropriate.

[0451] Drive recorders for businesses are likely to have features that accommodate a variety of communication fee plans depending on how the company uses its vehicles.

[0452] Low-speed flat-rate unlimited data plans, nighttime plans, upload plans, corporate share plans, etc. These are useful features for low-speed flat-rate unlimited data plans priced between 300 and 500 yen per month. This feature supports SIM cards with speeds of around 200 kbps, which are unable to transmit high-speed data. 200 kbps allows for audio and still image (JPEG) transmission at speeds faster than real-time. Even mono 64 kbps provides sufficient sound quality to determine whether the sound is from an accident. Therefore, past data can be transmitted in a shorter time than real-time. This feature calculates the bit rate that shortens the time required for past data transmission based on the current (most recent) communication bit rate or the SIM's contracted bit rate, and compresses and transmits the data at that bit rate. This feature allows past data to be sent as a compressed file and current data to be sent via streaming (IP phone is also acceptable). This SIM card has a data limit of up to a certain amount of data per period, such as 366 MB over three days. It would be useful to have a function that displays the remaining transfer capacity on the dashcam and viewer. When you are approaching your data limit, it is a good idea to limit the type of information you send. For example, you can change it so that you don't send photo data but only audio data. A low-speed SIM card that can perform burst transfers (burst function: the first 1-2 seconds will produce a few Mbps). When an event occurs, start sending by sending several photos (even just one) from different points in time related to the event, then send the audio data. Photos are transferred in bursts.

[0453] A function that supports nighttime plans. One company's "nighttime plan" limits download and upload speeds to a maximum of 200 kbps between 6:00 and 22:00, but offers high-speed data transfers of up to 375 Mbps download and 50 Mbps upload during the nighttime hours from 22:00 to 6:00, at a low cost of 3 GB / month. It would be ideal to have a function that allows you to set times when high-speed data transfer is available or when data transfer restrictions are imposed, and upload videos during these times. It would also be especially ideal to have a function that automatically turns on dashcams in parked vehicles with the engine off during times when high-speed data transfer is available, and begins transmitting video data.

[0454] Functions compatible with upload-only plans. One company's "upload-only plan" limits the download data speed to a maximum of 200 kbit / s, but ensures a wide upload data speed of up to 50 Mbit / s for transmitting data. It is a relatively inexpensive plan at 1.1 GB. Since it is likely that the bandwidth for downstream IP telephones can be secured, it would be good to have a function that limits the server to instruction commands and downstream IP telephone communications. In other words, even if a dashcam has a function to receive video footage from other dashcams, it would be good to limit that function.

[0455] Functions that support corporate sharing plans. In particular, corporate plans are expected to use data sharing plans, such as sharing 50GB among 20 lines. It would be good to have a function that obtains information from the management screen provided by SIM providers via scraping or API, counts SIM usage on the dashcam, obtains this information at the center, and displays it in the viewer. It would also be good to have a function that displays the information comparatively. The total packet usage of dashcams that share packets It would be a good idea to provide a function in the viewer that displays packet usage and remaining available capacity. It would also be good to have a function that issues an alert when remaining capacity is low and to display a graph of predicted capacity usage. It would also be good to provide a function that displays individual packet usage and remaining available capacity for dashcams that share packets in a ranking format in the viewer. For example, by looking at the event status of a vehicle with a top-ranked dashcam, it would be easy to verify whether the driver is driving poorly or whether the dashcam settings are incorrect. For companies that have multiple sharing groups, it would be good to provide a function that displays usage statuses compared between sharing groups. This would provide a reference for optimizing the allocation of SIM cards for each dashcam to each group. It would also be good to use automatic analysis, such as AI, to advise users to move specific SIM cards to other groups. It would also be good to provide a display of file size in the selection field for what to display in the viewer.

[0456] For SMS-compatible SIMs, it is recommended that when a trigger occurs, the SIM card be equipped with a function that generates a string of characters that includes information about the level of maximum G and the importance of the trigger factor, and sends this string via SMS to the center phone number.The center (e.g., a viewer) should decide whether to send a transmission request instruction to the dashcam based on the SMS data.The center should then determine information such as the type of data to be transmitted (e.g., video) and the transmission range, and send the transmission request instruction to the dashcam.

[0457] In the case of a SIM card that supports voice calls, it is recommended that the dashcam be equipped with a voice call function and that the center be equipped with a function to forward external calls to the center (office). The center should obtain the dashcam's operating status via LTE or other means and have a function that does not forward external calls when set conditions are met, such as when the dashcam is in a driving state. The dashcam should also be equipped with a function that calls the center using the appropriate method based on the dashcam's status, such as the price comparison between IP and voice calls, the person being called, and past driving history. It is also recommended that the dashcam be equipped with a function that calls by voice when a strong impact occurs and by IP phone when a weak impact occurs, and a function that calls the center via VoIP and connects to external lines via phone.

[0458] When using both public Wi-Fi and LTE, it is recommended to have a function to connect to the public Wi-Fi, send restricted data via LTE, and send video data with no or loose restrictions via the public Wi-Fi. It is also recommended to have a function to upload from free Wi-Fi at convenience stores, etc.

[0459] There are many different pricing plans for dashcams aimed at businesses. For the low-speed, low-cost 200k plan, audio is sent first. The server identifies whether an accident sound is recorded and, if the driver wants to see the accident, sends a video transmission request to the dashcam. A few still images can also be sent. It's a good idea to vary the bit rate. Past data can be sent as a compressed file, and the current data can be sent as a stream. Past data can be stored in a file rather than an event, and transmitted in small chunks. The current data can be sent as a stream of audio and video. It's like connecting to an IP phone when an accident occurs. Audio is received by the center. SIM cards, for example, have a three-day data transmission limit. This limit applies to users with heavy usage. For such SIM cards, usage is displayed on the dashcam's display screen, showing the remaining transmission capacity. The viewer can also transmit data in real time and display it in a list. It's a good idea to limit the type of information sent when approaching the data limit.

[0460] Among low-speed SIM cards, there are some that are designed specifically for burst transfers. A few megabits are available for the first 1-2 seconds, but after that, the speed is limited to 200k. Event-related video can be sent first, followed by audio. Video can be sent during times when high-speed transmission is possible, such as with special nighttime plans, and audio only can be sent during restricted times. Video can be sent from dashcams in parked cars at night. With special upload plans, downloads are restricted. For example, if you are receiving footage from someone else's dashcam, this is restricted. With corporate share plans, the data can be acquired and viewed on the viewer. You can also obtain SIM usage information in the viewer.

[0461] It is a good idea to compare carrier usage with actual measurements and display it. Display when you are approaching the limit. It is a good idea to display a ranking of packet usage. Display usage by group. It is a good idea to display file size.

[0462] If the SIM card supports SMS, a string of characters indicating the time and minute of the trigger occurrence is generated and sent to the dashcam based on the SMS data. It is best to use it like an old pager. Only important data is sent via SMS for immediate response.

[0463] With a voice-enabled SIM, calls can be made using a regular phone number, and external calls to the center are forwarded to the dashcam's phone number. The dashcam can be used to talk hands-free. In the event of a strong impact, calls are made by voice, and in the case of a weak impact, calls are made by IP phone. In the event of an emergency, a stable network is used. Calls to the center are made via VOIP, packet communication, or IP phone.

[0464] In the case of public wireless LAN, the dashcam is equipped with a public wireless LAN connection function, and when connected via public wireless LAN, it will constantly record and upload the data.

[0465] The scope of the present invention is not limited to the structures expressly described in the specification, but also includes combinations of various aspects of the invention disclosed herein. While the structures of the invention sought to be patented are specified in the accompanying claims, it is the intention to claim structures disclosed herein in the future, even if they are not currently specified in the claims.

[0466] The present invention is not limited to the configurations described in the above-described embodiments. The components of each of the above-described embodiments and variations may be arbitrarily selected and combined. Furthermore, any component of each embodiment or variation may be arbitrarily combined with any component described in the Summary of the Invention or any component embodying any component described in the Summary of the Invention. The present application also intends to obtain rights to these by filing an amendment or divisional application of the present application.

[0467] Furthermore, by filing a conversion application to a design application, the applicant intends to obtain rights to the overall design or partial design. The drawings depict the entire device in solid lines, but they also include partial designs claimed for parts of the device. For example, it is possible to include some components of the device as partial designs, and it is also possible to include some parts of the device as partial designs regardless of the components. A part of the device may be a part of the device, or it may be part of that component. The applicant intends to obtain rights not only to the overall design, but also to partial designs in which any part of the solid line portion of the drawings is shown as a broken line. [Explanation of symbols]

[0468] 1: Drive recorder, 3: Windshield, 4: Room mirror, 5: Cigarette lighter socket, 6: Power cable, 10: SD card slot, 11: Display, 12: Operation buttons, 13: Joint rail, 14: Memory card reader / writer, 15: Speaker, 16: GPS receiver, 17: Camera, 18: Acceleration sensor, 19: Communication circuit, 20: Controller, 20b: ROM, 20c: RAM, 20d: Timer, 21: LTE module, 23: SD card, 30: Recording settings window, 30a: Recording method selection frame 30b: Recording resolution selection frame, 30c: Recording frame rate selection frame, 30d: Event recording frame rate selection frame, 30e: Event recording time selection frame, 30f: Transmission setting frame, 30g: First transmission resolution setting frame, 30h: First transmission frame rate setting frame, 30i: Second transmission resolution setting frame, 30j: Second transmission frame rate setting frame, 30k: Save instruction button, 50: Computer, 51: Display control device, 52: Display device, 54: Memory card reader / writer, 55: Input device, 56: Memory, 57: Communication circuit, 58: CPU, 96: Input device, 100: Playback window 101: Menu bar, 102: Folder icon, 103: Camera icon, 104: Printer icon, 105: Reel icon, 106: FD icon, 107: Log icon, 108: Gear icon, 111: Video display area, 112: Video, 112a: Video part, 112b: Video part, 113: Video, 113a: Video part, 113b: Video part, 114: Video, 114a: Video part, 114b: Video part, 121: Map image display area, 131: Vehicle Information display area, 132: Vehicle status display area, 133: Operation buttons, 134: Interior highlight button, 135: Normal display button, 136: Time display area, 138: Acceleration graph display area, 141: List display area, 160: Video display window, 161: Menu, 162: Cursor, 180: Display screen, 190: Display screen, 200: Computer, 201: Administrator, 210: External display device, 211: Driver, C1: Brightness conversion curve, C2: Brightness conversion curve, t3: Time when event occurs

Claims

[Claim 1] A program executed on another computer to play back video recorded by a drive recorder capable of setting a frame rate for continuous recording video and a frame rate for event recording video to different frame rates, the program comprising: A program for causing the other computer to realize a function of displaying both the frame rate of video recording on a drive recorder that can set different frame rates for the video of continuous recording and the frame rate of video of event recording, and the frame rate of playback on the other computer on the video being played back, and displaying the frame rate of playback on the other computer as changing depending on the capabilities of the other computer, thereby making it clear that the frame rate of video recording on the drive recorder cannot be reproduced.

Images