Regeneration control system and computer program

Abstract

To provide a novel technique capable of suppressing a user's sense of discomfort caused by mismatch between the orientation of a camera and the travel direction of a mobile.SOLUTION: A system according to one aspect of a disclosure acquires moving image data indicative of an image captured by a camera 20 provided in a mobile. Further, in order that a captured image whose front direction is adjusted to the travel direction of the mobile is displayed on a display device 40 as a reproduced image based on the moving image data, control data for image-processing the moving image data and displaying the reproduced image on the display device is generated. In addition, the control data is written in a storage device in association with the moving image data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a playback control system and a computer program. [Background technology]

[0002] An imaging device equipped with a center position correction function is known to reduce the discomfort felt by users viewing a captured image due to the vanishing point being displaced from the center of the captured image when a vehicle turns a corner (see, for example, Patent Document 1). This imaging device calculates the position of the vanishing point based on the optical flow of the captured image, and corrects the captured image so that the position of the vanishing point coincides with the center of the captured image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-10145 Summary of the Invention [Problem to be solved by the invention]

[0004] The phenomenon of a user viewing a captured image feeling unnatural can occur when the camera is pointing in a direction that is not aligned with the vehicle's direction of travel, even when the vehicle is traveling in a straight line. In this case, correcting the captured image can also be considered.

[0005] However, there is room for improvement in the conventional techniques for correcting captured images. For example, a drive recorder is known that records moving image data on a portable recording medium or a server device.

[0006] A user can play back video data recorded on a recording medium or a server device on another device outside the vehicle, such as a personal computer. However, with conventional technology, the user cannot use a correction function to suppress the sense of discomfort on the other device, and has to view the captured image without correction.

[0007] Therefore, according to one aspect of the present disclosure, it is desirable to provide a novel technology suitable for suppressing the sense of discomfort felt by a user in a reproduced image caused by the camera orientation being deviated from the traveling direction of a moving object. For example, it is desirable to provide a novel technology that can suppress the sense of discomfort felt by a user caused by the camera orientation being deviated from the traveling direction of a moving object even in an environment where video data can be reproduced on a separate device away from the drive recorder. [Means for solving the problem]

[0008] According to one aspect of the present disclosure, there is provided a playback control system including an acquisition unit and a setting unit, wherein the acquisition unit is configured to acquire video image data representing images captured by a camera provided on a mobile object.

[0009] The setting unit is configured to generate control data for processing the moving image data and displaying a reproduced image on the display device, the reproduced image being a captured image whose front direction is adjusted to the traveling direction of the moving object, as a reproduced image based on the moving image data. The setting unit is further configured to write the control data to the storage device in association with the moving image data.

[0010] According to this playback control system, as described above, control data associated with video data is written to a storage device. Therefore, when a user attempts to view a playback image based on the video data, the playback device for the video data can process the video data based on the control data associated with the video data so that the front direction of the playback image coincides with the direction of travel of the moving object, and display the playback image on a display device, without analyzing the video data or requiring the user to specify or search for appropriate image processing.

[0011] Therefore, according to one aspect of the present disclosure, a new technology can be provided that can advantageously suppress the sense of discomfort felt by the user due to the camera orientation not matching the direction of travel of the moving object.

[0012] According to one aspect of the present disclosure, the setting unit may be configured to determine a relative relationship between the orientation of the camera and the direction of travel of the moving object based on the video data, and to generate control data based on the relative relationship. With this configuration, even if the user does not accurately understand the relative relationship, appropriate control data can be generated that allows the playback image to be displayed in a way that suppresses the sense of incongruity.

[0013] According to one aspect of the present disclosure, the setting unit may be configured to display a setting screen on the display device that is separate from the playback screen on which the playback image is displayed, and to accept setting operations through the setting screen.

[0014] The setting screen may be a screen for accepting a setting operation by a user regarding the adjustment. The setting unit may be configured to generate control data based on the setting operation. With this configuration, the control data can be generated according to the user's desire or with assistance from the user.

[0015] According to an aspect of the present disclosure, the setting unit may be configured to determine the relative relationship between the orientation of the camera and the direction of travel of the moving object based on the setting operation, and generate the control data.

[0016] According to an aspect of the present disclosure, the moving object may be a vehicle. The traveling direction of the moving object may be a straight-ahead direction of the vehicle. The setting unit may be configured to determine the relative relationship based on a specific reference object inside or outside the vehicle that is captured in the captured image.

[0017] There are suitable reference objects inside and outside the vehicle for determining the relative relationship. Therefore, based on the specific reference objects shown in the replay image, the relative relationship can be determined appropriately, and image processing for the adjustment can be performed appropriately.

[0018] According to an aspect of the present disclosure, the setting unit may be configured to determine the relative relationship based on an optical flow of the captured image. Based on the optical flow, the relative relationship can be determined even when the specific reference object is not clearly visible in the captured image.

[0019] According to an aspect of the present disclosure, the setting operation may include an operation of pointing to a specific reference object inside or outside the vehicle that appears in the captured image within the captured image displayed on the setting screen. The setting unit may determine a relative relationship based on the specific reference object pointed to by the setting operation.

[0020] With this configuration, the relative relationship can be determined with assistance from the user, and the relative relationship can be determined more easily and with higher accuracy than if a specific reference object were identified by image analysis without user instruction.

[0021] According to one aspect of the present disclosure, the specific reference object may include, for example, one of the windshield inside the vehicle, a road dividing line outside the vehicle, and the horizon outside the vehicle that appear in the captured image.

[0022] According to an aspect of the present disclosure, the setting operation may include an operation of specifying an operation mode of the setting unit, and the setting unit may be configured to determine a relative relationship between the operation mode specified by the setting operation among the plurality of operation modes.

[0023] The multiple operating modes may include two or more of an operating mode that determines the relative relationship based on a first reference object inside the vehicle that appears in the captured image as a specific reference object, an operating mode that determines the relative relationship based on a second reference object outside the vehicle that appears in the captured image as a specific reference object, and an operating mode that determines the relative relationship based on optical flow in the captured image.

[0024] By determining the relative relationship and generating control data based on a specified one of multiple operation modes, it is possible to determine and generate control data in an operation mode that meets the user's wishes, or in an operation mode that the user considers to be optimal in terms of reducing discomfort and / or discrimination accuracy.

[0025] According to an aspect of the present disclosure, there may be provided a playback control system including an acquisition unit, a playback control unit, and a setting unit, wherein the acquisition unit is configured to acquire video data representing images captured by a camera provided on a mobile object.

[0026] The playback control unit can be configured to control the display device so that a playback image corresponding to the captured image is displayed on a playback screen of the display device based on the moving image data acquired by the acquisition unit.

[0027] The setting unit may be configured to display a setting screen separate from the playback screen on the display device and to accept a setting operation through the setting screen. The setting screen may be a screen for accepting a setting operation by a user to adjust the front direction of the playback image to the traveling direction of the moving object.

[0028] The setting unit may be further configured to set an image processing method for the moving image data for the adjustment based on the setting operation. The playback control unit may be configured to cause the display device to display, as a playback image, a captured image whose front direction has been adjusted to coincide with the traveling direction of the moving object by processing the moving image data in accordance with the image processing method set by the setting unit.

[0029] With this configuration, an appropriate image processing method can be set based on user operation, and the front direction of the reproduced image can be aligned with the direction of travel of the moving object with high precision, effectively suppressing any sense of discomfort the user may have with the reproduced image.

[0030] According to an aspect of the present disclosure, when the moving object is a vehicle, the setting operation may include an operation of pointing to a specific reference object inside or outside the vehicle that appears in the captured image within the captured image displayed on the setting screen. The setting unit may be configured to determine a relative relationship between the orientation of the camera and the straight-ahead direction of the vehicle based on the specific reference object pointed to by the setting operation, and to set an image processing method for the video image data for adjusting the forward direction of the reproduced image to the straight-ahead direction of the vehicle based on the relative relationship.

[0031] According to one aspect of the present disclosure, some or all of the functions of the playback control system described above may be realized by a computer program.

[0032] According to one aspect of the present disclosure, a computer program may be provided to cause a computer to perform processing including acquiring video data representing an image captured by a camera mounted on a moving body, image processing the video data to generate control data for displaying the reproduced image on a display device as a reproduced image based on the video data, with the captured image having its front direction adjusted to the direction of travel of the moving body, and writing the control data to a storage device in association with the video data.

[0033] According to one aspect of the present disclosure, there may be provided a computer program causing a computer to execute a process including: acquiring video data representing images captured by a camera provided on a moving object; controlling the display device based on the acquired video data so that a playback image corresponding to the captured image is displayed on a playback screen of the display device; causing the display device to display, as a setting screen separate from the playback screen, a screen for accepting a setting operation by a user to adjust the front direction of the playback image to the traveling direction of the moving object, and accepting the setting operation through the setting screen; and setting an image processing method for the video data for the adjustment based on the setting operation. Controlling the display device may include causing the display device to display, as a playback image, the captured image whose front direction has been adjusted to the traveling direction of the moving object by image processing the video data in accordance with the set image processing method.

[0034] According to one aspect of the present disclosure, a computer program may be provided to cause a computer to execute a process of acquiring the video data and control data associated with the video data from a storage device, and displaying a playback image corresponding to a captured image recorded in the video data on a display device based on the acquired video data and control data.

[0035] According to one aspect of the present disclosure, a computer program may be provided that causes a computer to acquire video data and ancillary data associated with the video data from a storage device, and to display on a display device a replay image corresponding to a captured image recorded in the video data by image processing the acquired video data, wherein the captured image is an image captured by a camera mounted on a moving body, and the ancillary data includes information that can identify the difference between the orientation of the camera and the direction of travel of the moving body, and displaying the replay image on the display device includes image processing the video data based on the acquired ancillary data so that the captured image is displayed on the display device with its front direction adjusted to the direction of travel of the moving body as the replay image.

[0036] These computer programs can be recorded on a computer-readable non-transitory recording medium or provided via an electric communication line. [Brief explanation of the drawings]

[0037] [Figure 1] FIG. 2 is a block diagram illustrating a configuration of a drive recorder. [Figure 2] FIG. 2 is a functional block diagram of a control unit. [Figure 3] 10 is a flowchart showing a reproduction-related process executed by a control unit. [Figure 4] 10 is a flowchart showing a setting process executed by a control unit. [Figure 5] FIG. 10 is a diagram illustrating a setting mode designation operation. [Figure 6] FIG. 10 is a diagram illustrating an example of a screen displaying a setting completion message. [Figure 7] FIG. 10 is a diagram illustrating a setting operation using the "windshield." [Figure 8] 10A and 10B are diagrams illustrating a setting operation using the "horizon line." [Figure 9] FIG. 10 is a diagram illustrating a setting operation using a "white line." [Figure 10] FIG. 10A is an explanatory diagram regarding the display of a reproduced image, and FIG. 10B is an explanatory diagram regarding the display of a reproduced image without adjustment as a comparative example. [Figure 11] 10A and 10B are explanatory diagrams relating to display of a playback image in a full view panoramic display mode. [Figure 12] FIG. 12A is a diagram illustrating an adjusted round image, and FIG. 12B is a diagram illustrating an unadjusted round image as a comparative example. [Figure 13] FIG. 10 is an explanatory diagram of a modified drive recorder in which moving image data is taken out of the vehicle via a server device. DETAILED DESCRIPTION OF THE INVENTION

[0038] Exemplary embodiments of the present disclosure will now be described with reference to the drawings. A drive recorder 1 according to this embodiment shown in FIG. 1 is an electronic device that is mounted on and used in a vehicle such as a four-wheeled automobile as a mobile object.

[0039] For example, the drive recorder 1 is used by being fixed to a room mirror (in other words, a rearview mirror) inside the vehicle cabin that is used by the driver to view the area behind the vehicle. The room mirror is usually located in the center in the left-right direction above the windshield (in other words, the windshield). Alternatively, the drive recorder 1 is configured as a mirror-type drive recorder that also functions as a room mirror. Alternatively, the drive recorder 1 is fixed to the windshield or another location inside the vehicle cabin via a stay and is used independently of the room mirror.

[0040] 1, the drive recorder 1 includes a control unit 10, a camera 20, sensors 30, a display unit 40, an operation unit 50, and a media reader / writer 60. The control unit 10 includes a processor 11 and a memory 13.

[0041] The memory 13 includes a volatile memory (RAM) used as a working area when the processor 11 executes processing, and a non-volatile memory (NVRAM) such as an electrically rewritable flash memory. The non-volatile memory stores computer programs and data that should be retained even when the computer is shut down.

[0042] The processor 11 controls each part of the drive recorder 1 by executing processes according to a computer program stored in the memory 13. The processes executed by the control unit 10 described below are realized by the processes executed by the processor 11 according to the computer program.

[0043] The camera 20 is provided, for example, on a surface of the drive recorder 1 facing the windshield, and is used to capture images of the surroundings of the vehicle including the area ahead of the vehicle. The camera 20 may be a wide-angle camera or an omnidirectional camera capable of capturing images in a 360-degree azimuth angle.

[0044] Examples of wide-angle cameras include ultra-wide-angle cameras that can capture images of an azimuth angle of 180 degrees or more. Ultra-wide-angle cameras and omnidirectional cameras may be cameras configured by combining multiple wide-angle cameras. Video data representing captured images of the vehicle's surroundings generated by camera 20 is input to control unit 10.

[0045] The sensors 30 are composed of one or more devices for measuring the vehicle's driving conditions. The sensors 30 include, for example, a positioning device such as a GPS receiver for measuring the vehicle's position and an acceleration sensor for measuring the vehicle's acceleration. The acceleration sensor can be used, for example, to record video data representing captured images of the vehicle's surroundings when a vehicle impact is detected as important data. Measurement signals from the sensors 30 are input to the control unit 10.

[0046] The display unit 40 has a display surface facing the interior of the vehicle and is configured as a display device that displays information for vehicle occupants such as the driver through the display surface. The display unit 40 is configured as, for example, a liquid crystal display. The display unit 40 is controlled by the control unit 10 to display various types of information.

[0047] The operation unit 50 is configured as a user interface capable of receiving operations from a user corresponding to the driver or other vehicle occupants. For example, the operation unit 50 is provided with a touch panel on the display surface of the display unit 40, and is configured to receive user operations on the display surface through user operations on the touch panel. Operation signals from the user are input to the control unit 10 via the operation unit 50.

[0048] The media reader / writer 60 is configured to allow a portable recording medium 70 to be connected and disconnected, and is configured to read and write data from and to the connected recording medium 70 in response to a request from the control unit 10. The recording medium 70 may be a memory card with a built-in flash memory. Examples of the recording medium 70 include an SD card.

[0049] Next, the functions realized by the control unit 10 will be described with reference to Fig. 2. The control unit 10 functions as a video recording unit 110, a playback unit 120, an image processing unit 130, and a setting unit 150 by executing processes according to a computer program.

[0050] The video recording unit 110 is configured to write video data input from the camera 20 to the recording medium 70 via the media reader / writer 60. For example, the video recording unit 110 is configured to divide the video data input from the camera 20 into predetermined time segments and write the video data to the recording medium 70 as a recording file in which images captured at the corresponding predetermined time segments are recorded. The main function of the drive recorder 1, that is, the function of recording captured images of the vehicle's surroundings while the vehicle is traveling, is realized by the video recording unit 110.

[0051] The video recording unit 110 may be configured to record video data representing images captured during a limited period of time, based on input from the sensors 30, onto the recording media 70, rather than continuously recording video data.

[0052] The video data recorded on the recording medium 70 may include data that can identify the shooting date and time and the shooting location. The shooting location may be the GPS location at the time of shooting, which can be obtained from the sensors 30.

[0053] When a playback operation is performed by the user in accordance with an input signal from the operation unit 50, the playback unit 120 reads out video image data in which images captured during a period designated by the user are recorded (for example, one or more video files in which images captured during the corresponding period are recorded) from the recording medium 70 via the media reader / writer 60. The playback unit 120 performs playback processing on the data acquired by the readout, thereby outputting playback images, specifically video images, corresponding to the images captured during the designated period.

[0054] The reproduced image output from the reproduction unit 120 is a captured image of the entire imaging range corresponding to the angle of view 2α of the camera 20. For example, if the camera 20 is an omnidirectional camera having an angle of view 2α of 360 degrees in azimuth, the reproduced image output from the reproduction unit 120 may be a panoramic image with an azimuth angle of 360 degrees.

[0055] The image processing unit 130 functions as a playback control unit in cooperation with the playback unit 120, and is configured to perform image processing on the playback image input from the playback unit 120, convert it into a playback image for display, and display the converted playback image for display on the display unit 40. The image processing is performed in accordance with playback control data attached to the video data that is recorded together with the video data on the recording medium 70. The playback control data and image processing will be described in detail below.

[0056] The playback image input from playback unit 120 to image processing unit 130 is converted by the image processing described above into a playback image for display whose front direction corresponds to the traveling direction of the vehicle at the time of shooting. As a result, a playback image that does not look strange to the user is displayed on display unit 40.

[0057] In addition, the reproduced image displayed on the display unit 40 has an aspect ratio of 16:9 and is an image with an angle of view 2β narrower than the angle of view 2α of the camera 20. That is, in the image processing, a reproduced image with an angle of view 2β is generated by extracting a portion of the reproduced image with the angle of view 2α.

[0058] In order to cause the image processing unit 130 to execute the above-mentioned image processing of aligning the front direction of the playback image with the direction of travel of the vehicle, the setting unit 150 is configured to generate playback control data in accordance with the user's setting operation via the operation unit 50, and to record this playback control data in association with the video data on the same recording medium 70 as the video data.

[0059] Next, the details of the playback-related processing that the control unit 10 executes to function as the playback unit 120, the image processing unit 130, and the setting unit 150 will be described with reference to Figures 3 and 4. The control unit 10 repeatedly executes the playback-related processing shown in Figure 3 while the drive recorder 1 is running.

[0060] When the playback-related process starts, the control unit 10 waits until an operation signal is input from the user via the operation unit 50 (S110). When an operation signal is input (Yes in S110), the control unit 10 executes the process from S120 onwards.

[0061] In S120, the control unit 10 determines whether the user's operation according to the input operation signal is a setting operation for adjusting the front direction of the playback image to the traveling direction of the vehicle (S120). If it is determined to be a setting operation (Yes in S120), the control unit 10 proceeds to S125 and executes setting processing for the above adjustment. Thereafter, the control unit 10 ends the playback-related processing.

[0062] On the other hand, if it is determined that the user's operation is not the setting operation but an operation to play video data recorded on the recording medium 70 (No in S120 and Yes in S130), the control unit 10 switches the screen displayed by the display unit 40 to a playback screen and accepts an operation to specify a playback period (S140). The playback screen can be provided with a graphical user interface for accepting an operation to specify a playback period.

[0063] When the user specifies the playback period by a specification operation, the control unit 10 executes a process of displaying on the playback screen of the display unit 40 playback images corresponding to the captured images recorded in the video image data in the recording medium 70 that were captured during the period corresponding to the playback period (S145).

[0064] Here, the playback screen displays the playback image, and instead of the graphical user interface for accepting the operation of specifying the playback period, it can display a graphical user interface for accepting operations from the user regarding video playback, such as changing the playback speed, fast-forwarding, rewinding, and stopping.

[0065] In S145, the control unit 10 functions as the playback unit 120 and the image processing unit 130, and performs image processing that takes into account the discrepancy between the orientation of the camera 20 and the direction of travel of the vehicle, based on the playback control data recorded on the recording media 70 together with the video image data.

[0066] That is, the control unit 10 performs playback processing on the video image data and performs image processing in a direction that suppresses misalignment, thereby causing the display unit 40 to display a playback image corresponding to the captured image as seen from a line of sight along the traveling direction of the vehicle at the time of capture. After playback is completed, the control unit 10 returns the screen displayed on the display unit 40 from the playback screen to the screen before switching (S150), and ends the playback-related processing shown in FIG.

[0067] In addition, when a user operation other than the setting operation and the playback operation is performed (No in S120 and No in S130), the control unit 10 executes a process corresponding to the performed operation (S160) and ends the playback-related process.

[0068] Next, details of the setting process executed by control unit 10 in S125 will be explained using Figure 4. By executing the setting process, control unit 10 functions as setting unit 150. When the setting process starts, control unit 10 switches the screen displayed by display unit 40 to a setting screen (S210). The screen before switching to the setting screen may be a main screen (not shown) in which a graphical user interface for accepting main operations is superimposed on an image currently captured by camera 20.

[0069] The setting screen after switching can be a screen in which a graphical user interface for specifying a setting mode for the above adjustment is superimposed on the image currently being captured by the camera 20, as shown in FIG.

[0070] According to this embodiment, the drive recorder 1 has a "vector" setting mode, a "windshield" setting mode, a "horizon" setting mode, and a "white line" setting mode as setting modes related to the above adjustment.

[0071] As shown in Fig. 5, the setting screen may have a graphical user interface for accepting designation of one of these setting modes from the user. The setting screen exemplified in Fig. 5 has selection buttons for each setting mode as a graphical user interface. When one of these selection buttons is pressed via the operation unit 50, the control unit 10 can determine that the setting mode corresponding to the pressed selection button has been designated. The setting screen may further have a graphical user interface for canceling the setting operation.

[0072] After displaying the setting screen, the control unit 10 waits until an operation to designate one of the setting modes or a cancel operation is performed (S220, S230, S240, S250, S280). When an operation to designate the "vector" setting mode is performed as an operation to designate one of the setting modes (Yes in S220), the control unit 10 detects a vanishing point in the captured image based on the optical flow of the captured image obtained by analyzing captured images from the present to a predetermined period ago that are recorded in the video image data (S225).

[0073] The control unit 10 further estimates the difference Dv in the horizontal direction of the captured image between the detected vanishing point and the center line extending vertically in the captured image as the difference δθ in the azimuth angle between the front direction of the camera 20 and the straight-ahead direction of the vehicle (S225). The horizontal direction of the captured image corresponds to the azimuth angle. The difference Dv can be expressed as a signed distance in the horizontal direction of the captured image from the center line of the captured image to the vanishing point.

[0074] Thereafter, the control unit 10 generates playback control data describing the estimated difference Dv and writes the generated playback control data to the recording medium 70 (S260). This playback control data is interpreted as data instructing that a playback image be generated by performing image processing on the video data so as to shift the center of the captured image recorded in the video data in the horizontal direction of the captured image by the amount of the difference Dv.

[0075] By image processing based on this reproduction control data, a reproduction image for display is generated in which the front direction of the reproduction image corresponds to the straight-ahead direction of the vehicle.

[0076] If playback control data has already been recorded on the recording medium 70 before the playback control data is written in S260, the control unit 10 can write the newly generated playback control data to the recording medium 70 in a format that overwrites the recorded playback control data.

[0077] As another example, the control unit 10 can write newly generated playback control data to the recording medium 70 in an appended format, without overwriting already recorded playback control data. In this case, the playback control data can include registration date and time information, and the image processing unit 130 (in other words, the control unit 10) can process the playback image from the playback unit 120 based on the latest playback control data among the playback control data registered before the corresponding shooting date and time. Alternatively, the image processing unit 130 can process the image based on playback control data specified by the user.

[0078] After the processing in S260, the control unit 10 temporarily displays a setting completion message explaining the completion of the setting on the display unit 40 as shown in FIG. 6, and then returns the screen displayed by the display unit 40 to the screen before switching to the setting screen in S210 (S270), thereby terminating the setting processing.

[0079] Examples of the screen before switching include the main screen described above. On the main screen after the setting process, a captured image that has been subjected to image processing based on the playback control data generated immediately before may be displayed as a playback image corresponding to the image currently captured by camera 20. That is, a captured image whose front direction has been adjusted to the vehicle's straight-ahead direction may be displayed on display unit 40 as a playback image. Based on playback control data describing the difference Dv, an image captured by camera 20 that has been image-processed so that the vanishing point is positioned on the center line of the playback image may be displayed on display unit 40.

[0080] The screen displaying the setting completion message output in the processing of S270 may also display the captured image that has been subjected to image processing based on the playback control data generated immediately before, along with the setting completion message, as shown in FIG. 6.

[0081] In addition, when the user performs an operation to specify the "windshield" setting mode through the setting screen (Yes in S230), the control unit 10 receives an operation from the user to trace the boundary of the windshield, and causes the display unit 40 to display a panoramic image of the image currently captured by the camera 20 on the setting screen, as shown in Fig. 7, and also displays a message on the setting screen instructing the user to trace the boundary of the windshield that appears in the panoramic image (S231). The panoramic image is a captured image of the entire shooting range corresponding to the angle of view 2α of the camera 20.

[0082] The user can trace the boundary of the windshield shown in this panoramic image using the operation unit 50. The dotted line in Fig. 7 indicates the boundary of the windshield to be traced. The panoramic display on the setting screen is performed so that the entire windshield is visible, thereby enabling the user to trace the boundary of the entire windshield.

[0083] When the user finishes tracing the boundary of the windshield, the control unit 10 estimates the difference Df in the azimuth angle between the front direction of the camera 20 and the front direction of the windshield as the difference δθ based on the distortion of the shape of the windshield in the panoramic image specified by the user's operation (S235). The front direction of the windshield corresponds to the straight-ahead direction of the vehicle. The difference Df can be expressed as a signed angle in the azimuth angle from the front direction of the windshield to the front direction of the camera 20.

[0084] An actual windshield has a bilaterally symmetrical shape. Therefore, if the difference Df is zero, the windshield shown in the panoramic image is bilaterally symmetrical with respect to the center line extending vertically from the center of its shape. In contrast, if the difference Df is non-zero, the opposite sides extending horizontally are not parallel, and the vertical lengths of the windshield at positions equidistant from the center line are different. From this difference in length, the difference Df can be geometrically estimated. When estimating the difference Df, the lens characteristics of the camera 20 (e.g., distortion aberration) can also be taken into consideration.

[0085] After estimating the difference Df in S235, the control unit 10 executes the processes of S260 and S270 described above, and then ends the setting process. As a result, the difference Df estimated in S235 is written in the playback control data and stored in the recording medium 70. This playback control data is interpreted as data instructing that a playback image be generated by performing image processing on the video data so as to shift the center of the captured image recorded in the video data in the azimuth direction of the captured image by the difference Df.

[0086] In addition, when the user performs an operation to specify the "horizon" setting mode through the setting screen (Yes in S240), the control unit 10 receives an operation from the user to trace the horizon, and causes the display unit 40 to display a panoramic view of the image currently being captured by the camera 20 on the setting screen, as shown in FIG. 8, and also displays a message on the setting screen instructing the user to trace the horizon that appears in the image captured by the camera 20 (S241).

[0087] When the user finishes tracing the horizon, the control unit 10 estimates the difference Dh in the azimuth angle between the front direction of the camera 20 and the front direction of the windshield as the difference δθ based on the inclination of the horizon in the panoramic image identified by the user's operation with respect to the bottom edge of the windshield identified by image analysis (S245). The difference Dh can be expressed as a signed angle in the azimuth angle from the front direction of the windshield to the front direction of the camera 20.

[0088] The horizon appears the same in the panoramic image regardless of the orientation of the camera 20 in terms of azimuth. In contrast, the bottom edge of the windshield changes depending on the orientation of the camera 20, as explained above in the "Windshield" setting mode. If the difference Dh is zero, the horizon and the bottom edge of the windshield are symmetrical with respect to the center line of the panoramic image, and if there is no distortion due to the lens characteristics (distortion aberration) of the camera 20, the horizon and the bottom edge of the windshield are approximately parallel. In contrast, if the difference Dh is non-zero, the bottom edge of the windshield is not symmetrical with respect to the center line of the panoramic image, and the horizon appears tilted relative to the bottom edge of the windshield. From this tilt, the difference Dh can be estimated.

[0089] After estimating the difference Dh in S245, the control unit 10 executes the processes of S260 and S270 described above, and then ends the setting process. As a result, the difference Dh estimated in S245 is written in the playback control data and stored in the recording medium 70. This playback control data is interpreted as data instructing that a playback image be generated by performing image processing on the video data so as to shift the center of the captured image recorded in the video data in the azimuth direction of the captured image by the difference Dh.

[0090] In addition, when the user performs an operation to specify the "white line" setting mode through the setting screen (Yes in S250), the control unit 10 proceeds to S251, and in order to receive an operation from the user to trace a white line as a road dividing line, as shown in FIG. 9, the display unit 40 displays a panoramic display of the image currently captured by the camera 20 on the setting screen, and also displays a message on the setting screen instructing the user to trace the white line appearing in the image captured by the camera 20 (S251).

[0091] The white lines in this case refer to the two outer perimeter lines on both sides of the road. The outer perimeter lines are usually displayed as solid white lines on the road. The user is required to trace the two outer perimeter lines.

[0092] When the user finishes tracing the white lines, the control unit 10 estimates the difference Dw in the horizontal direction of the panoramic image between the vanishing point and the center line extending up and down the panoramic image as the difference δθ based on the inclination of the white lines in the panoramic image specified by the user's tracing (S255). The intersection of the two white lines when extended corresponds to the vanishing point. Therefore, the difference Dw can be estimated based on the difference between the intersection and the center line.

[0093] After estimating the difference Dw in S255, the control unit 10 executes the processes of S260 and S270 described above, and then ends the setting process. As a result, the difference Dw estimated in S255 is written to the playback control data and stored in the recording medium 70. This playback control data is interpreted as data instructing that a playback image be generated by performing image processing on the moving image data so as to shift the center of the captured image recorded in the moving image data in the horizontal direction by the difference Dw.

[0094] In addition, when a cancel operation is performed (Yes in S280), the control unit 10 returns the screen displayed on the display unit 40 to the screen before switching to the setting screen in S210 (S290), and ends the setting process.

[0095] Next, an image processing method in image processing unit 130 based on the playback control data will be described. As described above, the playback control data stores differences Dv, Df, Dh, and Dw that are substantially equivalent to the difference δθ in the azimuth angle between the front direction of camera 20 and the straight-ahead direction of the vehicle. Therefore, hereinafter, an image processing method when there is a difference δθ in the azimuth angle between the front direction of camera 20 and the straight-ahead direction of the vehicle will be described with reference to FIG. 10A.

[0096] Additionally, the forward direction of the camera 20 referred to here corresponds to the line of sight direction from the camera 20 toward a subject located at the center of an image captured by the camera 20, in terms of azimuth angle. For the sake of explanation, the azimuth angle θ is defined below as an azimuth angle in which the forward direction of the vehicle, i.e., the straight-ahead direction of the vehicle, is set to zero.

[0097] 10A shows that the front direction of camera 20 is shifted by azimuth angle δθ from the straight-ahead direction of the vehicle. Here, it is assumed that the angle of view of camera 20 is 2α, and that when the front direction of camera 20 coincides with the straight-ahead direction of the vehicle, the captured images recorded in the video image data are captured images that capture the field of view within the azimuth angle range of -α≦θ≦+α.

[0098] 10A, when the front direction of camera 20 is shifted by δθ in the negative azimuth direction with respect to the straight-ahead direction of the vehicle, images captured within the azimuth angle range of -α-δθ≦θ≦+α-δθ are recorded in the video data. That is, the capture range of the video data is within the azimuth angle range of -α-δθ≦θ≦+α-δθ.

[0099] At this time, the reproduced image from the reproduction unit 120 is a moving image with a shooting range of azimuth angles -α-δθ≦θ≦+α-δθ, with the image captured at azimuth angle δθ being placed at the center of the reproduced image as if it were at an azimuth angle of zero. In other words, the reproduced image from the reproduction unit 120 is a reproduced image with a field angle of 2α, with the center, in other words, the front direction of the reproduced image, at an azimuth angle of δθ.

[0100] When displaying this playback image on display unit 40, image processing unit 130 extracts a playback image with an azimuth angle of -β≦θ≦+β from the playback image with an angle of view 2α output from playback unit 120 based on the playback control data, as shown in Fig. 10A. This extraction generates a playback image for display. As described above, the playback image for display is a playback image with an aspect ratio of 16:9, and is different from the above-mentioned panoramic image that represents the entire shooting range corresponding to angle of view 2α of camera 20.

[0101] Image processing unit 130 displays the playback image for display generated by the extraction on display unit 40, thereby displaying a playback image with a field angle of 2β in which the front direction of the playback image (in other words, the center of the playback image) coincides with the straight-ahead direction of the vehicle on display unit 40. As a result, even if camera 20 is facing in a direction deviated from the straight-ahead direction of the vehicle, display unit 40 displays a moving image as the image captured by camera 20, with the image center adjusted to an azimuth angle of zero and the display range being in the range of azimuth angles -β≦θ≦+β.

[0102] FIG. 10B shows, as a comparative example, the correspondence between the display range of the reproduced image on the display unit 40, the front of the camera 20, and the vehicle's straight-ahead direction (azimuth angle zero) when the image processing unit 130 does not consider the difference Δθ, regards the center of the reproduced image from the reproduction unit 120 as having an azimuth angle of zero, and extracts a reproduced image with a narrower angle of view 2β from the reproduced image with an angle of view 2α to generate a reproduced image for display. In this case, the front direction of the reproduced image displayed on the display unit 40 does not match the straight-ahead direction of the vehicle. As a result, the reproduced image displayed on the display unit 40 feels strange to the user. People are accustomed to the flow of their field of vision when facing forward, so if the front direction of the reproduced image displayed on the display unit 40 does not match the straight-ahead direction of the vehicle, the user will feel uncomfortable.

[0103] According to the drive recorder 1 of the present embodiment described above, the setting unit 150 generates playback control data for image processing of moving image data in accordance with the user's setting operation via the operation unit 50 and the setting screen so that a captured image with the front direction adjusted to the straight-ahead direction of the vehicle is displayed on the display device, and writes this data to the same recording medium 70 as the moving image data. The generation of the playback control data and the writing of the data to the recording medium 70 correspond to the setting of the image processing method.

[0104] Therefore, even when the recording medium 70 is removed from the media reader / writer 60 and the user views a playback image based on the video data recorded on the recording medium 70 on another device such as the user's own personal computer, the other device can process the video data so that the front direction and the direction of travel coincide, and display the playback image on a display device, without analyzing the video data or without the user specifying or searching for appropriate image processing.

[0105] For example, if software for causing a personal computer to execute a playback process similar to the process executed by the control unit 10 at S145 is provided to the user of the drive recorder 1, for example, via a web page, the user can effectively utilize the playback control data generated by the drive recorder 1 to view a playback image on the personal computer, with the front direction matching the direction of travel of the vehicle, just like the drive recorder 1, through a display device built into or connected to the personal computer.

[0106] The display unit 40 of the drive recorder 1 has a smaller display surface size than a display device used in a personal computer. Therefore, it would be convenient for the user if the playback images based on the moving image data recorded on the recording medium 70 could be viewed on the personal computer.

[0107] According to this embodiment, the user can not only view playback images based on the video data on a personal computer, but also view playback images that have the front direction adjusted to the direction of travel of the vehicle without any troublesome settings on the personal computer, which makes it possible to provide a convenient drive recorder 1 for the user.

[0108] In particular, with a rearview mirror type or rearview mirror-mounted drive recorder 1, the orientation of the camera 20 changes relative to the vehicle's straight-ahead direction not only when the drive recorder 1 is mounted but also when the orientation of the rearview mirror is changed, making the functions of the drive recorder 1 of this embodiment extremely useful.

[0109] According to the present embodiment, it is also advantageous that a plurality of setting modes for adjusting the front direction of the playback image to the traveling direction of the vehicle, i.e., a plurality of operation modes of the setting unit 150, are provided. The setting unit 150 can switch the setting mode in response to a user's operation to specify a setting mode, and can generate playback control data by accepting the user's setting operation in the specified setting mode.

[0110] The setting unit 150 determines the differences Df, Dh, and Dw that indicate the relative relationship between the orientation of the camera 20 and the straight-ahead direction of the vehicle using the windshield inside the vehicle, the horizon outside the vehicle, and road dividing lines (white lines) outside the vehicle as specific reference objects inside or outside the vehicle that appear in the captured image.

[0111] For example, in the "windshield" setting mode, the setting unit 150 determines the relative relationship based on the distortion of the shape of the windshield inside the vehicle, based on the user pointing at the windshield by tracing the boundary of the windshield through the setting screen. At this time, the display unit 40 displays a screen including a panoramic image with a full angle of view different from the 16:9 captured image displayed on the playback screen as a setting screen separate from the playback screen, to allow the user to perform appropriate setting operations by pointing at the windshield.

[0112] In the "horizon" setting mode, the setting unit 150 determines the relative relationship based on the positional relationship between the horizon outside the vehicle shown in the panoramic image and the bottom edge of the windshield inside the vehicle, based on the user pointing to the horizon through a setting screen that also displays a panoramic image different from the playback screen.

[0113] In the "white line" setting mode, the setting unit 150 determines the relative relationship based on the arrangement of white lines outside the vehicle shown in the panoramic image, based on the user pointing to the white lines through a setting screen that also displays a panoramic image different from the playback screen.

[0114] The reference object suitable for determining the relative relationship varies depending on the scenery captured in the image captured by camera 20. By selecting a setting mode, the user can select an appropriate reference object and perform the setting operation for the adjustment described above, which allows image processing unit 130 to align the front direction of the reproduced image with the straight-ahead direction of the vehicle with high precision. In particular, the setting screen displays a panoramic image rather than a 16:9 image so that the specific reference object is easily captured, allowing the user to appropriately point to the specific reference object and perform the setting operation for the adjustment.

[0115] Furthermore, according to this embodiment, the setting unit 150 can determine the relative relationship based on the optical flow of the captured image in the "vector" setting mode, without using a specific reference object. Therefore, according to this embodiment, even in an environment around the vehicle where a specific reference object is not clearly visible in the captured image, the direction between the front direction of the reproduced image and the traveling direction of the vehicle can be appropriately aligned by driving the vehicle.

[0116] Furthermore, the present disclosure is not limited to the above-described embodiment, and various other aspects may be employed. For example, the drive recorder 1 may have a plurality of display modes for displaying playback images. In S145, the control unit 10 may control the display unit 40 to display images captured during the playback target period as playback images in a display mode designated by the operation unit 50. In this case, the image processing unit 130 may perform image processing so that the playback images input from the playback unit 120 are displayed in the designated display mode.

[0117] For example, when the camera 20 is an omnidirectional camera, the drive recorder 1 can have a standard display mode, which is a display mode that displays a playback image with an aspect ratio of 16:9, as well as a panoramic display mode and a round display mode.

[0118] The full view panoramic display mode is a mode in which the entire captured image recorded in the video data, with a field of view 2α of 360 degrees (=2π), is displayed panorama-style on the display unit 40. That is, α=β=180 degrees (=π).

[0119] In the panoramic display mode, as shown in FIG. 11, the image processing unit 130 can convert the playback image input from the playback unit 120 into a playback image for display by treating the image area with an azimuth angle of -π-δθ≦θ<-π of the moving image whose shooting range is in the range of azimuth angles -π-δθ≦θ≦+π-δθ as an image area with an azimuth angle of +π-δθ≦θ<+π.

[0120] That is, the image processing unit 130 can execute image processing to generate, as a reproduced image for display, a captured image in the range of azimuth angles -π≦θ≦+π, with azimuth angle zero at the center of the reproduced image, by changing the arrangement of corresponding image areas in the reproduced image input from the reproduction unit 120 so as to move image areas with azimuth angles -π-δθ≦θ<-π to image areas with azimuth angles +π-δθ≦θ<+π. The image processing by the image processing unit 130 or the reproduction processing by the reproduction unit 120 can include processing to suppress image distortion due to lens characteristics, for a good panoramic display.

[0121] As a result, the display unit 40 displays a 360-degree panoramic image captured by the camera 20, centered on the vehicle's straight-ahead direction, i.e., a 360-degree panoramic image in which the front direction of the reproduced image coincides with the vehicle's straight-ahead direction.

[0122] In addition, the round display mode is a mode in which the entire 360-degree captured image recorded in the video data is displayed in a circular shape on the display unit 40, as shown in Fig. 12A. The circumferential direction of the circle corresponds to the azimuth angle direction. Here, the 360-degree captured image displayed in a circular shape is referred to as a round image.

[0123] In the round view display mode, the image processing unit 130 processes the playback image input from the playback unit 120 based on the playback control data, and displays a round view image on the display unit 40, in which the subject portion with an azimuth angle of zero, which corresponds to the traveling direction of the vehicle, is positioned at the top of the circle. The front direction of the round view image corresponds to the upward direction of the circle.

[0124] If it is not based on the playback control data, the upward direction of the round image will not match the direction in which the vehicle is traveling, and the round image will be displayed with the direction in which the vehicle is traveling tilted from the upward direction of the round image by the difference δθ, as shown in Figure 12B.

[0125] The image processing unit 130 can rotate such tilted round image by the difference δθ through image processing based on the playback control data, and display on the display unit 40 a round image in which the subject part with an azimuth angle of zero, which corresponds to the direction of travel of the vehicle, is facing up the circle.

[0126] In the above embodiment, image processing has been described in a direction that cancels out the difference δθ in the azimuth angle between the front direction of camera 20 and the straight-ahead direction of the vehicle. However, along with the difference δθ in the azimuth angle, the difference δφ in the elevation angle between the front direction of camera 20 and the reference direction may also be estimated, and the estimated value may be recorded in the playback control data.

[0127] During playback processing of video data, image processing may be performed in a direction that cancels out the difference δθ in the azimuth angle direction and the difference δφ in the elevation / depression angle direction so that a playback image taking the elevation / depression angles into account is displayed. The reference direction here may be a direction parallel to a horizontal plane along the straight-ahead direction of the vehicle, or may be a direction inclined downward from the horizontal plane, taking into account the general line of sight of a vehicle occupant.

[0128] In the above embodiment, an example has been described in which video data is provided to a device outside the vehicle via recording medium 70. However, the video data may also be provided to an external device 95, which is a device outside the vehicle, via server device 90, for example. In this case, the playback control data may be included in the video data as metadata, and may be recorded in storage 91 of server device 90 in a state associated with the main body of the video data. Server device 90 may also be understood to correspond to cloud storage.

[0129] According to the modified example shown in FIG. 13, assuming that the drive recorder 1 can communicate with the server device 90 via a wide area network, the control unit 10 of the drive recorder 1 writes the generated playback control data as metadata into the video data at S260, and uploads the written video data to the server device 90.

[0130] The video data including the playback control data uploaded to the server device 90 is downloaded by an external device 95. Examples of the external device 95 include a personal computer, a smartphone, and a tablet. When playing back the video data, the external device 95 refers to the playback control data included as metadata and performs image processing that takes into account the discrepancy between the orientation of the camera 20 and the traveling direction of the vehicle.

[0131] Since the drive recorder 1 can provide video data and playback control data to the external device 95 through the server device 90, the user can conveniently view playback images based on the video data on the external device 95 without handling the recording media 70.

[0132] In particular, the user can view a natural playback image in which the front direction is adjusted to match the traveling direction of the vehicle without any troublesome settings on the external device 95. Therefore, according to this modification, a convenient drive recorder 1 can be provided for the user.

[0133] In the above embodiment, the playback control data is generated by the drive recorder 1, but the video data may be provided to the external device 95 via the server device 90, and the external device 95 may execute processing corresponding to the setting processing shown in Fig. 4 to generate the playback control data. In this case, the playback control data may be recorded in the server device 90 as metadata of the video data.

[0134] The video data with the playback control data can be downloaded from the server device 90 to the drive recorder 1 or the external device 95 and played back. Information processing terminals equipped with touch panels, such as smartphones, are easy to use. Therefore, by generating the playback control data in accordance with user operations via the information processing terminal, it is possible to improve the ease of use for user settings.

[0135] 4, the drive recorder 1 may have a manual setting mode in addition to the above-described "vector" setting mode, etc. In the manual setting mode, the control unit 10 displays a panoramic image captured by the camera 20 on the setting screen and can accept an operation from the user to adjust the center position of the panoramic image.

[0136] The control unit 10 regards the amount of adjustment of the center position of the adjusted panoramic image from the position before adjustment as the deviation between the orientation of the camera 20 and the direction of travel of the vehicle, and generates playback control data.When playing back video data, the control unit 10 can perform image processing based on the playback control data so that the adjusted center position is displayed at the center of the screen.

[0137] The technology of the present disclosure may be applied not only to a drive recorder mounted on a vehicle as a moving object, but also to a mobile terminal equipped with a camera, such as a smartphone. The drive recorder 1 described above has a built-in camera 20, but the drive recorder 1 may also be a drive recorder connected by cable or wirelessly to a separate camera mounted on the vehicle. The graphical user interface does not need to be superimposed on the playback image or captured image, and may be displayed in an area separate from the playback image or captured image area.

[0138] The function of one component in the above embodiments may be distributed among multiple components. The functions of multiple components may be integrated into one component. Part of the configuration of the above embodiments may be omitted. At least part of the configuration of the above embodiments may be added to or substituted for the configuration of another of the above embodiments. All aspects included in the technical idea identified from the wording of the claims are embodiments of the present disclosure. [Explanation of symbols]

[0139] 1...Drive recorder, 10...Control unit, 11...Processor, 13...Memory, 20...Camera, 30...Sensors, 40...Display unit, 50...Operation unit, 60...Media reader / writer, 70...Recording media, 90...Server device, 91...Storage, 95...External device, 110...Video recording unit, 120...Playback unit, 130...Image processing unit, 150...Setting unit.

Claims

1. an acquisition unit configured to acquire video image data representing images captured by a camera provided on the vehicle; a setting unit configured to generate control data for processing the moving image data to display a captured image on the display device, the captured image having a front direction adjusted to the straight-ahead direction of the vehicle, as a playback image based on the moving image data, and to write the control data to a storage device in association with the moving image data; Equipped with the setting unit displays, on the display device, a screen for accepting setting operations by a user as a setting screen separate from the playback screen on which the playback image is displayed, accepts, as the setting operation, an operation of pointing, within the captured image displayed on the separate setting screen, at a specific reference object inside or outside the vehicle that appears in the captured image, determines a relative relationship between the orientation of the camera and the straight-ahead direction of the vehicle based on the specific reference object pointed by the setting operation, and generates the control data; A playback control system in which the specific reference object includes one of the windshield inside the vehicle and the horizon outside the vehicle that appears in the captured image.

2. 2. The playback control system of claim 1, the setting operation includes an operation of specifying an operation mode of the setting unit, the setting unit determines the relative relationship based on an operation mode designated by the setting operation among a plurality of operation modes; The plurality of operation modes include: an operation mode in which the relative relationship is determined based on a windshield inside the vehicle that appears in the captured image as the specific reference object; and an operation mode in which the relative relationship is determined based on the horizon outside the vehicle that appears in the captured image as the specific reference object; A playback control system including:

3. an acquisition unit configured to acquire video image data representing images captured by a camera provided on the vehicle; a playback control unit configured to control the display device so that a playback image corresponding to the captured image is displayed on a playback screen of the display device based on the moving image data acquired by the acquisition unit; and a setting unit configured to display, on the display device, a screen for accepting a setting operation by a user as a setting screen separate from the playback screen, accept, as the setting operation, an operation of pointing to a specific reference object inside or outside the vehicle that appears in the captured image displayed on the separate setting screen, determine a relative relationship between the orientation of the camera and the straight-ahead direction of the vehicle based on the specific reference object pointed to by the setting operation, and set an image processing method for the video image data based on the relative relationship; Equipped with the playback control unit is configured to process the moving image data in accordance with the image processing method set by the setting unit, thereby causing the display device to display, as the playback image, a captured image whose front direction is adjusted to a straight-ahead direction of the vehicle; A playback control system in which the specific reference object includes one of the windshield inside the vehicle and the horizon outside the vehicle that appears in the captured image.

4. 4. The regeneration control system according to claim 3, the setting operation includes an operation of specifying an operation mode of the setting unit, the setting unit determines the relative relationship based on an operation mode designated by the setting operation among a plurality of operation modes; The plurality of operation modes include: an operation mode in which the relative relationship is determined based on a windshield inside the vehicle that appears in the captured image as the specific reference object; and an operation mode in which the relative relationship is determined based on the horizon outside the vehicle that appears in the captured image as the specific reference object; A playback control system including:

5. A computer program for causing a computer to execute a process, The process comprises: acquiring video image data representing images captured by a camera installed in a vehicle; generating control data for performing image processing on the moving image data to display a captured image, the front direction of which is adjusted to the straight-ahead direction of the vehicle, on the display device as a reproduced image based on the moving image data; and writing the control data to a storage device in association with the video data; Including, Generating the control data includes displaying, on the display device, a screen for accepting setting operations by a user as a setting screen separate from the playback screen on which the playback image is displayed, accepting, as the setting operation, an operation of pointing, within the captured image displayed on the separate setting screen, at a specific reference object inside or outside the vehicle that appears in the captured image, determining a relative relationship between the orientation of the camera and the straight-ahead direction of the vehicle based on the specific reference object pointed by the setting operation, and generating the control data. The computer program, wherein the specific reference object includes one of the windshield inside the vehicle and the horizon outside the vehicle that appears in the captured image.

6. A computer program for causing a computer to execute a process, The process comprises: acquiring video image data representing images captured by a camera installed in a vehicle; controlling the display device so that a playback image corresponding to the captured image is displayed on a playback screen of the display device based on the acquired moving image data; displaying, on the display device, a screen for accepting a setting operation by a user as a setting screen separate from the playback screen, and accepting, as the setting operation, an operation of pointing, within the captured image displayed on the separate setting screen, to a specific reference object inside or outside the vehicle that appears in the captured image; determining a relative relationship between the orientation of the camera and the straight-ahead direction of the vehicle based on the specific reference object indicated by the setting operation; setting an image processing method for the moving image data based on the relative relationship; Including, controlling the display device includes processing the moving image data in accordance with the set image processing method to display, on the display device, as the playback image, a captured image whose front direction is adjusted to the straight-ahead direction of the vehicle; The computer program, wherein the specific reference object includes one of the windshield inside the vehicle and the horizon outside the vehicle that appears in the captured image.

Images