System, information processing method, program, etc

Abstract

To provide a system and the like capable of performing appropriate image processing based on the state or situation of a moving object.SOLUTION: A system according to an embodiment of the present invention includes a control unit. The control unit is configured to execute the following steps. In an acquisition step, specific information relating to a state or situation of the moving object is acquired. In a first correction step, on the basis of the specific information, a first correction process is executed on image data generated by a photographing device possessed by the moving object at a time corresponding to the specific information. In a display control step, the image data on which the first correction process has been executed is displayed in a reproducible manner.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and the like. [Background technology]

[0002] A technique has been disclosed in which, when a user selects an indoor emphasis button, brightness correction is performed within a position on video data representing a dark video displayed in a video display area (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-129530 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology of Patent Document 1 only has buttons on the screen for performing image processing on video data, and is unable to perform image processing desired by users for various scenes.

[0005] In view of the above circumstances, the present invention provides a system that is superior to conventional systems, such as a system that can perform appropriate image processing based on the state or situation of a moving object.

[0006] The object of the present invention is not limited to this, and the applicant intends to obtain rights for configurations that aim to achieve the effects achieved by parts of the configuration disclosed in the specification and drawings, etc., through divisional applications, amendments, etc. For example, this specification discloses problems in which the phrases "can" and "is possible" are read as "the problem is." Each problem is described as an independent problem, and the applicant intends to obtain rights for configurations that solve each problem separately through divisional applications, amendments, etc. Even if the problem is implicitly understood from the description in the specification, the applicant intends to claim part of the configuration described in this specification through amendments or divisional applications. Furthermore, the applicant has disclosed configurations that solve problems that combine these independent problems, and the applicant intends to obtain rights for them. [Means for solving the problem]

[0007] (1) According to one aspect of the present invention, a system is provided. The system may include a control unit. The control unit may be configured to execute the following steps: In the acquisition step, specific information regarding a state or situation of the moving object may be acquired; in the first correction step, based on the specific information, a first correction process may be executed on image data generated by a photographing device possessed by the moving object at a time corresponding to the specific information; and in the display control step, the image data on which the first correction process has been executed may be displayed in a reproducible manner.

[0008] In this way, the user can view an image reproduced from image data after the first correction process has been performed on image data generated by the image capture device at a time corresponding to the state or situation of the moving object. In particular, the correction process on the image data should be appropriate image processing. The image processing should be correction processing that corrects the image data to be easier for the user to view than image data generated by the image capture device possessed by the moving object. In particular, the correction process should be performed constantly. According to this aspect, appropriate image processing can be performed based on the state or situation of the moving object, thereby realizing a system that can constantly present image data that is easy for the user to view, regardless of the situation or scene.

[0009] For example, it may be provided in the following aspects.

[0010] (2) In the system, in the first correction step, the first correction process is performed on the image data while varying a correction range, which is the range to be corrected, based on the specific information.

[0011] According to this aspect, since it is not necessary to correct the entire range, the processing load can be reduced. Furthermore, since the correction is applied to a preferred range, the user's attention can be directed to the preferred range regardless of the spatial range or the temporal range.

[0012] (3) In the system, the correction range is a region of interest set in a frame of the image data.

[0013] According to this aspect, it is possible to narrow down the spatial area where it is originally desired to increase visibility and apply correction, so that the user's attention can be directed to a preferred range on a frame-by-frame basis.

[0014] (4) In the system, the control unit is configured to further execute the following steps: in a first receiving step, receiving a first input from a user; and in a canceling step, canceling the first correction processing on the image data based on the first input.

[0015] According to this aspect, when the correction process presented by the system is unnecessary, the user can disable it through their own input, thereby improving visibility and presenting an appropriate image depending on the situation or scene.

[0016] (5) In the system, the control unit is configured to further execute the following steps: in a second receiving step, receiving a second input from a user; and in a second correction step, performing a second correction process on the image data based on the second input.

[0017] According to this aspect, in addition to the correction processing presented by the device, it is possible to improve visibility by adding corrections at the user's discretion, and it is possible to present images that suit the user's preferences depending on the situation and scene.

[0018] (6) In the system, the display control step displays the image data in a reproducible manner in a manner that allows the first correction process and the second correction process to be distinguished from each other.

[0019] According to this aspect, the user can understand the type of correction processing, which is user-friendly.

[0020] (7) In the system, the specific information is at least one of the position, orientation, speed, acceleration, time, time zone, weather, driving condition, and pedal work of the moving object.

[0021] According to this aspect, at least one of the specific conditions of the moving object, namely, position, speed, acceleration, running state, and pedal work, is taken into consideration, and therefore, more appropriate image processing can be performed based on this information.

[0022] (8) In the system, the control unit is configured to further execute an acquisition step, wherein the specific information is the position of the moving body, the acquisition step acquires road network information, and the first correction step executes a first correction process on the image data based on the position and the road network information.

[0023] According to this aspect, it is possible to perform appropriate image processing depending on the location and road conditions at the time. For example, by performing appropriate image processing depending on the traffic volume obtained from road network information, it is possible to present an image that is easier for the user to view.

[0024] (9) In the system, the control unit is configured to further execute an acquisition step, wherein the specific information is the position of the moving body, the acquisition step acquires topographical information, and the first correction step executes a first correction process on the image data based on the position and the topographical information.

[0025] According to this aspect, it is possible to apply appropriate image processing depending on the location and the terrain, environment, etc. For example, by applying different image processing to a busy street and a forest, it is possible to present an image that is easier for the user to view.

[0026] (10) In the system, the specific information is the position and orientation of the moving body, and in the first correction step, a first correction process is performed on the image data based on the position of the sun estimated from the position and orientation.

[0027] According to this aspect, it is possible to perform appropriate image processing taking into account the position of the sun. For example, in a situation where the setting sun is shining in, the brightness of the area that is originally intended to be seen tends to decrease due to backlighting, so by performing image processing to correct this, it is possible to present an image that is easy for the user to see.

[0028] (11) In the system, the first correction step includes a preparation step and a processing step, and the control unit is configured to be able to call a pre-stored first program and a second program different from the first program, and the first program executes the preparation step, in which parameters included in the image data are extracted and the values ​​of the extracted parameters are passed to the second program, and the second program executes the processing step and the display control step, in which the first correction process is performed on the parameters in the processing step to change their values, and the display control step reproducibly displays the image data based on the parameter values ​​changed in the processing step.

[0029] According to this aspect, it is possible to use a separate program that is more specialized for correction processing, which contributes to reducing calculation costs and improving image processing performance. In particular, by passing parameter values ​​to the second program without duplicating the original image data, etc., the second program suitable for image processing can read the parameter values ​​and reproduce image data that has been subjected to image processing in accordance with the parameter values.

[0030] (12) In the system, the parameters include at least one of hue, brightness, saturation, contrast, gamma, sharpness, playback start time, playback end time, and whether or not to loop.

[0031] According to this aspect, the parameters such as hue, brightness, saturation, contrast, gamma, sharpness, playback start time, playback end time, and whether or not to loop are common parameters that image data has, so a more general-purpose image processing program can be used as the second program, and a highly usable system can be realized that is not limited to dedicated software.

[0032] (13) In the system, in the preparation step, operation information regarding the operation of the second program is passed to the second program, and in the display control step, at least some of the functions of the second program are restricted based on the operation information, and the image data is displayed in a reproducible manner.

[0033] According to this aspect, the second program can be used solely for correction and playback, and the user does not need to use any unnecessary functions of the second program, thereby preventing or eliminating problems such as incorrect operations or users finding the program difficult to use due to the large number of functions.

[0034] (14) In the system, the first program is configured to execute the display control step, and in the display control step, the second program is not installed. If you do not have it installed, a screen will appear recommending this installation.

[0035] According to this aspect, even if the second program is not installed, the user can be notified of a recommended second program, and as a result, the user can install the second program on his or her computer, and thereby reproduce image data that has been appropriately subjected to the first correction process.

[0036] (15) In the system, the second program is a program stored in an external device, and the control unit is configured to call the second program via a network.

[0037] According to this aspect, even if the second program is not installed, the first correction process can be performed via the second program. For example, even if the capacity of the computer being used is limited or a user does not have the authority to install new programs, the first correction process can be performed appropriately.

[0038] (16) A system comprising a control unit, the control unit being configured to execute the following steps: in a recording step, an image captured by a camera is recorded as image data; in a display control step, the image captured by the camera is displayed as a live view, and parameters of the recorded image data and the displayed image are set to be different.

[0039] According to this aspect, by differentiating the image viewed as a live view from the image data that is recorded, it is possible to ensure both visibility and customizability.

[0040] (17) In the system, the recording step records the image as the image data without correction, and the display control step corrects the image and displays it as the live view.

[0041] According to this aspect, it is possible to improve the visibility of the image viewed as a live view, while preventing information loss due to processing of the image data to be recorded.

[0042] (18) In the system, in the recording step, parameters of the displayed image are recorded in chronological order together with the image data as a parameter log.

[0043] According to this aspect, the visibility of the image viewed as a live view can be stored as a record, and this information can be used to reproduce the visibility later when playing back the image data.

[0044] (19) In the system, the control unit is configured to further execute a third reception step, in which the third reception step receives a third input from a user, and in the display control step, reads out the image data and the parameter log based on the third input and displays the corrected image data in a reproducible manner.

[0045] According to this aspect, when the user desires, it is possible to instantly play back video with high visibility, such as in live view, and it is possible to realize a system with high usability.

[0046] (20) In the system, the control unit is configured to be able to call a pre-stored third program and a fourth program different from the third program, and in the display control step, when the image data is played back using the fourth program, information encouraging playback using the third program is displayed.

[0047] According to this aspect, even if an unintended program is played back, it is possible to guide the user to play back a preferred program, and appropriate information can be provided to the user.

[0048] (21) In the system, the information prompting playback is displayed together with the playback screen of the image data.

[0049] According to this aspect, it is possible to provide the user with appropriate information that the current appearance is undesirable, but that there is no problem with the image data itself.

[0050] (22) An information processing method comprising the steps of the system.

[0051] According to such a method, the functionality of the above-described system can be realized and provided in the form of a method.

[0052] (23) A program that causes a computer to function as a control unit of the system.

[0053] According to this embodiment, the functions of the above-described system can be realized and provided in the form of a program. However, this is not a limitation.

[0054] The inventions described in (1) to (23) above can be combined arbitrarily. For example, a configuration may be created by adding at least a portion of the configuration of at least one of the inventions described in (2) and subsequent items to all or a portion of the configuration of the invention described in (1). In particular, an invention may be created by adding at least a portion of the configuration of at least one of the inventions described in (2) and subsequent items to the invention described in (1). Furthermore, any configuration may be extracted from the inventions described in (1) to (23) and combined. The applicant of this application intends to obtain rights to inventions including these configurations. Furthermore, even if a description is made of "in the case of..." or "when...," it is not intended to describe a configuration limited to that case or time. These are merely examples of better configurations, and the applicant intends to obtain rights to configurations other than those cases or times. Furthermore, any descriptions that specify an order are not limited to this order. Configurations in which some parts are deleted or the order is changed are also disclosed, and the applicant intends to obtain rights to them. [Effects of the Invention]

[0055] According to the present invention, appropriate image processing can be performed based on the state or situation of the moving object, thereby realizing a system that can consistently present image data that is easy for the user to see, regardless of the situation or scene.

[0056] The effects of the present invention are not limited to these, and effects achieved by the configuration disclosed in the present specification and drawings, etc. are also disclosed, and the applicant intends to obtain rights to the configuration achieving such effects through divisional applications, amendments, etc. For example, in this specification, phrases such as "can" and "is possible" are descriptions that clearly indicate the effects achieved, and there are also parts that demonstrate effects even without the phrases "can" and "is possible." Furthermore, there are effects that can be understood from the configuration even without such phrases. [Brief explanation of the drawings]

[0057] [Figure 1] FIG. 1 is a diagram showing an example of a system configuration of an information processing system according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating the configuration of the vehicle according to the first embodiment. [Figure 3] FIG. 3 is a diagram showing an example of the external configuration of the imaging device. [Figure 4] FIG. 4 is a diagram showing the configuration of the bracket. [Figure 5] FIG. 5 is a block diagram showing the electrical configuration of the imaging device. [Figure 6] FIG. 6 is a diagram illustrating an example of a hardware configuration of a server according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a hardware configuration of the information display terminal according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a functional configuration of the server according to the first embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a functional configuration of the information display terminal according to the first embodiment. [Figure 10] FIG. 10 is an activity diagram showing the flow of information processing by the server according to the first embodiment. [Figure 11] FIG. 11 is an activity diagram showing the flow of information processing of the information display terminal according to the first embodiment. [Figure 12]FIG. 12 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit according to the first embodiment. [Figure 13] FIG. 13 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit according to the first embodiment. [Figure 14] FIG. 14 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit of the first modification of the first embodiment. [Figure 15] FIG. 15 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit of Modification 2 of Embodiment 1. In FIG. [Figure 16] FIG. 16 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit of Modification 3 of Embodiment 1. In FIG. [Figure 17] FIG. 17 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit of the fourth modification of the first embodiment. [Figure 18] FIG. 18 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit of Modification 5 of Embodiment 1. In FIG. [Figure 19] FIG. 19 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit of Modification 6 of Embodiment 1. In FIG. [Figure 20] FIG. 20 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit of the seventh modification of the first embodiment. [Figure 21] FIG. 21 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit of Modification 8 of Embodiment 1. In FIG. [Figure 22] FIG. 22 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit of Modification 9 of Embodiment 1. In FIG. [Figure 23] FIG. 23 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the tenth modification of the first embodiment. [Figure 24] FIG. 24 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the eleventh modification of the first embodiment. [Figure 25] FIG. 25 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the twelfth modification of the first embodiment. [Figure 26] FIG. 26 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the thirteenth modification of the first embodiment. [Figure 27] FIG. 27 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the fourteenth modification of the first embodiment. [Figure 28] FIG. 28 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the fifteenth modification of the first embodiment. [Figure 29] FIG. 29 is a diagram showing an example of a screen displayed by the display control unit of the sixteenth modification of the first embodiment. [Figure 30] FIG. 30 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the seventeenth modification of the first embodiment. [Figure 31] FIG. 31 is a diagram showing an example of a screen displayed by the display control unit of Modification 18 of Embodiment 1. In FIG. [Figure 32] FIG. 32 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit in the nineteenth modification of the first embodiment. [Figure 33] FIG. 33 is a diagram showing an example of a screen displayed by the display control unit of the twentieth modification of the first embodiment. [Figure 34] FIG. 34 is a diagram showing an example of a screen displayed by the display control unit of the twenty-first modification of the first embodiment. [Figure 35] FIG. 35 is a diagram showing an example of a screen displayed by the display control unit in Modification 22 of Embodiment 1. In FIG. [Figure 36] FIG. 36 is a diagram showing an example of a screen displayed by the display control unit in the twenty-third modification of the first embodiment. [Figure 37] FIG. 37 is a schematic diagram showing the system configuration of an information processing system according to a modified example of the second embodiment. [Figure 38] FIG. 38 is a schematic diagram illustrating a functional configuration of an information processing terminal according to a modified example of the second embodiment. [Figure 39] FIG. 39 is a functional block diagram realized by the storage unit 22 and the control unit 23. [Figure 40]FIG. 40 is an activity diagram showing the flow of information processing according to the fourth embodiment. [Figure 41] FIG. 41 is a schematic diagram showing an example of a spatial correction range in the first correction process. [Figure 42] FIG. 42 is a schematic diagram showing an example of the temporal correction range of the first correction process. [Figure 43] FIG. 43 is a schematic diagram showing additional information 43 added to the playback screen 4 when the first correction process and the second correction process are executed. [Figure 44] FIG. 44 is an activity diagram showing the flow of information processing according to the third modification of the fourth embodiment. [Figure 45] FIG. 45 is a functional block diagram realized by the control unit 33. [Figure 46] FIG. 46 is an activity diagram showing the flow of information processing according to the fifth embodiment. [Figure 47] FIG. 47 is a schematic diagram showing the state of image data IM displayed as a live view and image data IM recorded so that it can be played back later. [Figure 48] FIG. 48 is a schematic diagram showing an aspect in which additional information 43 prompting playback of the recommended third program P3 is superimposed on the playback screen 4. DETAILED DESCRIPTION OF THE INVENTION

[0058] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment shown below is one embodiment of the present invention, and the content of the present invention should not be interpreted as being limited based on the following description.

[0059] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable medium, or may be provided so that it can be downloaded from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0060] In this embodiment, the term "unit" may also include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, various types of information are handled in this embodiment, and this information may be represented by, for example, physical values ​​of signal values ​​representing voltages and currents, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations may be performed on a circuit in the broad sense.

[0061] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0062] [First Embodiment] [System Configuration of Information Processing System] An example of an information processing system 3000 will be described below. FIG. 1 is a diagram showing an example of the system configuration of the information processing system 3000 according to the first embodiment. As shown in FIG. 1, the information processing system 3000 includes a vehicle 3400, a server 3700, a plurality of information display terminals 3800A, 3800B, and 3800C, and a network 3900. The vehicle 3400 is configured to be able to communicate with the server 3700 via the network 3900. More specifically, the vehicle 3400 transmits information held by the vehicle 3400 to the server 3700 via the network 3900. The plurality of information display terminals 3800A, 3800B, and 3800C are configured to be able to communicate with the server 3700 via the network 3900. More specifically, the plurality of information display terminals 3800A, 3800B, and 3800C acquire information stored in the server 3700 via the network 3900. It should be noted that the information display terminals 3800A, 3800B, and 3800C may be any device that functions as a UI (User Interface), and specific examples include a personal computer, a smartphone, and a tablet.

[0063] [Overall Configuration of Vehicle 3400] FIG. 2 is a diagram illustrating the configuration of vehicle 3400 according to the first embodiment. FIG. 2 is a schematic side view of vehicle 3400. Vehicle 3400 includes image capturing device 3100 as a first image capturing device and image capturing device 3200 as a second image capturing device. Vehicle 3400 may be, for example, an internal combustion engine vehicle using an engine as a drive source, a hybrid vehicle using an engine and a traction motor as a drive source, or an electric vehicle using a traction motor as a drive source. Vehicle 3400 may be, for example, a four-wheeled automobile, but may also be any vehicle capable of accommodating image capturing device 3100 and image capturing device 3200. The vehicle may be, for example, a large transport vehicle with four or more wheels, such as a car, bus, or truck; a two-wheeled vehicle such as a motorcycle or bicycle; or another vehicle. The vehicle may also be, for example, a public transportation vehicle, such as a train, monorail, or linear motor car.

[0064] The image capturing device 3100 is a front camera arranged on the front side of the vehicle 3400. For example, the image capturing device 3100 is attached to a predetermined position on the front side of the cabin of the vehicle 3400, and captures an image of the front of the vehicle 3400 through the windshield. The image capturing device 3100 is a drive recorder. More specifically, the image capturing device 3100 has a function of capturing an image, a function of recording image data representing the captured image, and a function of recording image data acquired from the image capturing device 200.

[0065] The image capturing device 3200 is a rear camera disposed at the rear of the vehicle 3400. For example, the image capturing device 3200 is attached to a predetermined position at the rear of the cabin of the vehicle 3400, and captures an image of the rear of the vehicle through the rear window. The image capturing device 3200 has a function of capturing an image and a function of outputting image data representing the captured image to the image capturing device 3100.

[0066] The camera 3100 and the camera 3200 are connected via a cable 3300. The cable 3300 is a wired communication path connecting the camera 3100 and the camera 3200. The cable 3300 includes, for example, a power line for supplying operating power from the camera 3100 to the camera 3200 and a signal line for transmitting various signals between the camera 3100 and the camera 3200. The camera 3200 operates by receiving power from the camera 3100 via the cable 3300. Note that the camera 3100 and the camera 3200 may be connected via a wireless communication path such as Wi-Fi (registered trademark), Bluetooth (registered trademark), or another standard, instead of a wired communication path. The camera 3100 may also be used without being connected to the camera 200 via communication.

[0067] [External Configuration of Image Capturing Device 3100] Figure 3 is a diagram showing an example of the external configuration of the image capturing device 3100. Figure 3(A) is a view of the image capturing device 3100 seen from the front, diagonally upward to the right. Figure 3(B) is a view of the image capturing device 3100 seen from the rear, diagonally upward to the right. The image capturing device 3100 has a housing 3101. The housing 3101 is a rectangular parallelepiped that is longer in the left-right direction than in the up-down direction and has a relatively small thickness. The housing 3101 has a top surface 31011 that faces upward when attached to the vehicle 3400, a first side surface 31012, a second side surface 31013, a third side surface 31014 located opposite the second side surface 31013, and a fourth side surface 31015 located opposite the first side surface 31012.

[0068] The top surface 31011 is provided with a joint rail 3102 and a camera jack 3191. The joint rail 3102 is detachably mountable to a bracket (for example, a bracket 3600, described later) for mounting the image capturing device 3100 to a predetermined mounting position on the vehicle 3400. The mounting position may be, for example, the windshield of the vehicle 3400 (for example, near the top edge of the windshield), or the rearview mirror or ceiling of the vehicle 3400. When the image capturing device 3100 is mounted on the vehicle 3400, the first side surface 31012 faces the front side of the vehicle 3400. In this case, the second side surface 31013 faces the right side when viewed from the rear of the vehicle 3400. The third side surface 31014 faces the left side when viewed from the rear of the vehicle 3400. The fourth side surface 31015 faces the rear side of the vehicle 3400.

[0069] The camera jack 3191 is a terminal to which one end of the cable 3300 is connected. The camera jack 3191 is compatible with, for example, the USB Type C standard, and may be a terminal through which the image capturing device 3100 communicates with the image capturing device 3200 according to the Ethernet standard.

[0070] The first side surface 31012 is provided with a photographing lens 3151, a sound output hole 3103, and a microphone hole 3104. The photographing lens 3151 is a lens for collecting light included in an imaging unit (photographing unit 3015 described later) included in the photographing device 3100. The sound output hole 3103 is provided above the photographing lens 3151 and is a hole that transmits sound output from an audio output unit (audio output unit 3014 described later) included in the photographing device 3100 from the inside to the outside of the housing 3101. The microphone hole 3104 is provided below the photographing lens 3151 and is a hole that transmits external sound from the outside to the inside of the housing 3101. The sound that has transmitted into the inside of the housing 3101 is input to a microphone (microphone 3121 described later) included in the photographing device 3100.

[0071] An event recording button 3122 is provided on the second side surface 31013. The event recording button 3122 is an operating means for instructing the start or end of recording (video recording) of images captured by the imaging unit 3015. When the user operates the event recording button 3122 while event recording is not in progress, the imaging device 3100 starts event recording. Event recording will be described in detail later. The event recording button 3122 is provided on the second side surface 31013 facing the driver's seat so that it can be easily operated by a driver of a right-hand drive vehicle 3400.

[0072] A terminal 3192 and a storage medium insertion port 3181 are provided on the third side surface 31014. The terminal 3192 is a terminal for receiving power supply from an external device. The terminal 3192 is, for example, a DC jack. A connector on one end of a power cord (for example, a cigarette lighter plug cord) is connected to the terminal 3192. The connector on the other end of the power cord is connected to a power supply terminal (for example, a cigarette lighter socket) provided on the vehicle 3400, for example.

[0073] The terminal 3192 may be connected to an OBDII adapter that can be connected to the OBDII ("II" is the Roman numeral for "2") connector of the vehicle 3400. The OBDII connector is also called a fault diagnosis connector, and is connected to the vehicle's ECU (Engine Control Unit) and is a terminal that outputs various vehicle information at predetermined intervals (for example, every 0.5 seconds). By connecting the terminal 3192 to the OBDII connector using the OBDII adapter, the image capturing device 3100 can receive a supply of power for operation and acquire vehicle information.

[0074] The vehicle information is information relating to the state of the vehicle 3400. The vehicle information may be, for example, at least one of the following: the speed of the vehicle 3400 (vehicle speed), engine speed, engine load factor, throttle degree, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air flow rate (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of remaining fuel in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on / off)), brake opening, steering wheel rotation angle, gear position, and door open / close state information.

[0075] Storage medium insertion slot 3181 is an insertion slot for inserting storage medium 3500, which serves as external storage means, into image capturing device 3100. Storage medium 3500 is a storage medium, such as an SD card, on which images captured by image capturing device 3100 or image capturing device 3200 are recorded. SD cards include any shape, such as an SD memory card, a miniSD card, or a microSD card. Storage medium 3500 may further store a viewer (e.g., a dedicated viewer) program for playing back stored images on an information display terminal such as a personal computer.

[0076] The fourth side surface 31015 is provided with an operation unit 3123, a display surface 3131, and a light emitting unit 3021. The operation unit 3123 includes a first button 31231, a second button 31232, and a third button 31233. and a fourth button 31234. The first button 31231, the second button 31232, the third button 31233, and the fourth button 31234 are arranged vertically along one side of the right side of the display surface 3131. The functions that can be assigned to each of these buttons include, for example, the following functions.

[0077] When the first button 31231 is pressed and held, it functions as a button for switching images, and when pressed and held, it functions as a button for instructing formatting of the storage medium 3500. The image displayed on the display surface 3131 is, for example, one or both of the image currently being captured by the image capturing device 3100 and the image currently being captured by the image capturing device 3200. Formatting the storage medium 3500 is to initialize the storage medium 3500, and can be understood as, for example, at least one of erasing data such as images stored in the storage medium 3500, writing setting information indicating the contents of operation settings to the storage medium 3500 so that the image capturing device 3100 can use the storage medium 3500 (for example, so that images can be recorded and read), and putting the storage medium 3500 into a specific file state.

[0078] The second button 31232 is a button for displaying a selection screen for selecting an image to be played back by the image capturing device 3100. The third button 31233 is a button for displaying a menu related to settings of the image capturing device 3100 and the image capturing device 3200. The fourth button 31234 is a button for instructing the start and stop of image recording. For example, if the fourth button 31234 is pressed briefly during recording using the continuous recording function described below, the recording is paused. If the fourth button 31234 is pressed briefly during this pause, the recording of images using the continuous recording function is resumed. If the fourth button 31234 is pressed and held down, the frame rate at which images are recorded can be changed.

[0079] The display surface 3131 is an area where an image is displayed by a display unit (display unit 3013, described later) of the imaging device 3100. The display surface 3131 is, for example, a rectangular or square area. A touch sensor 3124 is provided over the display surface 3131 to detect a touch operation by the user.

[0080] The light emitting portion 3021 is provided above the first button 31231 and emits light in a predetermined color.

[0081] The image capturing device 3200 may also have a function as a drive recorder, and may have, for example, the same configuration as the image capturing device 3100. Furthermore, as an image capturing device connected to the image capturing device 3100 via communication, one or more image capturing devices capturing images in other directions may be used instead of or in addition to the image capturing device 3200. The other directions include the right rear, left rear, and width direction (side) of the vehicle 3400.

[0082] [Configuration of bracket 3600] Figure 4 is a diagram showing the configuration of bracket 3600. Figure 4(A) is a diagram of bracket 3600 viewed from the front, diagonally upward right direction. Figure 4(B) is a diagram of bracket 3600 viewed from the side. Figure 4(C) is a diagram of bracket 3600 viewed from the front, diagonally downward right direction. Bracket 3600 is an example of a mounting member for mounting image capturing device 3100 to vehicle 3400.

[0083] Bracket 3600 is an attachment member that uses a ball joint mechanism. In bracket 3600, flat base portion 3610 has attachment surface 3611 that is attached to the windshield of vehicle 3400. Base portion 3610 is inclined at a predetermined angle relative to the support of ball stud 3620. An adhesive member such as double-sided tape is attached to attachment surface 3611, and bracket 3600 is attached to the windshield or the like of vehicle 3400 via the adhesive member.

[0084] The ball stud 3620 is a portion of the base portion 3610 that rises from the surface opposite the mounting surface 3611. The ball portion 3621 of the ball stud 3620 is attached to the socket portion 3630. The nut 3640 is detachably attached to the periphery of the socket portion 3630. A thread groove is formed on the outer periphery of the socket portion 3630. The nut 3640, which fits into the thread groove, is attached to the outer periphery of the socket portion 3630. With the ball portion 3621 of the ball stud 3620 attached to the socket portion 3630, before the nut 3640 is tightened, the socket portion 3630 can rotate in a desired direction along the circumferential surface of the ball portion 3621, thereby changing the orientation and position of the base portion 3650. When the nut 3640 is tightened, the orientation and position of the base portion 3650 are fixed.

[0085] The base portion 3650 is a part that is formed as a pair with the socket portion 3630 and is used for mounting on the imaging device 3100. The base portion 3650 has a pair of guide rails 3651 on its underside. The pair of guide rails 3651 are configured to be slidable along the joint rails 3102 of the imaging device 3100. A claw-shaped tip portion 3652 is provided at the tip of the base portion 3650. The tip portion 3652 is hooked onto the joint rails 3102 near the end of the front side of the imaging device 3100, thereby mounting the bracket 3600 to the imaging device 3100.

[0086] [Electrical Configuration of the Image Capturing Apparatus 3100] FIG. 5 is a block diagram showing the electrical configuration of the image capturing apparatus 3100. The control unit 3011 controls each unit of the image capturing apparatus 3100. The control unit 3011 is, for example, a computer including a processor 3111 and a memory 3112. The processor 3111 has, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA). The memory 3112 is, for example, a main storage device having a random access memory (RAM) and a read-only memory (ROM). The processor 3111 temporarily stores a program read from the ROM of the memory 3112 in the RAM. The RAM of the memory 3112 provides a working area for the processor 3111. The processor 3111 performs various controls by performing arithmetic processing while temporarily storing data generated during program execution in RAM, and the control unit 3011 further includes a clock unit 3113 that measures time. The clock unit 3113 is, for example, a real-time clock. The clock unit 3113 may be mounted on the motherboard of the processor 3111, or may be external to the processor 3111.

[0087] The input unit 3012 accepts information input from the user. The input unit 3012 has, for example, the above-mentioned microphone 3121, event record button 3122, operation unit 3123, and touch sensor 3124. The microphone 3121 converts sound incident via the microphone hole 3104 or the like into an electrical signal. The microphone 3121 is, for example, a condenser microphone. The touch sensor 3124 detects a position on the display surface 3131 touched by the user. The touch sensor 3124 is, for example, of a capacitance type.

[0088] The display unit 3013 displays an image on a display surface 3131. The display unit 3013 is, for example, a liquid crystal display (LCD).

[0089] The audio output unit 3014 outputs audio. Examples of the audio include notification sounds, background music, audio messages, etc. The audio output unit 3014 includes, for example, an audio processing circuit and a speaker.

[0090] The photographing unit 3015 photographs and generates image data obtained by photographing. The photographing unit 3015 has, for example, a photographing lens 3151 and a photographing element that photographs light collected by the photographing lens 3151. The photographing element is, for example, a CMOS (Complementary MOS) or a CCD (Charge Coupled Device). The photographing unit 3015 generates a color (multi-color) image consisting of, for example, red (R), green (G), and blue (B) color components.

[0091] The communication unit 3016 communicates with an external device. The communication unit 3016 has a communication circuit for wirelessly communicating with the external device, for example, by Wi-Fi (registered trademark), Bluetooth (registered trademark), or other wireless LAN (Local Area Network) communication or short-range wireless communication. The communication unit 3016 may have a communication circuit for performing communication in compliance with standards of mobile communication systems such as LTE (Long Term Evolution), 4G, and 5G.

[0092] The sensor unit 3017 has various sensors. The sensor unit 3017 has, for example, at least one of an acceleration sensor, a gyro sensor, a barometric pressure sensor, and an illuminance sensor. The acceleration sensor is, for example, a three-axis acceleration sensor that measures the vehicle's acceleration in the front-to-back, left-to-right, and up-to-down directions. The gyro sensor is a sensor that measures the tilt of the image capture device 3100. The acceleration sensor and gyro sensor may be used to estimate the position of the vehicle 3400 using autonomous navigation, for example, when signals from GNSS satellites cannot be received. The barometric pressure sensor measures barometric pressure. The barometric pressure sensor is used, for example, to detect elevation differences and determine whether the vehicle is on a highway or an ordinary road. The illuminance sensor is a sensor that detects illuminance, which indicates the brightness inside the vehicle cabin, which is the area around the image capture device 3100. The illuminance sensor is used, for example, to adjust the display brightness of the display unit 3013.

[0093] The reader / writer 3018 functions as a medium holding unit that holds the storage medium 3500 inserted into the inside of the imaging device 3100 through the storage medium insertion slot 3181. The reader / writer 3018 writes data to the storage medium 3500 and reads data from the storage medium 3500. The reader / writer 3018 may hold only one storage medium 3500, but may also be configured to be able to hold two or more storage media 3500 simultaneously.

[0094] The terminal unit 3019 has a terminal for electrically connecting to an external device. The terminal unit 3019 has the above-mentioned camera jack 3191 and terminal 3192. An external battery may be used as a device connected to the terminal unit 3019 so that the image capturing device 3100 and the image capturing device 3200 can operate even without power supply from the vehicle 3400. The device connected to the terminal unit 3019 may be, for example, a device having a function to support safe driving by a user. Examples of such devices include a device having a function to capture an image of the driver (e.g., face) and detect and report the driver's state, such as distracted driving and drowsy driving, and a device having a function to detect and report obstacles around the vehicle 3400 (e.g., a device used for vehicle detection for forward vehicle collision warning systems (FCWS)). The device connected to the terminal unit 3019 may also be an in-vehicle device such as a radar detector, a laser detector, a car navigation device, or a display device.

[0095] The position information acquisition unit 3020 acquires position information indicating the position of the image capturing device 3100 (more specifically, the current position). The position of the image capturing device 3100 can be considered to be the same as the position of the vehicle 3400 in which the image capturing device 3100 is placed, the position of the image capturing device 3200 placed in the vehicle 3400, and the position of the driver and other people riding in the vehicle 3400. The position information acquisition unit 3020 acquires position information (latitude information and longitude information) of the image capturing device 3100 based on signals from the GPS (Global Positioning System), which is one of the GNSS (Global Navigation Satellite System). The position information acquisition unit 3020 acquires position information (latitude information and longitude information) of the image capturing device 3100 based on signals from the QZSS (Quasi-Zenith Satellite System). It may also be used in conjunction with Michibiki (quasi-zenith satellite system).

[0096] The light emitting section 3021 emits light in a predetermined color and includes, for example, a light emitting diode.

[0097] The power supply control unit 3022 controls the supply of power to each unit of the image capturing device 3100 and the image capturing device 3200. The power supply control unit 3022 has, for example, a power switch and a power supply control circuit. The power supply control unit 3022 supplies power supplied from the vehicle 3400 via the terminal unit 3019 to each unit of the image capturing device 3100 and the image capturing device 3200. The power supply control unit 3022 may further have a secondary battery, a button battery, or an electric double layer capacitor (also called a supercapacitor) as a power storage means.

[0098] The image capturing device 3100 may further include an auxiliary storage device such as a flash memory (e.g., eMMC, SSD) as an internal storage means. As the auxiliary storage device, various storage media such as optical storage media, magnetic storage media, and semiconductor storage media can be used.

[0099] [Image Recording Function of the Camera Device 3100] The camera device 3100 has one or more of the following image recording functions. The image recording function is a function for recording images captured by the camera device 3100 as image data in a predetermined file format. The image data may be image data in a moving image format, such as an MPEG (Moving Picture Experts Group) format (e.g., MPEG2, MPEG4), AVI, MOV, WMV, etc. The image data includes information such as the date and time of capture and the location of capture as Exif (Exchangeable image file format) information.

[0100] [Continuous Recording Function] The continuous recording function (also referred to as the continuous video recording function) is a function that continuously (i.e., constantly) records images captured by one or both of the image capturing devices 3100 and 3200 while the image capturing device 3100 is operating. When the control unit 11 is executing the continuous recording function, it stores video data captured from the start to the stop of the drive source of the vehicle 3400. The control unit 3011 detects the start of the drive source, for example, by the power supply (e.g., accessory power supply) of the vehicle 3400 being turned on. The control unit 3011 detects the stop of the drive source, for example, by the power supply being turned off of the vehicle 3400.

[0101] [Event Recording Function] The event recording function records images captured by one or both of the image capturing devices 3100 and 3200 in response to the occurrence of a specific event. An event is an occurrence for which images captured by the image capturing device 3100 or 3200 should be recorded. For example, an event may be an operation such as a sudden steering or braking by the user while the vehicle 3400 is traveling, or a collision of the vehicle 3400 with another object. The control unit 3011 determines that an event has occurred based on, for example, acceleration measured by the acceleration sensor of the sensor unit 3017. More specifically, the control unit 3011 determines that an event has occurred when the measured acceleration exceeds a predetermined threshold or when it shows a predetermined change over time. The conditions for determining the occurrence of an event are not limited to these. The control unit 3011 may also determine that an event has occurred based on vehicle information, for example, when the state of the vehicle 3400, such as the vehicle speed or steering state, satisfies a predetermined condition. The control unit 3011 may analyze an image captured by the image capturing unit 3015 or the image capturing device 3200, and determine that an event has occurred when dangerous driving (for example, tailgating, approaching, or abnormal approach) of the vehicle 3400 or another vehicle is detected. The control unit 3011 also determines that an event has occurred when the event record button 3122 is operated.

[0102] When the control unit 3011 determines that an event has occurred, it records images captured during a predetermined period before and after the event (hereinafter referred to as the "event recording period") in the storage medium 3500. The control unit 3011 may temporarily record images captured by the image capturing unit 3015 and the image capturing device 3200 in memory 3112 (e.g., RAM), and when it determines that an event has occurred, it may read images from the memory 3112 and record them in the storage medium 3500. The control unit 3011 may create a single file containing images for a total of 40 seconds, including 20 seconds before and 20 seconds after the event. The event recording period is an example and may vary depending on the type of event or may be user-configurable. The control unit 3011 may record multiple files of images for each event in the storage medium 3500. The control unit 3011 may further record in the storage medium 3500 values ​​measured by the sensor unit 3017 during the event recording period (e.g., acceleration in each of the three axes) and location information acquired by the location information acquisition unit 3020, in association with the image.

[0103] [Parking Monitoring Function] The parking monitoring function is a function for recording images captured by one or both of the image capturing devices 3100 and 3200 while the vehicle 3400 is parked. The parking monitoring function is a function for monitoring the interior of the parked vehicle 3400 or the exterior surrounding the vehicle 3400. When the power supply of the vehicle 3400 is off, the control unit 3011 receives power from an external battery and records images in the storage medium 3500. The control unit 3011 determines whether the vehicle 3400 is parked based on one or more of the following: the power supply of the vehicle 3400 is off; the power source of the vehicle 3400 is off; the power supply from the external battery has started; the vehicle speed is 0 km / h or a predetermined speed or less; and the location information acquired by the location information acquisition unit 3020 is predetermined location information (for example, location information of the home, workplace, or parking lot).

[0104] The parking monitoring function may include a time lapse mode and a motion detection mode. More specifically, when the user selects the time lapse mode, the control unit 3011 records images at a frame rate lower than that of other image recording functions, such as the continuous recording function and the event recording function. For example, the frame rate of the other image recording functions may be 20 to 30 frames per second, while the frame rate of the time lapse mode may be 1 frame per second. The motion detection mode is a mode in which images are recorded in response to the detection of a moving object. More specifically, when the user selects the motion detection mode, and the control unit 3011 detects a moving object based on changes in the images captured by the image capturing device 3100 and the image capturing device 3200, the control unit 3011 records images captured within a predetermined period before and after the detection in the storage medium 3500. The frame rate may be the same as that of the continuous recording function, the event recording function, and the like.

[0105] Note that imaging devices that capture spherical images such as a full celestial sphere or a hemisphere may be used as imaging devices 3100 and 3200. Also, an imaging device that does not have display unit 3013 may be used as imaging device 3100. Also, the housing of imaging device 3100 does not have to be rectangular parallelepiped-shaped, and may be, for example, cylindrical.

[0106] [Hardware Configuration of Server 3700] Hereinafter, the hardware configuration of the server 3700 will be described with reference to FIG.

[0107] 6 is a diagram showing an example of the hardware configuration of a server 3700 according to the first embodiment. In the first embodiment, the server 3700 functions as an image processing device. The server 3700 includes, as its hardware configuration, a control unit 3710, a storage unit 3720, and a communication unit 3730. The control unit 3710, the storage unit 3720, and the communication unit 3730 are electrically connected to each other via a communication bus.

[0108] The control unit 3710 is configured with a microcomputer including a CPU and a semiconductor memory, and controls the overall operation of the server 3700. In particular, the control unit 3710 realizes various functions related to the server 3700 by reading and executing a predetermined program stored in the storage unit 3720. That is, the control unit 3710 executes processing based on the predetermined program stored in the storage unit 3720, thereby realizing the functional configuration shown in FIG. 8 (described later) and the processing of the activity diagram shown in FIG. 10 (described later). Note that the number of control units 3710 is not limited to one. The server 3700 may be configured to include multiple control units 3710.

[0109] The storage unit 3720 is configured with any one of a hard disk drive (HDD), a random access memory (RAM), a read only memory (ROM), and a solid state drive (SSD), or any combination thereof. The storage unit 3720 stores image information sent from the vehicle 3400, information other than images sent from the vehicle 3400, various data related to various programs including part of the image processing program according to the first embodiment, data used when the control unit 3710 executes processing based on the program, and the like.

[0110] The communication unit 3730 is a so-called network interface that is communicably connected to the vehicle 3400 and each of the information display terminals 3800A, 3800B, and 3800C via a network 3900 such as the Internet or a LAN. In the first embodiment, the communication unit 3730 of the server 3700 that performs image processing is an image processing device that can communicate with the communication unit 3830 of the information display terminal 3800. The main information that the server 3700 receives via the communication unit 3730 includes image information captured by the image capturing device 3100 of the vehicle 3400, image information captured by the image capturing device 3200, information about the status of the vehicle 3400 (such as the speed of the vehicle 3400 (vehicle speed), acceleration, engine RPM, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air flow rate (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of fuel remaining in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on / off)), brake opening, steering wheel rotation angle, gear position, and door open / close status information, location information, etc.), date and time information, atmospheric pressure, illuminance, image processing instructions, image download request instructions, request instructions for information other than images, etc. The main information transmitted by the server 3700 via the communication unit 3730 includes image information before image processing is performed, image information after image processing is performed, information about the image processing method, information about the state of the vehicle 3400 (the speed of the vehicle 3400 (vehicle speed), acceleration, engine RPM, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air flow rate (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of fuel remaining in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on / off)), brake opening, steering wheel rotation angle, gear position, and door open / close status information, location information, etc.), date and time information, atmospheric pressure, illuminance, etc.

[0111] [Hardware Configuration of Information Display Terminal 3800] Next, the hardware configuration of the information display terminal 3800 will be described with reference to FIG.

[0112] 7 is a diagram showing an example of the hardware configuration of an information display terminal 3800 according to embodiment 1. The hardware configuration of the information display terminal 3800 includes a control unit 3810, a storage unit 3820, a communication unit 3830, an input unit 3840, and an output unit 3850. The control unit 3810, the storage unit 3820, the communication unit 3830, the input unit 3840, and the output unit 3850 are electrically connected to each other via a communication bus.

[0113] The control unit 3810 is configured by a microcomputer including a CPU and a semiconductor memory, and controls the overall operation of the information display terminal 3800. In particular, the control unit 3810 controls the memory unit 382. 9 and the activity diagram shown in FIG. 11, which will be described later, are realized by the control unit 3810 executing processing based on the predetermined program stored in the storage unit 3820. Note that the number of control units 3810 is not limited to one. The information display terminal 3800 may be configured to include multiple control units 3810.

[0114] The storage unit 3820 is configured with any one of an HDD, an SSD, a RAM, a ROM, etc., or any combination of these. The storage unit 3820 stores a program for exchanging data related to image processing with the server 3700, data used when the control unit 3810 executes processing based on the program, and the like.

[0115] The communication unit 3830 is a so-called network interface that is communicably connected to the network 3900 in order to transmit and receive various data between the information display terminal 3800 and the server 3700. That is, the main information that the information display terminal 3800 receives via the communication unit 3830 includes image information before image processing is performed, image information after image processing is performed, information related to the image processing method, information related to the state of the vehicle 3400 (such as the speed of the vehicle 3400 (vehicle speed), acceleration, engine RPM, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air flow rate (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of fuel remaining in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on / off)), brake opening, steering wheel rotation angle, gear position, and door open / close status), date and time information, location information, atmospheric pressure, and illuminance. The main information transmitted by the information display terminal 3800 via the communication unit 3830 includes an instruction for image processing, an instruction requesting download of an image, an instruction requesting information other than an image, and the like.

[0116] The input unit 3840 accepts operations to the information display terminal 3800. Examples of the input unit 3840 include a mouse, a keyboard, a touch panel, a microphone, and buttons.

[0117] The output unit 3850 outputs the results of operations performed on the input unit 3840. Examples of the output unit 3850 include a display, a speaker, a light, and a button.

[0118] [Functional Configuration of Server 3700] Hereinafter, the functional configuration of the control unit 3710 of the server 3700 will be described with reference to FIG.

[0119] Fig. 8 is a diagram showing an example of the functional configuration of the server 3700 according to embodiment 1. As shown in Fig. 8, the server 3700 includes an image processing unit 3711, a communication control unit 3712, and a storage control unit 3713 as its functional configuration.

[0120] The image processing unit 3711 performs image processing on an image stored in the storage unit 3720 or transmitted from the information display terminal 3800, based on a request from the information display terminal 3800. This allows the server 3700, which can communicate with the information display terminal 3800, to perform image processing. In the first embodiment, the image processing unit 3711 is described as performing various image processing operations based on a request from the information display terminal 3800. However, the server 3700 may have an image processing unit for each type of image processing. Furthermore, image processing may not be performed by software on the server 3700, but may be performed by dedicated hardware (e.g., a GPU, etc.) on the server 3700. When dedicated hardware performs image processing, one piece of hardware on the server 3700 may be configured to perform multiple image processing operations, or multiple pieces of hardware on the server 3700 corresponding to each of the multiple image processing operations may be configured to perform the image processing.

[0121] The communication control unit 3712 controls the exchange of information with the vehicle 3400 and the information display terminal 3800 via the network 3900 .

[0122] The storage control unit 3713 stores information in the storage unit 3720 and retrieves the stored information.

[0123] [Functional Configuration of Information Display Terminal 3800] Hereinafter, the functional configuration of the control unit 3810 of the information display terminal 3800 will be described with reference to FIG.

[0124] Fig. 9 is a diagram showing an example of the functional configuration of the information display terminal 3800 according to embodiment 1. As shown in Fig. 9, the information display terminal 3800 includes, as its functional configuration, a display control unit 3811, a communication control unit 3812, and a storage control unit 3813.

[0125] The display control unit 3811 controls the information to be output to the output unit 3850 of the information display terminal 3800. More specifically, the display control unit 3811 controls the display of a screen including information related to an image.

[0126] The communication control unit 3812 controls the exchange of information with the server 3700 via the network 3900. More specifically, the communication control unit 3812 acquires, from the server 3700, images captured by the image capturing device 3100, images captured by the image capturing device 3200, etc.

[0127] The storage control unit 3813 stores information in the storage unit 3820 and retrieves the stored information.

[0128] [Outline of Information Processing] The flow of information processing by the information processing system 3000 according to the first embodiment will be described with reference to FIGS.

[0129] [Information Processing by Server 3700] First, we will explain the flow of information processing by the server 3700. Fig. 10 is an activity diagram showing the flow of information processing by the server 3700 according to embodiment 1. Below, we will outline the flow of information processing based on this activity diagram.

[0130] In A4001, the communication control unit 3712 waits to receive information from the vehicle 3400. If the communication control unit 3712 determines that information has been received from the vehicle 3400, the process proceeds to A4002. The following description will be given assuming that the communication control unit 3712 receives, as information from the vehicle 3400, identification information for identifying the vehicle 3400, image information captured by the image capturing device 3100, image information captured by the image capturing device 3200, vehicle information about the vehicle 3400, and other information (for example, information about the time the image was captured, etc.).

[0131] In A4002, the storage control unit 3713 stores the received image information, vehicle information, and other information in the storage unit 3720. More specifically, the storage control unit 3713 stores the image information, vehicle information, and other information in a predetermined area of ​​the storage unit 3720 in association with the identification information that identifies the received vehicle 3400.

[0132] In A4003, the image processing unit 3711 analyzes the image information, vehicle information, and other information. In the first embodiment, the image processing unit 3711 analyzes the acquired image information to determine, for example, that the vehicle 3400 is traveling in fog from 7:22:10 on April 29, 2021 to 7:24:18 on April 29, 2021. The image processing unit 3711 associates the analysis results for each piece of time information with identification information that identifies the vehicle 3400, and stores the results in a predetermined area of ​​the storage unit 3720 together with the image information, vehicle information, and other information. Note that, in this embodiment, the server 3700 is described as analyzing the image information; however, the information display terminal 3800 that receives the image information may analyze the image information.

[0133] In A4004, the communication control unit 3712 transmits image information, vehicle information, analysis results, and other information to the information display terminal 3800 based on an instruction from the information display terminal 3800. More specifically, the communication control unit 3712 transmits to the information display terminal 3800 image information, vehicle information, and other information relating to the vehicle identified by the identification information instructed by the information display terminal 3800, from the specified date and time onwards.

[0134] In A4005, the communications control unit 3712 waits to receive an image processing request from the information display terminal 3800. If the communications control unit 3712 determines that an image processing request has been received from the information display terminal 3800, the communications control unit 3712 proceeds to processing at A4006. For example, the communications control unit 3712 receives a request to perform defogging on the entire range of an image as the image processing request from the information display terminal 3800.

[0135] In A4006, the image processing unit 3711 executes image processing on the image based on the image processing request. For example, the image processing unit 3711 performs defog on the entire image based on the image processing request. Defog is image processing that adjusts the brightness of the image to improve poor visibility due to fog.

[0136] In A4007, the communication control unit 3712 transmits the result of the image processing in the image processing unit 3711 to the information display terminal 3800.

[0137] [Information Processing by Information Display Terminal 3800] Next, a description will be given of the flow of information processing by information display terminal 3800. Fig. 11 is an activity diagram showing the flow of information processing by information display terminal 3800 according to embodiment 1. Below, the flow of information processing will be outlined based on this activity diagram.

[0138] In A4011, the display control unit 3811 displays a launch button for the external video player on the viewer. When the launch button on the viewer is selected, the communication control unit 3812 launches the external video player in the server 3700. At this time, the communication control unit 3812 transmits to the server 3700, as "command line arguments," the file name of the video, video effect settings, parameters for disabling at least a portion of the operation units of the external video player, and the like. At this time, the communication control unit 3812 may also transmit the playback start time, playback end time, whether or not to loop, and the like. Thereafter, the communication control unit 3812 controls the external video player by sending control commands. At this time, the communication control unit 3812 may transmit an instruction to disable at least a portion of the operation units of the external video player.

[0139] At A4012, the communication control unit 3812 waits to receive information from the server 3700. If the communication control unit 3812 determines that it has received information from the server 3700, it advances the process to A4013. The following description will be given assuming that the communication control unit 3812 receives, as information from the server 3700, image information, vehicle information, analysis information analyzed by the image processing unit 3711, and other information. More specifically, in the first embodiment, the description will be given assuming that the communication control unit 3812 receives, from the server 3700, image information, acceleration information, vehicle speed information, position information, analysis information, date and time information, etc.

[0140] In A4013, the display control unit 3811 displays the received image information, vehicle information, analysis information, and other information. More specifically, in the first embodiment, the display control unit 3811 displays image information, acceleration information, vehicle speed information, position information, analysis information, date and time information, etc.

[0141] At A4014, the display control unit 3811 determines whether or not a range specification for image processing has been received. If the display control unit 3811 determines that a range specification for image processing has been received, the process proceeds to A4015. If the display control unit 3811 determines that a range specification for image processing has not been received, the process proceeds to A4016. If a range specification for image processing is not received, the entire range of the image will be described as being subject to image processing. That is, although omitted in FIG. 11 for simplification, if a range specification for image processing is not received, "the entire image" is held as range information. In the first embodiment, the description will be given assuming that "the entire image" is held as range information.

[0142] In A4015, the communication control unit 3812 holds the range information of the image processing selected by the operator of the information display terminal 3800 via a screen or the like.

[0143] In A4016, the display control unit 3811 waits for reception of an instruction to execute image processing selected from a plurality of image processing from the user via the screen. In the first embodiment, the display control unit 3811 receives an instruction to execute defogger from the plurality of image processing from the user via the screen. The following explanation will be given assuming that a line instruction has been received.

[0144] When the user issues an instruction to execute image processing, in A4017 the communication control unit 3812 transmits the instruction to execute image processing to the server 3700. In the first embodiment, the communication control unit 3812 transmits an instruction to execute defogging for the image to the server 3700.

[0145] In A4018, the communication control unit 3812 waits to receive an image resulting from processing by the server 3700. Upon receiving the image, the communication control unit 3812 advances the process to A4019. In the first embodiment, the communication control unit 3812 receives the image after fog correction processed by the server 3700.

[0146] In A4019, the display control unit 3811 displays on the screen the information acquired from the server 3700. In the first embodiment, the display control unit 3811 displays on the screen the image after fog correction acquired from the server 3700.

[0147] In this embodiment, the external video player in the server 3700 is controlled from a PC browser (PC application) that is a viewer on the information display terminal 3800. Examples of external video players include free software such as VLC, cloud image processing services, and image sharpening services. When installing the viewer of this embodiment, the display control unit 3811 asks whether to also install an external video player. If an instruction to install the external video player is received, the storage control unit 3813 installs the external video player. When installing or immediately after launching the external video player, the display control unit 3811 displays information about a button to launch the external video player and notes on using the external video player. The display control unit 3811 may display notes on using the external player, such as "When you operate a button on the external video player, the display state of the viewer may differ" and "If the display state of the viewer differs, it is recommended that you close the external video player and launch the external video player again from the viewer."

[0148] When a cloud-based system is used, the storage control unit 3813 may be configured to have a function of storing an ID or the like for automatically logging in to the cloud image sharpening service in the storage unit 3820. The communication control unit 3812 may also be configured to have a function of automatically sending information, a function of uploading, and a function of determining whether conversion is complete and automatically downloading the image once conversion is complete, or a function of notifying the user that the image is available for download. Furthermore, the display control unit 3811 may have a function of playing the downloaded image in synchronization with the image before conversion.

[0149] Furthermore, the display control unit 3811 controls the display to display that when playback is performed with a non-recommended viewer, playback will be clearer when played with a dedicated viewer. Recording dark video may be corrected and displayed clearly when played with a dedicated viewer, but the video may appear dark when played with a standard player for an OS such as Windows (registered trademark). Therefore, for example, the display control unit 3811 controls the display to display black bands above and below the video when playback is performed with a non-recommended viewer, and display the message "This video will play clearly when played with a dedicated viewer" in these black bands. At this time, the display control unit 3811 may also display a code (e.g., a QR code (registered trademark)) referencing a URL where the viewer can be installed along with the text. Alternatively, the display control unit 3811 may display the text "This video will play clearly when played with a dedicated viewer" at the beginning of the video when playback is performed with a non-recommended viewer, or a code (e.g., a QR code (registered trademark)) referencing a URL where the viewer can be installed.

[0150] [Details of Information Processing] Next, the above-mentioned information processing will be described in detail.

[0151] FIG. 12 is a diagram showing an example of a screen displayed by the display control unit 3811 of the first embodiment before image processing. The display control unit 3811 controls the display of information stored in the server 3700 on the viewer 4100. The viewer 4100 is configured to be able to call various functions and display information. More specifically, the viewer 4100 includes a file button 4101, a display button 4102, a play button 4103, a tool button 4104, and an information button 4105, and these buttons enable the user to call and view various functions. The viewer 4100 also includes a video display area 4110, an acceleration graph display area 4120, a map image display area 4130, a date and time display area 4150, a vehicle status display area 4160, and a speed graph display area 4170, and these areas enable the user to view the image currently being played and information related to the image. Furthermore, the viewer 4100 includes a video display area 4110, a defog button 4140, an HDR button 4141, a sharpen button 4142, an other processing button 4143, a partial processing button 4144, a stream display button 4145, an operation button 4146, and a playlist display area 4180. In response to operations on these buttons, the display control unit 3811 requests the server 3700 to perform image processing, image playback, and the like. That is, the viewer 4100 is configured to be able to accept multiple operation instructions from the user. At this time, each of the multiple operation instructions instructs the display of a different processing result related to the image.

[0152] The file button 4101 is configured to be able to execute processes related to various file operations. More specifically, when the file button 4101 is selected, the display control unit 3811 displays file-related processes such as "view file," "save over," and "save as" in a pull-down format on the screen, and when one of the file-related processes is selected, the display control unit 3811 requests the control unit 3810 etc. to execute the selected process.

[0153] The display button 4102 makes it possible to execute various processes related to the display of the video display area 4110. More specifically, when the display button 4102 is selected, the display control unit 3811 displays display-related processes such as "side-by-side display," "display in a row," and "minimize" in a pull-down menu, and when one of the display-related processes is selected, the selected process is executed.

[0154] The play button 4103 makes it possible to execute various processes related to the playback of video. More specifically, when the play button 4103 is selected, the display control unit 3811 displays in a pull-down format processes related to playback, such as "play next playlist," "play previous playlist," "play from next image processing," and "play from previous image processing," and when one of the display-related processes is selected, the selected process is executed.

[0155] Various functions can be called up using the tool button 4104. More specifically, when the tool button 4104 is selected, the display control unit 3811 displays a pull-down menu of options such as "Macro" and "User Settings," and when one of the processes is selected, the display control unit 3811 requests the control unit 3810 to execute the selected process.

[0156] Various information can be referenced using the information button 4105. More specifically, when the tool button 4104 is selected, the display control unit 3811 displays buttons relating to information such as "version information" in a pull-down format, and by selecting one of the buttons, the selected information is displayed so that the user can reference it.

[0157] The video display area 4110 is an area where images captured by the camera device 3100 and the camera device 3200 and images processed by the image processing unit 3711 are displayed. The display format of the video display area 4110 can be changed arbitrarily in response to a user instruction. More specifically, the video display area 4110 may display a plurality of images that have been processed using different methods side by side based on a user instruction, or may display an image after image processing and an image before image processing side by side. Furthermore, the video display area 4110 may display any information, such as an acceleration graph display area 4120.

[0158] The acceleration graph display area 4120 is an area where the acceleration of the vehicle from which the video was captured is displayed in three dimensions. The acceleration graph display area 4120 has an X axis in the depth direction, a Y axis in the horizontal direction, and a Z axis in the vertical direction. Here, the X axis represents the acceleration in the front and rear of the vehicle, the Y axis represents the acceleration in the lateral direction of the vehicle, and the Z axis represents the acceleration in the vertical direction of the vehicle. Acceleration in the front of the vehicle (positive acceleration) is graphed upward on the X axis, and acceleration in the rear of the vehicle (negative acceleration) is graphed downward on the X axis. Acceleration in the right direction as viewed from the vehicle is graphed upward on the Y axis, and acceleration in the left direction as viewed from the vehicle is graphed downward on the Y axis. Acceleration in the upward direction of the vehicle is graphed leftward on the Z axis, and acceleration in the downward direction of the vehicle is graphed rightward on the Z axis.

[0159] The map image display area 4130 is an area where an image of a map in the vicinity of the shooting location of the video displayed in the video display area 4110 is displayed. Data showing the vehicle's position is transmitted from the information display terminal 3800 to a map server (not shown) present on the network 3900, and data showing a map in the vicinity of the vehicle's position is transmitted from this map server to the information display terminal 3800, whereby an image of the map is displayed in the map image display area 4130. At this time, a current location icon 4131 is displayed on the map image display area 4130.

[0160] The viewer 4100 includes a defog button 4140. When the defog button 4140 is selected, the image processing unit 3711 performs defog on the image displayed in the video display area 4110. More specifically, when the defog button 4140 is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform defog processing on the image. Upon receiving the instruction, the server 3700 performs defog processing on the image using the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the defog-processed image to the information display terminal 3800. The display control unit 3811 controls the display of the video display area 4110, including the defog-processed image. In other words, video correction can be performed by selecting a button in the PC Browser (PC application). Furthermore, at this time, if the defog button 4140 is operated again while defog is being performed, the image processing unit 3711 can control the display to cancel defog.

[0161] The display control unit 3811 can also control the display to adjust the degree of defogging depending on the number of times the defog button 4140 is selected or the selection time. More specifically, the display control unit 3811 controls the display to increase the degree of defogging depending on the number of times the defog button 4140 is pressed, or to increase the degree of defogging depending on the time the defog button 4140 is pressed. In other words, the display control unit 3811 receives an operation instruction for the defog button 4140 selected by the user, receives an instruction to display the processing results related to the image corresponding to the operation instruction, and controls the display to display the video display area 4110 including the processing results related to the image. The defog button 4140 is an example of an object. The operation of selecting the defog button 4140 is also an example of an object operation.

[0162] The viewer 4100 includes an HDR (High Dynamic Range) button 4141. When the HDR button 4141 is selected, the image processing unit 3711 performs HDR processing on the image displayed in the video display area 4110. Here, HDR processing is a process of synthesizing multiple photos with different brightness levels to create an image without blown-out highlights or crushed shadows. More specifically, when the HDR button 4141 is selected, the communication control unit of the information display terminal 3800 The information display terminal 3800 then displays the image data on the display 3800. The image processing unit 3712 then outputs a command to the server 3700 to perform HDR processing on the image. Upon receiving the command, the server 3700 performs HDR processing by synthesizing multiple photos with different brightness levels using the image processing unit 3711. The communication control unit 3712 of the server 3700 then transmits the HDR-processed image to the information display terminal 3800. The display control unit 3811 then controls the display of the video display area 4110 including the HDR-processed image.

[0163] The display control unit 3811 also receives an instruction to display the processing results for the image corresponding to the button selected by the user, and controls to display a screen including the processing results for the image. More specifically, the image processing unit 3711 receives an instruction to perform defogging when the defog button 4140 is selected, and receives an instruction to perform HDR processing when the HDR button 4141 is selected, and controls to display the processing results in the video display area 4110. Here, buttons related to image processing, such as the defog button 4140 and the HDR button 4141, are different for different processing results for the image. The defog button 4140 and the HDR button 4141 are examples of objects. This allows any image processing corresponding to the button to be executed by user selection. Furthermore, when multiple buttons are displayed, the display control unit 3811 can control to prevent these buttons from being displayed simultaneously. More specifically, the display control unit 3811 can control to prevent the defog button 4140 and the HDR button 4141 from being displayed simultaneously.

[0164] Furthermore, at this time, when the defog button 4140 and the HDR button 4141 are selected via the screen, the display control unit 3811 may receive an instruction to display a first processing result for the image, corresponding to the defog button 4140, and an instruction to display a second processing result for the image, corresponding to the HDR button 4141, and perform control to display a screen including the first processing result for the image and the second processing result for the image. More specifically, when the defog button 4140 and the HDR button 4141 are selected via the viewer 4100, the display control unit 3811 receives an instruction to display a processing result of defogging for the image, corresponding to the defog button 4140, and an instruction to display a processing result of HDR processing for the image, corresponding to the HDR button 4141, and performs control to display a screen including the processing result of defogging for the image and the processing result of HDR processing for the image. This makes it possible to simultaneously perform multiple arbitrary image processes. The defog button 4140 is an example of a first object. The HDR button 4141 is an example of a second object.

[0165] The viewer 4100 includes a sharpening button 4142. When the sharpening button 4142 is selected, the image processing unit 3711 performs sharpening processing on the image displayed in the image display area 4110. More specifically, when the sharpening button 4142 is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform sharpening processing. Upon receiving the instruction, the server 3700 performs sharpening processing on the image using the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the image that has been subjected to the sharpening processing to the information display terminal 3800. The display control unit 3811 controls the display of the image display area 4110 including the image that has been subjected to the sharpening processing.

[0166] The viewer 4100 includes an other processing button 4143. When the other processing button 4143 is selected, the display control unit 3811 controls the viewer 4100 to display various image processing options. Here, examples of the image processing options include an image processing adjustment area 6290, which will be described later. When this image processing option is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform image processing. Upon receiving the instruction, the server 3700 processes the image using the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the processed image to the information display terminal 3800. The display control unit 3811 controls the display of the video display area 4110, which includes the processed image.

[0167] The viewer 4100 includes a partial processing button 4144. When the partial processing button 4144 is selected, the display control unit 3811 controls the display so that a portion of the video display area 4110 can be selected. When this image processing option is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform image processing on a portion of the image. Upon receiving the instruction, the server 3700 performs image processing on the portion of the image using the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the image that has undergone image processing to the information display terminal 3800. The display control unit 3811 controls the video display area 4110 to display a portion of the image that has undergone image processing. Here, the display control unit 3811 may use the image before image processing and the image after image processing to perform control so that the portion of the image is displayed in the video display area 4110 including the image that has undergone image processing. Note that the method for performing partial processing may be different. For example, the display control unit 3811 accepts the specification of a correction target range for the video display area 4110. The display control unit 3811 may further provide a range registration checkbox. The display control unit 3811 may set the specified range as the correction target range while the range registration checkbox is checked and the correction button is pressed. In this case, the display control unit 3811 may provide a button to temporarily disable the correction while the range remains in a corrected state.

[0168] The viewer 4100 includes a stream display button 4145. When the stream display button 4145 is selected, the display control unit 3811 performs control so that a plurality of images are displayed side by side in the video display area 4110.

[0169] Each button displayed on the viewer 4100 may be in other forms. For example, each button displayed on the viewer 4100 may be a radio button, a check box, a toggle button, or the like, instead of a push button. Also, for example, each button has been described as a soft button displayed on the viewer 4100, but it may be provided in a location other than the viewer 4100, or may be a physical button such as a physical switch.

[0170] The date and time display area 4150 is an area that displays the recording date and time of the video displayed in the video display area 4110. The date and time display area 4150 may also be configured to display the current date and time.

[0171] The vehicle status display area 4160 is an area where information such as the vehicle speed at the time of recording the video is displayed. The vehicle status display area 4160 displays the status of the vehicle to which the image capturing device 3100, 3200 that recorded the video displayed in the video display area 4110 is attached.

[0172] The speed graph display area 4170 is an area where the speed of the vehicle is displayed in a graph form. In addition to the speed, the speed graph display area 4170 may also display acceleration in a graph form.

[0173] The playlist display area 4180 is an area where information about stored video data is displayed. When information about video data in the playlist display area 4180 is selected, the display control unit 3811 accesses the video data in the server 3700 and displays it in the video display area 4110. In this case, the video data may be stored in the server 3700, the information display terminal 3800, or an external storage device.

[0174] FIG. 13 is a diagram showing an example of a screen displayed by the display control unit 3811 according to the first embodiment after image processing. The display control unit 3811 controls the display of information processed in the server 3700 on the viewer 4200. The video display area 4210 of the viewer 4200 according to the first embodiment is an image that has been subjected to defogging by selecting the defog button 4140 for the video display area 4110 shown in FIG. 12. That is, the video display area 4210 includes the image after image processing. At this time, the defog button 4240 is displayed in a different color from the other buttons to indicate that it has been selected. Furthermore, at this time, the storage control unit 3813 stores the image captured by the imaging device 3100 in a storage area. This allows the information display terminal 3800 to display the image after image processing in the video display area 4210 while storing the image before image processing in the storage unit 3820.

[0175] In the first embodiment, an example has been described in which the storage control unit 3813 controls to store the image before image processing and the display control unit 3811 controls to display the image after image processing. However, the image before image processing may be reproduced and displayed using the image after image processing. More specifically, the storage control unit 3713 controls to store the image after image processing and the image processing performed on the image in the storage unit 3720. Next, the image processing unit 3711 controls to reproduce the image before image processing based on information about the image processing performed on the image after image processing. Thereafter, the display control unit 3811 controls to display this reproduced image before image processing. At this time, the display control unit 3811 may also display which data is the image displayed in the video display area 4210. More specifically, the display control unit 3811 may display whether the displayed data is the original data, the data after image processing, or the data reproduced from the data after image processing to the data before image processing.

[0176] [Modification 1] Modification 1 of the first embodiment will be described. FIG. 14 is a diagram showing an example of a screen after image processing, displayed by the display control unit 3811 of Modification 1 of the first embodiment. The video display area 4310 of the viewer 4300 in Modification 1 includes a defog area 4311 in which a portion of the video display area 4310 has been subjected to defogging, and a sharpened area 4313 in which a region different from the defog area 4311 has been subjected to sharpening processing. In this way, the image processing associated with the defog button 4340 of Modification 1 and the image processing associated with the sharpening button 4342 are configured to target different regions of the image. The defog button 4340 is an example of a first object. The sharpening button 4342 is an example of a second object. The information display terminal 3800 of Modification 1 can execute arbitrary image processing on different regions. For example, the user selects a predetermined region (the region corresponding to the defog area 4311) in the video display area 4110 and selects the defog button 4340. Furthermore, the user selects a predetermined area (an area corresponding to the sharpening area 4313) in the video display area 4110 and selects the sharpening button 4342. By making such a selection, a screen such as that shown in FIG. 14 can be displayed. Furthermore, the display control unit 3811 can display what processing has been performed on which area by displaying processing details in each processing area. That is, the display control unit 3811 can display a processing details display section 4312 in the defogger area 4311 so that it is clear that defogger has been performed on the defogger area 4311. Furthermore, the display control unit 3811 can display a processing details display section 4314 in the sharpening area 4313 so that it is clear that sharpening processing has been performed. Furthermore, the display control unit 3811 can display a color change for the defogger button 4340, the sharpening button 4342, and the partial processing button 4344 to indicate that these processes are being performed. In the example of Modification 1, defogging and sharpening are used as examples of image processing for different regions, but the present invention is not limited to this and other image processing may be used. Also, the same image processing (for example, defogging) may be performed to different degrees on different regions.

[0177] [Modification 2] Modification 2 of the first embodiment will be described. FIG. 15 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 2 of the first embodiment. The objects displayed in the viewer 4400 in Modification 2 are a sunny correction button 4440, a cloudy correction button 4441, a rain correction button 4442, and a fog correction button 4443. The sunny correction button 4440, the cloudy correction button 4441, the rain correction button 4442, and the fog correction button 4443 are examples of objects related to image processing related to weather. In Modification 2, the viewer uses weather information to apply different image corrections to sunny, rainy, and foggy images. The display control unit 3811 receives instructions for image quality corrections, such as gamma correction, color correction (color emphasis, color balance), and edge emphasis, corresponding to the weather, by selecting each button. For example, when the rain correction button 4442 is selected, the display control unit 3811 requests the image processing unit 3711 of the server 3700 to perform image processing such as gamma correction, color correction (color emphasis, color balance), and edge emphasis on the image so as to erase the rain. Furthermore, when the fog correction button 4443 is selected, the display control unit 3811 requests the image processing unit 3711 of the server 3700 to perform image processing such as gamma correction, color correction (color emphasis, color balance), and edge emphasis on the image so as to make the fog transparent and make the other side of the fog visible. Furthermore, the display control unit 3811 may display, as these button names, button names for performing image correction appropriate to the image, such as an operation button for erasing the snowstorm, a button for making the fog transparent and making the other side of the fog visible, and a button for erasing the rain. Furthermore, the display control unit 3811 may newly or alternatively provide objects related to weather conditions such as "clear," "cloudy," "rainy," and "fog" in the viewer 4400, such as "clear," "slightly cloudy," "smoke," "dust storm," "blizzard," "drizzle," "sleet," "snow," "graupel," "sleet," and "thunder." Instead of providing objects for each weather condition, an all-purpose automatic correction may be performed, which automatically detects and corrects these weather conditions. When the all-purpose automatic correction is enabled, the display control unit 3811 displays a button on the screen to enable or disable image correction. If the display control unit 3811 determines that there is a limit to the all-purpose automatic correction, it creates some other state.For example, the display control unit 3811 displays another correction means. This makes it possible to perform appropriate image processing in accordance with the weather.

[0178] [Modification 3] Modification 3 of the first embodiment will be described. FIG. 16 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 3 of the first embodiment before image processing. An image captured by the image capturing device 3100 of the vehicle 3400 traveling on a foggy road is displayed in the video display area 4510 of the viewer 4500 in Modification 3. At this time, the display control unit 3811 controls the display so that the fog correction button 4540 is highlighted as an appropriate correction method. That is, the display control unit 3811 changes the button displayed on the screen based on the state of the image. This makes it possible to perform appropriate image processing based on the information in the image. As one mode of button change, the display control unit 3811 can display a new button. As another mode of button change, the outline of the button can be emphasized. Furthermore, as yet another mode of button change, the display control unit 3811 can display the button by illuminating it. For example, when the vehicle 3400 is traveling in a city or a forest, the brightness changes depending on the location and / or direction of the vehicle, and the display control unit 3811 may accordingly perform control such as displaying a new brightness correction button, emphasizing the outline of the brightness correction button, or displaying the brightness correction button with a glow.

[0179] [Modification 4] Modification 4 of the first embodiment will be described. FIG. 17 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 4 of the first embodiment. The video display area 4610 of the viewer 4600 in Modification 4 displays an image captured by the camera 3100 of the vehicle 3400 just before it enters a tunnel. At this time, the current location icon 4631 in the map image display area 4630 indicates that the vehicle 3400 is just about to enter a tunnel. The communication control unit 3812 of the information display terminal 3800 acquires road network information related to the position information of the vehicle 3400 from a map server (not shown) via the network 3900. Then, the display control unit 3811 controls to display a tunnel correction button 4640 suitable for image processing inside a tunnel. That is, the display control unit 3811 controls to display a button on the screen that suggests image processing for the image based on the road network information. Here, by selecting the tunnel correction button 4640, the image processing unit 3711 automatically performs an appropriate type of correction for the tunnel position. Specifically, by selecting the tunnel correction button 4640, the image processing unit 3711 performs image processing to adjust the brightness to correct the darkness of the tunnel, to reduce the influence of lighting installed in the tunnel, and to reduce the influence of lighting devices of the vehicle ahead. Thus, it is possible to perform appropriate image processing based on the road network information. Here, the road network information is information about road conditions, and more specifically, it is information that includes, in addition to tunnel position information, speed limit information, road width information, 12-hour weekday traffic volume information, travel speed information (for a certain section, the distance of the section divided by the time required to travel, the required time including the speed at which stops due to traffic lights, congestion, etc. are made), number of lanes information, etc.

[0180] [Modification 5] Modification 5 of the first embodiment will be described. FIG. 18 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 5 of the first embodiment. The sensor unit 3017 of Modification 5 measures acceleration information of the vehicle 3400. That is, the vehicle information of Modification 5 includes acceleration information measured by the sensor unit 3017. The image display area 4710 of the viewer 4700 in Modification 5 displays an image captured by the image capture device 3100 of the vehicle 3400 rapidly accelerating. At this time, the acceleration graph display area 4720 is a graph showing a large upward slope on the X-axis, and the speed graph display area 4770 is a graph showing a rapid increase in speed. In this case, the viewer 4700 may be provided with an acceleration display unit 4721 such as "1 G or more detected." The display control unit 3811 controls to display an acceleration correction button 4740 suitable for image processing when the vehicle 3400 is accelerating. That is, the display control unit 3811 controls the display of an acceleration correction button on the screen based on information from the sensor unit 3017 that detects the acceleration of the vehicle 3400 equipped with the image capture device 3100. Here, when the acceleration correction button 4740 is selected, image processing is performed to correct screen shaking and / or poor visibility caused by sudden acceleration. This enables appropriate image processing to be performed based on information from the acceleration sensor of the moving object. The image capture device 3100 is an example of an image capture unit. The vehicle 3400 is an example of a moving object. The sensor unit 3017 is an example of an acceleration sensor. The acceleration correction button is an example of an object that suggests image processing for an image. The display control unit 3811 also accepts one or more acceleration correction settings by the user. Specifically, the display control unit 3811 accepts one or more acceleration correction settings by the user inputting a numerical value for acceleration correction or dragging to the acceleration graph display area 4720 or the speed graph display area 4770, for example. At this time, the display control unit 3811 displays bands of different colors in the range for each type of correction in the acceleration graph display area 4720 or the speed graph display area 4770, or displays the dragged range, or the like.

[0181] [Variation 6] Variation 6 of the first embodiment will be described. FIG. 19 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Variation 6 of the first embodiment. In the image display area 4810 of the viewer 4800 of Variation 6, an image captured by the image capturing device 3100 of the vehicle 3400 is displayed. At this time, a current location icon 4831 in a map image display area 4830 in which the vehicle 3400 is traveling indicates the location on the map where the vehicle 3400 is traveling. The display control unit 3811 can set a processing area 4832 in the map image display area 4830 based on a user operation or the like, and when the current location icon 4831 enters this area, the display control unit 3811 controls to display a button or the like on the screen that suggests image processing for the image based on the position information of the vehicle 3400 having the image capturing device 3100. This makes it possible to perform appropriate image processing based on the position information of the vehicle 3400. As described above, the processing area 4832 may be set based on a user operation or the like, or may be set automatically. For example, if it is set that an image processing button for brightening an image is to be displayed when the vehicle enters a tunnel, the display control unit 3811 performs control so that an image processing button for brightening an image is displayed on the screen when it is determined that the vehicle 3400 has entered a tunnel based on the position information of the vehicle 3400.

[0182] [Variation 7] Variation 7 of the first embodiment will be described. FIG. 20 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Variation 7 of the first embodiment. In Variation 7, an image display area 4910 of the viewer 4900 displays an image captured by the camera 3100 of the vehicle 3400 of a road where cloudy weather is expected to occur followed by rain. At this time, the viewer 4900 can display weather information 4911 for that road. In this embodiment, the viewer 4900 displays weather information 4911 relating to cloudy weather followed by rain. The communication control unit 3812 acquires weather information for the route taken by the vehicle 3400 from a weather information providing server (not shown) based on the location information of the vehicle 3400 and date and time information included in the image captured by the camera 3100. The display control unit 3811 then controls the display to display a button suggesting image processing based on the location information of the vehicle 3400 and the weather information corresponding to the location information. That is, the display control unit 3811 controls the display of an object suggesting image processing for an image on the screen based on the position information of the vehicle 3400 having the image capturing device 3100 and weather information for the date and time the image was captured. In the seventh modification, image correction can be applied according to the time period recorded by the viewer. For example, the image processing unit 3711 can apply different image corrections for morning, night, and evening based on instructions from the communication control unit 3812 and weather information 4911. In the example of the seventh modification, the display control unit 3811 controls the display of a rain correction button 4941 on the viewer 4900 based on the position information of the vehicle 3400 and weather information. Here, when the rain correction button 4941 is selected, image processing is executed to correct darkness due to rain clouds, poor visibility due to rain, and the like. This makes it possible to perform appropriate image processing based on the position information of the vehicle 3400 and weather information for the date and time at that location. Here, the weather information includes not only cloudy and rainy weather but also sunny, clear skies, slightly cloudy weather, smoke and mist, dust storm, blizzard, drizzle, sleet, fog, snow, hail, sleet, and thunder.

[0183] [Modification 8] Modification 8 of the first embodiment will be described. Fig. 21 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 8 of the first embodiment. In the image display area 5010 of the viewer 5000 in Modification 8, an image captured by the camera device 3100 of the vehicle 3400 approaching an intersection is displayed. At this time, the current location icon 5031 in the map image display area 5030 indicates that the vehicle 3400 is approaching an intersection. The speed graph display area 5070 is The image shows the vehicle 3400 decelerating as it approaches an intersection. The display control unit 3811 controls to display a brake correction button 5040 in response to the deceleration of the vehicle 3400. That is, the display control unit 3811 controls to display an object on the screen suggesting image processing for an image based on operation information of the vehicle 3400 having the image capturing device 3100. Here, by selecting the brake correction button 5040, blurring of the field of view due to deceleration can be reduced, and the influence of the brake lights on the image display area 5010 can be reduced. This makes it possible to perform appropriate image processing based on the operation information of the vehicle 3400. Here, the operation information of the vehicle 3400 is information related to operation among the vehicle information. In addition to brakes, the vehicle information also includes information obtained from the operation of turn signals, reverse, and other information obtained from OBD (On-board diagnostics).

[0184] [Modification 9] Modification 9 of the first embodiment will be described. FIG. 22 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 9 of the first embodiment before image processing. In the image display area 5110 of the viewer 5100 in Modification 9, an image captured by the imaging device 3100 of the vehicle 3400 is displayed. At this time, information related to the date and time is displayed in the date and time display area 5150. The example of FIG. 22 indicates that it is nighttime. The display control unit 3811 controls to display a night correction button 5140 based on date and time information, which is attribute information of the image. That is, the display control unit 3811 controls to display a button on the screen suggesting image processing for the image based on the date and time when the image was captured. Here, selecting the night correction button 5140 brightens the night so that what is around can be displayed. This makes it possible to perform appropriate image processing based on the date and time when the image was captured. Here, the night correction button is an example of a button that suggests image processing for an image based on the date and time, and may be, for example, a button for correcting brightness due to a sunset to clarify the field of view. Note that the system may be configured to execute automatic correction that combines the corrections of Modifications 3 to 9 described above. The combination of corrections of Modifications 3 to 9 may be set using an OR condition or an AND condition.

[0185] [Modification 10] Modification 10 of the first embodiment will be described. FIG. 23 is a diagram showing an example of a screen after image processing, displayed by the display control unit 3811 of Modification 10 of the first embodiment. An image captured by the camera device 3100 of a vehicle 3400 traveling on a foggy road is displayed in the video display area 5210 of the viewer 5200 in Modification 10. The result of fog correction is displayed in a defog area 5211, which is part of the video display area 5210. That is, the result of image processing performed on a portion of the image captured by the camera device 3100 is displayed in the defog area 5221. When an instruction to perform image processing on the defog area 5221 is issued, the image processing unit 3711 performs image processing on the portion of the image captured by the camera device 3100. The defog area 5211 is an area set in accordance with a user operation or the like before performing image processing. That is, the portion of the image is a set area of ​​the entire area of ​​the image captured by the camera device 3100. More specifically, the display control unit 3811 sets the range of image processing by selecting the partial processing button 5245 and specifying a range on the screen, for example. Next, when the defog button 5240 is selected, the display control unit 3811 determines that the set range is the defog area 5211 where defogging will be performed. The communication control unit 3812 of the information display terminal 3800 then requests the server 3700 via the network 3900 to perform defogging on the defog area 5211. Upon receiving the request, the image processing unit 3711 performs defogging on the defog area 5211. The communication control unit 3812 acquires the results of defogging on the defog area 5211. The display control unit 3811 displays the results of image processing on the defog area 5211. This makes it possible to perform image processing on a portion of the captured image. This also makes it possible to set the area where image processing will be performed.

[0186] [Modification 11] Modification 11 of the first embodiment will be described. FIG. 24 is a diagram showing an example of a screen after image processing, displayed by the display control unit 3811 of Modification 11 of the first embodiment. In the video display area 5310 of the viewer 5300 in Modification 11, for example, video captured by the camera device 3100 of the vehicle 3400 in the middle of the night is displayed. Furthermore, a night correction area 5311, which is a part of the video display area 5310, displays the results of night correction. Furthermore, the date and time display area 5350 indicates that it is night. When the date and time display area 5350 indicates night, the results of night correction are displayed in the night correction area 5311. That is, the image processing performed on the image is image processing performed on a portion of the image captured by the camera device 3100, and the portion of the image refers to the image captured during a set time period among the images captured by the camera device 3100. The display control unit 3811 sets the range of image processing based on the operation on the partial processing button 5345 and the date and time information of the image. Next, based on the operation on the night correction button 5340, the display control unit 3811 determines that the range is a night correction area 5311 in which correction is to be performed at night. Furthermore, the communication control unit 3812 instructs the server 3700 via the network 3900 to perform image processing on the night correction area 5311. Next, the image processing unit 3711 performs image processing on the night correction area 5311. Furthermore, the communication control unit 3812 acquires the results of the image processing on the night correction area 5311. Finally, the display control unit 3811 displays the results of the image processing on the night correction area 5311. This makes it possible to perform appropriate image processing on a portion of an image captured during a set time period. In the eleventh modification, the partial treatment button 5345 and the night correction button 5340 have been described as examples. However, the display control unit 3811 may be configured to provide a means for inputting a time period in the viewer 5300 and instruct the display control unit 3811 to perform the set image processing on images captured during a time period input via this means.

[0187] [Modification 12] Modification 12 of the first embodiment will be described. FIG. 25 is a diagram showing an example of a screen after image processing, displayed by the display control unit 3811 of Modification 12 of the first embodiment. The position information acquisition unit 3020 of Modification 12 acquires the position information of the vehicle 3400. That is, the other information of Modification 12 includes the position information acquired by the position information acquisition unit 3020. In the image display area 5410 of the viewer 5400 of Modification 12, an image captured by the image capture device 3100 of the vehicle 3400 is displayed. In addition, a defog area 5411, which is part of the image display area 5410, displays the results of defog. Furthermore, a current location icon 5431 in the map image display area 5430 indicates the location on the map where the vehicle 3400 is traveling. In addition, a processing area 5432 is displayed in the map image display area 5430. If the current location icon 5431 is located within this processing area 5432, the defogged area 5411 displays the results of defogging. That is, image processing for an image is image processing for a portion of an image captured by the image capturing device 3100. The portion of the image is an image captured within a range set by the location information of the vehicle 3400 carrying the image capturing device 3100. First, the display control unit 3811 sets the range of image processing based on an operation on the partial processing button 5445 and the location information. Next, based on an operation on the defog button 5440, the display control unit 3811 determines that this range is the defogged area 5411 where defogging will be performed for a specific position. Furthermore, the communication control unit 3812 instructs the server 3700 via the network 3900 to perform image processing on the defogged area 5411. Next, the image processing unit 3711 executes image processing on the defogged area 5411. Furthermore, the communication control unit 3812 acquires the results of image processing on the defogged area 5411. Finally, the display control unit 3811 displays the results of image processing on the defogged area 5411. This makes it possible to perform appropriate image processing on a partial area of ​​an image captured by the vehicle 3400 moving within the set position.In the twelfth modification, the partial treatment button 5445 and the defog button 5440 have been described as examples. However, the display control unit 3811 may provide a means for inputting latitude and longitude information to the viewer 5300, and instruct the display control unit 3811 to execute the set image processing on an image captured at the latitude and longitude input via this means. Note that the input of position information is not limited to latitude and longitude information, and the display control unit 3811 may instruct the viewer 5300 to execute the set image processing on an image captured at a position within a range selected by the user on a map displayed on the viewer 5300. Furthermore, the display control unit 3811 may display historical route information of the user's car on the viewer 5300, and instruct the viewer 5300 to execute the set image processing on an image captured at a position within a range selected by the user on the displayed historical route.

[0188] [Modification 13] Modification 13 of the first embodiment will be described. Fig. 26 is a diagram showing an example of a screen after image processing, displayed by the display control unit 3811 of Modification 13 of the first embodiment. In the image display area 5510 of the viewer 5500 of Modification 13, an image captured by the camera 3100 of the vehicle 3400 of a road where it is expected to be cloudy with rain later is displayed. In addition, a fog correction area 5511 that is part of the image display area 5510 displays the results of fog correction. Furthermore, the viewer 5500 can be configured to display weather information 5512 for the road. Here, the viewer 5500 displays weather information 5512 related to cloudy with rain later. If the weather at the date and time of the image being played back is cloudy, the results of fog correction are displayed in the fog correction area 5511. That is, the image processing for the image is image processing for a portion of the image captured by the image capture device 3100. The portion of the image is an image showing the weather at the location of the vehicle 3400 equipped with the image capture device 3100, with weather information set for the date and time the image was captured. First, the communication control unit 3812 of the information display terminal 3800 acquires weather information for the route taken by the vehicle 3400 from a weather information providing server (not shown) based on the location information of the vehicle 3400 and the date and time information included in the image captured by the image capture device 3100. Next, the display control unit 3811 sets the range of image processing based on the operation of the partial processing button 5545, time information, weather information, etc. Furthermore, based on the operation of the defogging button 5540, it determines that the range is a defogging correction area 5511 in which defogging is to be performed at a predetermined position. Furthermore, the communication control unit 3812 instructs the server 3700 via the network 3900 to perform image processing on the defogging correction area 5511. Next, the image processing unit 3711 performs image processing on the defogging correction area 5511. Furthermore, the communication control unit 3812 acquires the results of image processing on the fog correction region 5511. Finally, the display control unit 3811 displays the results of image processing on the fog correction region 5511. This makes it possible to perform appropriate image processing on a partial region of the captured image based on the weather at the position of the vehicle 3400.Here, the weather information includes not only cloudy and rainy weather but also sunny, clear skies, slightly cloudy weather, smoke and mist, dust storm, blizzard, drizzle, sleet, fog, snow, hail, sleet, and thunder.

[0189] [Modification 14] Modification 14 of embodiment 1 will be described. Fig. 27 shows an example of a screen after image processing, which is displayed by the display control unit 3811 of modification 14 of embodiment 1. 14 is a diagram showing the acceleration information of the vehicle 3400. The sensor unit 3017 of the modification 14 measures acceleration information of the vehicle 3400. That is, the vehicle information of the modification 14 includes the acceleration information measured by the sensor unit 3017. The image display area 5610 of the viewer 5600 of the modification 14 displays an image of the vehicle 3400 captured by the image capture device 3100 while the vehicle 3400 is rapidly accelerating. At this time, the acceleration graph display area 5620 is a graph showing a large upward slope on the X-axis, and the speed graph display area 5670 is a graph showing a rapid increase in speed. At this time, the viewer 5600 may be provided with an acceleration display section 5621 that displays information such as "1 G or more detected." Furthermore, the acceleration correction area 5611, which is part of the image display area 5610, displays the result of acceleration correction. Furthermore, when the acceleration sensor of the vehicle 3400 detects 1 G or more, the acceleration correction area 5611 displays the result of acceleration correction. That is, the image processing for the image is image processing for a portion of the image captured by the image capturing device 3100. The portion of the image is an image in which sensor information of the vehicle 3400 equipped with the image capturing device 3100 is within a set threshold. First, the communication control unit 3812 of the information display terminal 3800 acquires image information and acceleration information of the vehicle 3400 from the server 3700 via the network 3900. Next, the display control unit 3811 sets the range of image processing based on an operation of the partial processing button 5645 and the acceleration information of the vehicle 3400. Furthermore, based on an operation of the acceleration correction button 5640, the display control unit 3811 determines that the range is an acceleration correction area 5611 in which acceleration correction is performed for a predetermined position. Next, the communication control unit 3812 instructs the server 3700 via the network 3900 to perform image processing on the acceleration correction area 5611. Furthermore, the image processing unit 3711 executes image processing on the acceleration correction area 5611. Next, the communication control unit 3812 acquires the results of image processing on the acceleration correction area 5611. Finally, the display control unit 3811 displays the results of image processing on the acceleration correction area 5611. This makes it possible to perform appropriate image processing on a partial area of ​​the image captured based on the sensor information of the vehicle 3400.Here, the sensor information includes, in addition to the acceleration of the vehicle 3400, information such as the speed of the vehicle 3400 (vehicle speed), engine speed, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air flow rate (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of fuel remaining in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on / off)), brake opening, steering wheel rotation angle, gear position, and door open / close state. Note that the system may be configured to execute automatic corrections that combine the corrections of the above-described Modifications 10 to 14. The combination of the corrections of Modifications 10 to 14 may be set using an OR condition or an AND condition.

[0190] [Modification 15] Modification 15 of the first embodiment will be described. FIG. 28 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 15 of the first embodiment after image processing. An image captured by the image capturing device 3100 of the vehicle 3400 is displayed in the image display area 5710 of the viewer 5700 in Modification 15. The display control unit 3811 controls the display to set an automatic defogger area 5711 as a range to be corrected based on a preset setting, and to set a manual defogger area 5713 as a range to be corrected based on a user operation. The display control unit 3811 also controls the display to display a processing content display unit 5712 such as "automatic processing" for the automatic defogger area 5711, and a processing content display unit 5714 such as "manual processing" for the manual defogger area 5713. That is, the screen includes information on whether the range to be subjected to image processing was automatically or manually selected. This makes it possible to distinguish between manually selected ranges and automatically selected ranges in the ranges of image processing that have been performed.

[0191] [Modification 16] Modification 16 of the first embodiment will be described. FIG. 29 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 16 of the first embodiment. In the video display area 5810 of the viewer 5800 in Modification 16, an original image display unit 5811 before image processing, a processing 1 result display unit 5813 after image processing, a processing 2 result display unit 5815 after image processing, and a processing 3 result display unit 5817 after image processing are displayed as multiple different streams. Each display unit displays video captured by the imaging device 3100 of the vehicle 3400. That is, the screen includes an image captured by the imaging device 3100 and one or more images after image processing. The display control unit 3811 controls the display of multiple images by selecting a stream display button 5845. Here, the display control unit 3811 may display multiple images in a chronological order. Further, display control unit 3811 controls original image display unit 5811 to display original display unit 5812 such as "Original," processing 1 result display unit 5813 to display processing 1 content display unit 5814 such as "Process 1," processing 2 result display unit 5815 to display processing 2 content display unit 5816 such as "Process 2," and processing 3 result display unit 5817 to display processing 3 content display unit 5818 such as "Process 3." This allows a comparison of before and after image processing to be displayed. Furthermore, display control unit 3811 displays original image display unit 5811, processing 1 result display unit 5813, processing 2 result display unit 5815, and processing 3 result display unit 5817 as multiple streams. Furthermore, display control unit 3811 displays original image display unit 5811, processing 1 result display unit 5813, processing 2 result display unit 5815, and processing 3 result display unit 5817 so that they can each be selected as a target for writing, playback, etc. At this time, the display control unit 3811 can control the selected display unit to play back the image or to output the image as an image file.In Modification 16, the display control unit 3811 opens multiple candidate video windows, including a processing 1 result display unit 5813, a processing 2 result display unit 5815, and a processing 3 result display unit 5817, and plays them in time synchronization with the original image display unit 5811, which is the original video window. The display control unit 3811 accepts the user's selection of the best one from among them via operation of a selection button or the like. The memory control unit 3813 stores the selection result and the relationship with the original video information in the memory unit 3820. Furthermore, the memory control unit 3813 trains a learning model on the user's image processing tendencies based on the stored selection result and the relationship with the original video information, and stores the results in the memory unit 3820. The display control unit 3811 adjusts image processing based on the results of the machine learning. That is, the display control unit 3811 adjusts image processing parameters based on learning information regarding the selection of images after one or more image processes. This allows the display control unit 3811 to learn the selected image processing results and perform subsequent image processing more appropriately. Furthermore, the display control unit 3811 may perform image processing on a part of the image.

[0192] [Modification 17] Modification 17 of the first embodiment will be described. FIG. 30 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 17 of the first embodiment after image processing. An image captured by the image capturing device 3100 of the vehicle 3400 is displayed in the image display area 5910 of the viewer 5900 in Modification 17. When the defog button 5940 is selected, the display control unit 3811 controls the image display area 5910 to display the image processing results of defogging. The display control unit 3811 also controls the image display area 5910 to display a processing content display section 5911 such as "Defog in progress." That is, the screen includes information on what kind of image processing has been performed on the image captured by the image capturing device 3100. This makes it possible to confirm what kind of image processing has been performed.

[0193] [Modification 18] Modification 18 of the first embodiment will be described. FIG. 31 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 18 of the first embodiment. Recording information in the storage unit 3720 or 3820 is displayed in the playlist display area 6080 of the viewer 6000 in Modification 18. The storage control unit 3813 stores information about the type of image processing performed on images captured by the image capture device 3100 as log information in the storage unit 3820. At this time, the storage control unit 3813 can control the log information to be recorded separately into static information that does not change during capture and dynamic information that changes depending on the video state during capture, etc. More specifically, the storage control unit 3813 controls the recording of one static information per file and one dynamic information at least at each change point, preferably across multiple frames such as all frames. In addition, in the eighteenth modification, the server 3700 may have a function for recording according to a user's selection of whether to record with or without fog removal processing, and a function for recording information that allows determination of which selection was made in the viewer. The storage control unit 3713 of the server 3700 can perform similar processing on the storage unit 3720. The display control unit 3811 can control the reading and processing of these log information at different times. The storage control unit 3813 may also be configured to record drive recorder parameters related to image quality, such as parameters of the drive recorder's image sensor and ISP (Internet Protocol Service) (ISP), in association with drive recorder video data captured with those parameters. In this case, the display control unit 3811 may perform playback using those parameters when playing back on the viewer or the drive recorder. The display control unit 3811 may also have a function for switching between playback using those parameters and playback without those parameters. In this case, the display control unit 3811 can display information about the type of image processing performed on the image captured by the image capturing device 3100.That is, the display control unit 3811 controls the display to display, as in the playlist 6081, what kind of image processing was performed and at what timing on the images captured by the image capturing device 3100. This allows the user to confirm what kind of image processing was performed. Furthermore, when playing back the playlist 6081, the display control unit 3811 may display a screen allowing the user to select whether to play back the image before image processing or the image after image processing based on log information. Furthermore, at this time, the display control unit 3811 may be configured to convert the image after image processing back to the image before image processing based on the log information and output it on the screen. At this time, the image processing unit 3711 performs image processing to restore the original image based on the log information related to image processing stored in the storage unit 3720 and the image after image processing. Next, the communication control unit 3812 acquires the image restored to the original image. Furthermore, the display control unit 3811 displays the restored image. This allows the image after image processing to be restored to the original image before image processing.

[0194] [Modification 19] Modification 19 of the first embodiment will be described. Fig. 32 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 19 of the first embodiment. In the image display area 6110 of the viewer 6100 of Modification 19, an image captured by the image capturing device 3100 of the vehicle 3400 traveling in fog is displayed. The image processing unit 3711 of the server 3700 analyzes the image and detects fog. Next, the communication control unit 37 The device 12 transmits the fog detection result to the information display terminal 3800. Thereafter, the display control unit 3811 controls the detection information display unit 6111 to display a message such as "Fog detected" based on the information that fog has been detected. At this time, the display control unit 3811 also controls the display unit 6111 to display a fog correction button 6140 based on the information that fog has been detected. In other words, the display control unit 3811 changes the buttons included on the screen depending on the object detected from the image. This makes it possible to change the image processing method based on the object in the image.

[0195] [Modification 20] Modification 20 of the first embodiment will be described. FIG. 33 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 20 of the first embodiment. A viewer 6200 in Modification 20 includes an image processing adjustment area 6290. The display control unit 3811 accepts adjustment of image quality parameters through user operation of the image processing adjustment area 6290. The image processing adjustment area 6290 includes a hue adjustment object 6291, a brightness adjustment object 6292, a contrast adjustment object 6293, a saturation adjustment object 6294, a gamma value adjustment object 6295, a sigma value adjustment object 6296, a radius adjustment object 6297, and a variation adjustment object 6298. Selecting the hue adjustment object 6291 executes image processing to adjust the hue of the image. Selecting the brightness adjustment object 6292 executes image processing to adjust the brightness of the image. Selecting the contrast adjustment object 6293 executes image processing to adjust the contrast of the image. Furthermore, selecting a saturation adjustment object 6294 executes image processing to adjust the saturation of the image. Furthermore, selecting a gamma value adjustment object 6295 executes image processing to adjust the gamma value of the image. Furthermore, selecting a sigma value adjustment object 6296 executes image processing to adjust the sigma value of the image. Furthermore, if processing has been performed on only a portion of the image, selecting a radius adjustment object 6297 executes image processing to adjust the boundary between the area where image processing is performed and the area where image processing is not performed. Selecting a variation adjustment object 6298 executes image processing to adjust the number of images per unit time. In other words, the screen includes adjustment objects for adjusting the degree of image processing. In Modification 20, each adjustment object is displayed in the form of a slider; however, radio buttons, check boxes, pull-down menus, and the like may also be used instead of sliders. When the display control unit 3811 receives an operation on an operation unit provided on one of these adjustment objects, it adjusts the degree of image processing in accordance with the received operation.For example, in Modification 20, when the hue adjustment object 6291 is moved to the left end, the display control unit 3811 requests the image processing unit 3711 to perform image processing to set the hue to 0. Furthermore, when the hue adjustment object 6291 is moved to the center, the display control unit 3811 requests the image processing unit 3711 to perform image processing to set the hue to 50. Furthermore, when the hue adjustment object 6291 is moved to the right end, the display control unit 3811 requests the image processing unit 3711 to perform image processing to set the hue to 100. The same applies to the brightness adjustment object 6292, the contrast adjustment object 6293, the saturation adjustment object 6294, the gamma value adjustment object 6295, the sigma value adjustment object 6296, the radius adjustment object 6297, and the variation amount adjustment object 6298. That is, the display control unit 3811 adjusts the degree of image processing in accordance with the operation of the adjustment object. This makes it possible to adjust the degree of image processing, such as brightness and darkness. The adjustment object can also be configured to be adjusted while recording or playing back an image. In this case, the storage control unit 3813 controls the storage of the timing of the adjustment object operation and the timing of the image recording or playback in association with each other. This makes it possible to know the timing at which the degree of image processing was adjusted. Furthermore, the storage control unit 3813 stores the relationship between the adjustment result and the information on the original video in the storage unit 3820. Furthermore, the storage control unit 3813 trains a learning model to learn the user's image processing tendencies based on the stored adjustment result and the relationship with the information on the original video, and stores the results in the storage unit 3820. The display control unit 3811 adjusts the image processing based on the results of this machine learning. That is, the display control unit 3811 adjusts the degree of image processing based on learning information on the degree of adjustment of past image processing. This allows the display control unit 3811 to learn the results of the adjustment of the degree of image processing and perform subsequent image processing more appropriately. The display control unit 3811 may also perform image processing on a portion of the image. Furthermore, the display control unit 3811 may be configured to be able to instruct processing to refrain from further correcting image quality parameters that have been corrected by the drive recorder in the viewer.On the other hand, the display control unit 3811 may be configured to be able to instruct processing in the viewer to make more corrections to image quality parameters that have not been corrected or cannot be corrected in the drive recorder.

[0196] [Modification 21] Modification 21 of the first embodiment will be described. FIG. 34 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 21 of the first embodiment. The viewer 6300 in Modification 21 includes an image processing adjustment area 6390. The image processing adjustment area 6390 includes an image fixation instruction section 6391, a sharpness fixation instruction section 6392, a banding noise removal fixation instruction section 6393, and a film graining fixation instruction section 6394. In other words, the screen includes a fixation instruction section that prohibits adjustment using an adjustment object. The fixation instruction section is an example of a fixation instruction object. When a fixation instruction is issued to a fixation instruction section that fixes an adjustment object, the display control unit 3811 prohibits changes to the operation of the adjustment object. In other words, the display control unit 3811 can fix the adjustment of the degree of image processing by operating these fixation instruction sections. For example, in Modification 21, by checking the checkbox of the image fixation instruction section 6391, the display control section 3811 fixes the degree of adjustment for "hue," "brightness," "contrast," "saturation," and "gamma" within the image adjustment area. At this time, the display control section 3811 displays the image quality adjustment result of the fixed adjustment object. In other words, the display control section 3811 prohibits adjustment by the adjustment object in response to operation of the fixation instruction object. This fixes the degree of image processing. At this time, the display control section 3811 displays the "hue," "brightness," "contrast," "saturation," and "gamma" as if the adjustments are fixed. For example, by checking the checkbox for operation of the fixation instruction section, the display control section 3811 lightens the color of the operation section of the adjustment object to make it inoperable, or hides the operation section of the adjustment object to make it inoperable.

[0197] [Modification 22] Modification 22 of the first embodiment will be described. FIG. 35 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 22 of the first embodiment. In the viewer 6400 of Modification 22, a file button 6401 is expanded to display image output options such as "Browse File," "Save," "Save As," "Write Processing Results," "Write with Deleted Location Information," and "Close." That is, the screen includes an image output object. When the file button 6401 is selected, the display control unit 3811 controls to display a menu related to file processing such as "Browse File," "Save," "Save As," "Write Processing Results," "Write with Deleted Location Information," and "Close." When any file processing is selected, the corresponding processing is performed. Here, when "Write with Deleted Location Information" is selected, the control unit 3810 deletes the location information associated with the image information and outputs the image information. In other words, the control unit 3810 deletes the location information of the image and outputs the image in accordance with the operation of the image output object. This allows an operation to delete location information linked to an image. Here, when "Delete location information and write" is selected, the control unit 3810 may write out the video for posting with the location information deleted. Here, the video for posting is video converted into a format for video posting. At this time, the control unit 3810 may automatically set a title, thumbnail, etc. for the video for posting. Furthermore, when "Write processing results" is selected, the control unit 3810 outputs image information without deleting location information linked to the image information. Here, when "Write processing results" is selected, the control unit 3810 may write out the video for posting without deleting the location information. Furthermore, when outputting image information, the control unit 3810 may be configured to output only the image before image processing, or may output the image after image processing, or may be configured to output the image before image processing and the image after image processing simultaneously. Furthermore, when using the write function for posting, the control unit 3810 may be configured to have a function to store the original video and the processed video in a single file and upload it.In this case, the video to be written out may be a stream in which both videos are lined up, or a video stream in which both videos are lined up in chronological order and connected. Furthermore, in this case, the control unit 3810 may embed a display indicating that each video is the "original video" and the "processed video" before writing out the video. Furthermore, the control unit 3810 may be configured to have a function for selecting a spatial range into which the video is to be written out using a rectangle or the like when writing out the video. In this case, it is preferable that image quality correction is performed only within the selected range before re-encoding and writing out the video.

[0198] [Modification 23] Modification 23 of the first embodiment will be described. FIG. 36 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 23 of the first embodiment. In Modification 23, an image captured by the image capturing device 3100 of a vehicle 3400 traveling behind another vehicle 6511 is displayed in the image display area 6510 of the viewer 6500. In response to an operation of the HDR button 6541, HDR processing is performed in the image display area 6510, thereby including license plate information 6512 that can identify the other vehicle 6511. If the image display area 6510 includes information that can identify an individual, such as the license plate information 6512, the display control unit 3811 performs control to display a personal information processing button 6540. If the personal information processing button 6540 is operated, the display control unit 3811 performs control to display the license plate information 6512 in a masked state. That is, if the image that has undergone image processing includes information that can identify an individual, the display control unit 3811 requests that masking processing be performed on the information that can identify an individual. This allows for appropriate image processing to be performed so that personally identifiable information is not included. Personally identifiable information includes name, date of birth, face, license plate number, and other information that can be linked to a specific individual.

[0199] [Modification 24] In the above-described embodiment, for example, as shown in FIG. 12, an example has been described in which buttons corresponding to a plurality of image processes are displayed on the screen of the information display terminal 3800. However, the information display terminal 3800 may display fewer buttons than the number of image processes. The display control unit 3811 may then request image processing from the image processing unit 3711 or the like in response to the operation corresponding to the set button. More specifically, the display control unit 3811 may request image processing from, for example, one button. It is also possible to display an option on the screen, and when the button is clicked, a first image processing is requested to the image processing unit 3711 etc., when the button is double-clicked, a second image processing is requested to the image processing unit 3711 etc., and when the button is triple-clicked, a third image processing is requested to the image processing unit 3711 etc. According to Modification 24, it is possible to provide a screen that allows the user to instruct the image processing they desire with a small number of buttons.

[0200] [Embodiment 2] In the first embodiment, a case has been described in which image processing is performed by the server 3700 and the image is displayed on the information display terminal 3800. In a modified example, a case in which the information processing terminal 6800 processes the image and displays the image will be described with reference to Figs. 37 and 38.

[0201] [System Configuration of Information Processing System 6600] Fig. 37 is a schematic diagram showing the system configuration of an information processing system 6600 according to a modification of the second embodiment. As shown in Fig. 37, the information processing system 6600 includes a vehicle 3400, a plurality of information processing terminals 6800D, 6800E, and 6800F, and a network 3900. The vehicle 3400 is configured to be able to communicate with the plurality of information processing terminals 6800D, 6800E, and 6800F via the network 3900. More specifically, the vehicle 3400 transmits information held by the vehicle 3400 to the plurality of information processing terminals 6800D, 6800E, and 6800F via the network 3900. The plurality of information processing terminals 6800D, 6800E, and 6800F are configured to be able to communicate with the vehicle 3400 via the network 3900. More specifically, the information processing terminals 6800D, 6800E, and 6800F acquire information of the information processing terminals 6800D, 6800E, and 6800F via the network 3900.

[0202] [Functional Configuration of Information Processing Terminal 6800] Fig. 38 is a schematic diagram showing the functional configuration of an information processing terminal 6800 according to a variation of the second embodiment. The hardware configuration of the information processing terminal 6800 is the same as the hardware configuration of the information display terminal 3800 according to the first embodiment. That is, the control unit of the information processing terminal 6800 executes processing based on a program stored in the storage unit of the information processing terminal 6800, thereby realizing the functional configuration of the information processing terminal 6800 shown in Fig. 38. As shown in Fig. 38, the information processing terminal 6800 includes, as its functional configuration, a display control unit 6811, a communication control unit 6812, a memory control unit 6813, and an image processing unit 6814.

[0203] The display control unit 6811 controls information to be output to an output unit (not shown) of the information processing terminal 6800. More specifically, the display control unit 6811 controls to display a screen including information related to an image.

[0204] The communication control unit 6812 controls the exchange of information with the server 3700 via the network 3900 .

[0205] The storage control unit 6813 stores information in the storage unit of the information processing terminal 6800 and retrieves the stored information.

[0206] The image processing unit 6814 performs image processing on images stored in the storage unit of the information processing terminal 6800 based on operational instructions entered via the screen. This enables the information processing terminal 6800 to perform image processing. In the second embodiment, the image processing unit 6814 performs various types of image processing based on requests entered via the screen of the information display terminal 3800. However, the information processing terminal 6800 may have an image processing unit for each type of image processing. Furthermore, image processing may not be performed by software in the information processing terminal 6800, but may be performed by dedicated hardware (e.g., a GPU, etc.) in the information processing terminal 6800. When dedicated hardware performs image processing, one piece of hardware in the information processing terminal 6800 may be configured to perform multiple types of image processing, or multiple pieces of hardware in the information processing terminal 6800 corresponding to multiple types of image processing may be configured to perform each type of image processing.

[0207] According to the second embodiment, image processing can be performed on the information processing terminal side.

[0208] Third Embodiment In a third embodiment, an example in which image processing is performed within the image capturing device 3100 of the vehicle 3400 will be described.

[0209] The control unit 3011 records the image captured by the imaging unit 3015 in the storage medium 3500. The control unit 3011 performs image processing on the recorded image using the processor 3111. At this time, the control unit 3011 records in the storage medium 3500 what kind of image processing was performed and at what timing. At this time, the control unit 3011 can transmit the information recorded in the storage medium 3500 via the network 3900.

[0210] The control unit 3011 controls the image capturing device 3100 so that a processed image is written out when a predetermined operation is performed during playback in the image capturing device 3100. The control unit 3011 issues a predetermined notification while an image processed in the image capturing device 3100 is being written out. More specifically, while an image processed in the image capturing device 3100 is being written out, the control unit 3011 controls the display unit 3013 to display a message saying "Do not turn off the engine" and controls the audio output unit 3014 to output a voice saying "Do not turn off the engine." These notifications can be issued continuously (repeatedly).

[0211] In the third embodiment, the control unit 3011 may also have an image processing function capable of removing fog. In this case, the control unit 3011 removes fog from the target image in response to a user operation. Next, the control unit 3011 records the image after the fog removal process in response to a user operation. Furthermore, the control unit 3011 may be configured to reproduce and play back the image before the fog removal process. This reproduction may be performed by the control unit 3011 recording the image before the fog removal process in association with data after the fog removal process, or by recording information indicating the degree of fog removal by the fog removal process in association with the image after the fog removal process and using this information. For example, if there is a large difference between the state as seen by the driver and the state after the image correction, the recorded image may differ from the state as seen by the driver, which may be disadvantageous to the driver if he or she were to cause an accident. However, the degree of fog removal by the fog removal process can be understood.

[0212] The control unit 3011 may also have a function of recording in accordance with a user's selection of whether to record with or without defogging processing. Furthermore, the control unit 3011 may have a function of displaying this information on the information display terminal. The control unit 3011 may also have a function of recording information that enables a determination on a viewer of the information display terminal as to whether recording with or without defogging processing was selected. Note that the information may be transferred from the drive recorder to the information display terminal via a network 3900 or by using an information recording medium such as an SD card.

[0213] According to the third embodiment, image processing can be performed on the vehicle side.

[0214] [Fourth Embodiment] Next, a fourth embodiment will be described. The fourth embodiment may be implemented by the information processing system 3000 shown in Fig. 1. Fig. 39 is a functional block diagram realized by the storage unit 22 and the control unit 23. The control unit 23 shown in Fig. 39 may be, for example, a control unit 3810 provided in the information display terminals 3800A, 3800B, 3800C, etc. The storage unit 22 shown in Fig. 39 may be, for example, a storage unit 3820 provided in the information display terminals 3800A, 3800B, 3800C, etc.

[0215] The storage unit 22 stores various pieces of information defined above. This may be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the information display terminal 3800 executed by the control unit 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to program calculations. It may also be a combination of these. In particular, the storage unit 32 may store, as various programs related to the information display terminal 3800 executed by the control unit 33, a first program P1 and a second program P2 that function as viewers for image data IM, for example.

[0216] The control unit 23 may function as an acquisition unit 231, a reception unit 232, a correction unit 233, a release unit 234, and a display control unit 235 by executing various programs stored in the storage unit 22. That is, information processing by software stored in the storage unit 22 may be specifically realized by the control unit 23, which is an example of hardware, and may be executed as each functional unit included in the control unit 23. Each functional unit related to the control unit 23 will be described below.

[0217] The acquisition unit 231 may be configured to acquire various information. For example, the acquisition unit 231 may acquire specific information IF1 relating to the state or situation of the moving object as an acquisition step. This will be described in further detail later.

[0218] The receiving unit 232 may be configured to receive various types of information. For example, the receiving unit 232 may receive a first input from a user as a first receiving step. The receiving unit 232 may receive a second input from a user as a second receiving step. This will be described in further detail below.

[0219] The correction unit 233 may be configured to execute a correction process. For example, as a first correction step, the correction unit 233 may execute a first correction process on image data IM generated by the image capture device 3100 possessed by the moving object at a time corresponding to the identification information IF1, based on the identification information IF1. The first correction process may include, for example, automatic correction process by a program such as artificial intelligence. As a second correction step, the correction unit 233 may execute a second correction process on the image data IM based on a second input. The second correction process may include, for example, any manual correction process based on a user input. Furthermore, the correction unit 233 may execute a preparation step and a processing step. This will be described in further detail below.

[0220] The release unit 234 may be configured to release the correction process performed by the correction unit 233. For example, as a release step, the release unit 234 may release the first correction process performed on the image data IM based on a first input. This will be described in further detail later.

[0221] The display control unit 235 is configured to control various visual information to be displayed on a display unit capable of displaying the same. Here, the visual information may be information itself generated in a manner that is visible to the user, such as a screen, an image, an icon, or text, or may be rendering information for displaying the screen, image, icon, or text. For example, the display control unit 235 may, as a display control step, display the image data IM on which the first correction process has been performed in a reproducible manner. This will be described in further detail later.

[0222] [Outline of Information Processing] Next, an information processing method of the information processing system 3000 according to the fourth embodiment will be described. The information processing method includes steps of an information display terminal 3800, which is an example of an information processing device. FIG. 40 is an activity diagram showing the flow of information processing according to the fourth embodiment. Below, this information processing will be outlined along this activity diagram.

[0223] First, when the moving object is moving, the image capturing device 3100 may record the moving state as image data IM (activity A101). The image data IM may be, for example, moving image data with a predetermined frame rate. The frame rate of the image capturing device 3100 is not particularly limited, but is, for example, 10 fps or more, preferably 30 fps or more, and more preferably 60 fps or more, and specifically, for example, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 210, 220, 230, 240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 350, 360, 370, 380, 390, 400, 410, 420, 430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 550, 560, 570, 580, 590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710, 720, 730, 740, 750, 760, 770, 780, 790, 800, 810, 820, 8 The frame rate may be 0, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, or 250 fps, or may be within a range between any two of the values ​​exemplified here. The generated image data IM may be displayed as a live view on the display unit 3013 of the imaging device 3100. Recording of such image data IM may be performed continuously until the traveling of the moving object is completed.

[0224] Subsequently, the information display terminal 3800 connected to the photographing device 3100 via a network may receive the series of image data IM generated by continuous recording in activity A101 (activity A102). That is, the reception unit 332 may receive the image data IM from the photographing device 3100. Of course, instead of via a network, the reception unit 332 may also receive the image data IM via a storage medium such as an SD card (registered trademark). In the following description, for simplicity, via a network will be exemplified, but the present invention is not necessarily limited to this.

[0225] Next, the acquisition unit 231 may acquire specific information IF1 related to the state or situation of the moving object (activity A103) as an acquisition step. Specifically, the control unit 23 may acquire the specific information IF1 by reading a dedicated program stored in the storage unit 22 and store the specific information IF1 in the storage unit 22. The specific information IF1 may be included in the image data IM received by the reception unit 332, or may be received as separate information from the image capture device 3100 or an external device. Preferably, the specific information IF1 may be at least one of the moving object's position, orientation, speed, acceleration, time, time zone, weather, driving state, and pedal work. For example, an appropriate type of correction may be automatically performed for the tunnel position, or different image corrections may be applied for sunny, rainy, and foggy conditions. Furthermore, different image corrections may be applied for morning, night, and evening, or an appropriate correction method may be selected depending on the situation, weather, etc., such as fog, snow, and rain. Optimal parameters for each may be prepared in advance. More specifically, the buttons may be implemented as buttons for correcting images according to the purpose of the image, such as an operation button for erasing a snowstorm, a button for making fog transparent so that the other side of the fog can be seen, a button for erasing rain, etc. Furthermore, as an all-purpose automatic correction, an automatic correction that combines these as appropriate may be implemented.

[0226] According to this aspect, more appropriate image processing can be performed by taking into account at least one of the specific conditions of the moving object, such as position, speed, acceleration, running state, and pedal work, thereby further improving usability.

[0227] Next, as a first correction step, the correction unit 233 may execute a first correction process on image data IM generated by the image capturing device 3100 of the moving object at a time corresponding to the identification information IF1, based on the identification information IF1 (activity A104). Specifically, the control unit 23 may determine a correction range for the image data IM based on the identification information IF1 by reading a dedicated program stored in the storage unit 22. That is, preferably, as a first correction step, the correction unit 233 may execute the first correction process on the image data IM by varying the correction range, which is the range to be corrected, based on the identification information IF1. According to this aspect, it is not necessary to correct the entire range, thereby reducing the processing load. Furthermore, since correction is applied to a preferred range, a highly usable function can be realized. Conversely, the correction unit 233 may execute the first correction process on the image data IM by varying the range that should not be corrected, based on the identification information IF1. Furthermore, the first correction process may be executed appropriately by varying both the range that should be corrected and the range that should not be corrected.

[0228] Thereafter, the display control unit 235 may, as a display control step, display the image data IM on which the first correction process has been performed so as to be reproducible (activity A105). It is preferable that such a display be capable of reproducing the image data IM as video data by displaying the frames of the image data IM frame by frame.

[0229] During display control by the display control unit 235, the receiving unit 232 may receive a first input from the user as a first receiving step. The control unit 33 may be configured to switch between performing and not performing correction processing on the image data IM based on the first input (activity A106). For example, when correction processing is being performed on the image data IM, the canceling unit 234 may cancel the first correction processing on the image data IM based on the first input as a canceling step. For example, when correction processing on the image data IM is canceled, the correcting unit 233 may resume the first correction processing on the image data IM based on the first input. According to this aspect, if the correction processing presented by the device is unnecessary, the user can cancel it at their will, thereby improving visibility and usability. This can be achieved, for example, by providing a button that temporarily cancels the correction while leaving the correction applied to the area.

[0230] In summary, the information display terminal 3800, which is an example of an information processing device according to this embodiment, includes a control unit 23. The control unit 23 includes the following units. As a first correction step, the correction unit 233 performs a first correction process on image data IM generated by the imaging device 3100 possessed by the moving object at a time corresponding to the identification information IF1, based on the identification information IF1. As a display control step, the display control unit 235 displays the image data IM on which the first correction process has been performed in a reproducible manner. According to this aspect, appropriate image processing can be performed based on the state or situation of the moving object, thereby improving usability.

[0231] [Details of Information Processing] Next, the details of the information processing outlined above will be described. Fig. 41 is a schematic diagram showing an example of a spatial correction range of the first correction processing. Fig. 42 is a schematic diagram showing an example of a temporal correction range of the first correction processing. That is, the first correction processing may be applied with a limited location range or time range.

[0232] In the embodiment of FIG. 41, image data IM is displayed by the display control unit 235, and this is shown as a playback screen 4. The correction range is a region of interest R set in the frame of the image data IM. The playback screen 4 has a correction region 41 and a non-correction region 42. The region of interest R is shown as the correction region 41. Specifically, the correction region 41 may be determined according to various information included in the identification information IF1.

[0233] For example, the size of the correction area 41 may be changed depending on the speed and acceleration of the moving object. For example, when the moving object turns left, a correction process may be performed so that the left side of the moving object is visualized, in order to prevent a hit-and-run accident. The shape and size of the correction area 41 are entirely arbitrary, and may be, for example, rectangular, circular, or elliptical. The shape may be configured to be changeable based on user input. According to such an embodiment, correction can be applied by narrowing down the area where visibility is originally desired to be increased, thereby improving usability.

[0234] Furthermore, for example, appropriate correction processing may be performed to allow the user to determine the position of the sun from the time and time zone, making it easily visible even in backlit conditions. In the embodiment shown in FIG. 42, a correction time 51 is displayed in a seek bar 5 located below the playback screen 4 and indicating the playback time of the image data IM. The seek bar 5 may correspond to a concept such as a timeline in video editing software. For example, the correction time 51 is a time period during which the setting sun shines in, and which, depending on the direction of travel of the moving object, results in backlit shooting for the image capturing device 3100. In other words, the identification information IF1 is the position and orientation of the moving object. As a first correction step, the correction unit 233 may perform a first correction processing on the image data IM based on the position of the sun estimated from the position and orientation. According to this embodiment, appropriate image processing can be performed taking the position of the sun into account, further improving usability.

[0235] Furthermore, the various correction processes described above may be performed as manual correction by the user, which is an example of a second correction process. In other words, the receiving unit 232 may receive a second input from the user as a second receiving step, and the correction unit 233 may perform a second correction process on the image data IM based on the second input as a second correction step. According to this aspect, in addition to the correction process presented by the device, the user may add correction at their own discretion, thereby improving visibility and further improving usability.

[0236] Furthermore, in relation to the first correction process and the second correction process, the display control unit 235 may display the image data IM in a manner that allows the first correction process and the second correction process to be distinguished from each other. In other words, the display may be such that manually added ranges and automatically added ranges are distinguishable. FIG. 43 is a schematic diagram showing additional information 43 added to the playback screen 4 when the first correction process and the second correction process are performed. As shown in FIG. 43, additional information 43 such as "Automatic correction in progress" or "Manual correction in progress" may be added to the upper right corner of the playback screen 4. Here, the additional information 43 is a text annotation, but illustrations or icons may also be used instead. This manner allows the user to understand the type of correction process, making it user-friendly.

[0237] The correction method is not particularly limited, and at least one of brightness, dynamic range, contrast, gamma, saturation, hue, etc. may be appropriately corrected. It is preferable that the correction be performed based on what is easily visible to the human eye.

[0238] Conversely, image data IM that is difficult for the user to see may be realized. For example, from the perspective of protecting personal information, a function may be realized to blur the faces of an unspecified number of people when they are captured. Mosaicing may be performed at least one of the following: when recording, when a man is recognized, and when a woman is recognized. A function may also be realized to blur the license plates of other vehicles. The targets to be blurred may be determined by artificial intelligence or other methods. This type of blurring process is an example of a correction process that makes an image dirty after it has been cleaned up. Mosaicing is an example of "making it difficult to see" and may be realized by other methods.

[0239] Furthermore, unlike correction by an ISP (Imaging Signal Processor) that removes aberrations, noise, and the like due to optical elements and optical systems, this correction processing is a further correction processing performed on image data IM obtained through the ISP. Specifically, this may include color correction such as gamma correction, color enhancement, and color balance, and edge enhancement. Conversely, the original image data IM refers to image data IM obtained through correction by the ISP. This correction processing may be performed after correction by the ISP and before encoding into an appropriate file format such as H264, or may be performed by correcting image data IM that has already been encoded and then encoding it again.

[0240] [Modification 1] Next, Modification 1 of the fourth embodiment will be described. In this Modification 1, the acquisition unit 231 may acquire various information other than the specific information IF1 from an external device in addition to the specific information IF1. In Modification 1, for example, the specific information IF1 may be the position of the moving body. The acquisition unit 231 may acquire road network information IF2 as an acquisition step, and the correction unit 233 may execute a first correction process on the image data IM based on the position and the road network information IF2 as a first correction step. Specifically, the control unit 23 may acquire the road network information IF2 stored in the storage unit 22. By reading out the dedicated program, the road network information IF2 is acquired, and the first correction process is then executed using this information.

[0241] The road network information IF2 has various information as attribute data, such as speed limits, road width, weekday 12-hour traffic volume, travel speed (for a certain section, the distance of the section divided by the time required to travel, the required time including stops due to traffic lights, congestion, etc.), and the number of lanes. The correction unit 233 may determine an appropriate type or range of correction based on this information and execute the first correction process. According to this aspect, appropriate image processing can be performed depending on the location and the road conditions at that time, thereby further improving usability.

[0242] [Modification 2] Next, Modification 2 of Embodiment 4 will be described. In Modification 2, similar to Modification 1, the acquisition unit 231 may acquire various information other than the identification information IF1 from an external device in addition to the identification information IF1. In Modification 2, for example, the identification information IF1 may be the position of the moving object. The acquisition unit 231 may acquire topographical information IF3, and the correction unit 233 may execute a first correction process on the image data IM based on the position and the topographical information IF3 as a first correction step. Specifically, the control unit 23 may acquire the topographical information IF3 by reading a dedicated program stored in the storage unit 22, and then execute the first correction process using the acquired topographical information.

[0243] The topographical information IF3 may be acquired, for example, from an external server associated with an organization that issues map data. The topographical information IF3 may include information on topography, such as urban areas, suburban areas, expressways, general roads, gravel roads, mountains, and tunnels. The correction unit 233 may determine an appropriate type or range of correction based on this information and execute the first correction process. For example, in a busy downtown area, the influence of neon lights is strong, so correction processing may be executed to alleviate this influence. For example, in a forest, correction processing may be executed to enhance overall brightness. According to this aspect, appropriate image processing can be executed depending on the location, the topography, the environment, and the like at that time, thereby further improving usability.

[0244] [Variation 3] Next, Variation 3 of Embodiment 4 will be described. In Variation 3, the correction unit 233 may execute a preparatory step and a processing step. Furthermore, the control unit 23 may be configured to be able to call a pre-stored first program P1 and a second program P2 different from the first program P1. In particular, the first program P1 may be a player application recommended for playing image data IM, and the second program P2 may be a general-purpose player application with correction processing functions. In other words, since incorporating too many different functions into the first program P1, which is a standard viewer, would result in bloated performance, free software such as VLC or image processing services provided by various companies can be used instead. Figure 44 is an activity diagram illustrating the flow of information processing according to Variation 3 of Embodiment 4. Below, this information processing will be outlined in accordance with this activity diagram.

[0245] First, when the accepting unit 232 of the information display terminal 3800 accepts an input from the user for playing a video, the control unit 23 may call the first program P1 stored in the storage unit 22 (activity A201). Next, the first program P1 may be caused to read image data IM (activity A202). Next, as an operation based on the first program P1, the acquiring unit 231 may extract and acquire a plurality of parameters PT included in the image data IM (activity A203).

[0246] If the second program P2 is not pre-installed on the information display terminal 3800, the display control unit 235 may suggest installing the second program P2. In other words, if the second program P2 is not installed, the display control unit 235 may display a screen recommending its installation. The recommendation screen preferably includes the name of the second program P2, and more preferably includes the URL of the installation destination. According to this aspect, even if the second program is not installed, the first correction process can be performed via the second program.

[0247] Specifically, when the first program P1, which is a PC viewer, is installed, the user may be asked whether to install an external player, and if instructed to do so, the external player may be installed. During installation or immediately after startup, a button to launch the external player may be displayed. Cautions for using the external player may be displayed. Cautions for using the external player may include the following: when a button on the external player is operated, the display state of the first program P1, which is a PC viewer, may differ from that of the first program P1; in that case, it may be recommended to close the second program P2, which is an external player, and then launch the second program P2, which is an external player, again from the first program P1, which is a PC viewer.

[0248] If the second program P2 is pre-installed on the information display terminal 3800, the control unit 23 may call the second program P2 stored in the storage unit 22 (activity A205). That is, an external player launch button or the like may be provided on the UI of the first program P1, which is the viewer, and pressing this button may launch the second program P2, such as VLC. Then, as an operation based on the second program P2, the acquisition unit 231 may acquire multiple parameters PT included in the image data IM (activity A206). Here, the parameters PT may include at least one of hue, brightness, saturation, contrast, gamma, sharpness, playback start time, playback end time, and whether or not to loop. According to this embodiment, since each parameter is a general parameter possessed by image data, a more general-purpose image processing program can be used as the second program, thereby achieving a device with high usability. Furthermore, as an operation based on the second program P2, the acquisition unit 231 may acquire operation information IF4 related to the operation of the second program P2 (activity A207).

[0249] Specifically, instead of incorporating libvlc.dll into the viewer, the viewer can be launched by passing the video file name and video effect settings as "command line arguments" ( vlc.exe 20200101090000.avi --hue=0 --brightness=1.5 --contrast=0.6 --saturation=1.9 --gamma=0.3 --sharpen-sigma=0.9 Additionally, parameters such as the playback start time, playback end time, and whether or not to loop can be passed. Parameters that disable at least some of the controls of the second program P2 (external player) can also be passed. In addition, the first program P1 (viewer) can send control commands to control the second program P2 (external player). For example, if the second program P2 is a VLC player, it can be controlled from the first program P1 (viewer) via HTTP, telnet, etc.

[0250] In other words, the first program P1 may execute a preparatory step. Then, as the preparatory step, the correction unit 233 may extract parameters PT included in the image data IM and pass the values ​​of the extracted parameters PT to the second program P2. Furthermore, as the preparatory step, the correction unit 233 may pass operation information IF4 related to the operation of the second program P2 to the second program P2.

[0251] Subsequently, the correction unit 233 may change the value of the parameter PT by executing a first correction process on the parameter PT as a processing step (activity A208). Then, the display control unit 235 may display image data IM based on the value of the parameter PT changed by the processing unit in a reproducible manner. According to this embodiment, a separate program more specialized for the correction process can be used, which contributes to reducing calculation costs and improving image processing performance.

[0252] Preferably, the display control unit 235 may limit at least some of the functions of the second program P2 based on the operation information IF4, and display the image data IM in a reproducible manner (activity A209). According to this embodiment, the second program can be used solely for correction and reproduction, and the user can avoid using unnecessary functions of the second program, thereby realizing a device with high usability.

[0253] [Modification 4] Next, Modification 4 of Embodiment 4 will be described. In the above-described Modification 3, if the second program P2 is not installed, a screen recommending installation of the second program P2 is displayed. In this Modification 4, the second program P2 can be used via a network without being installed on the information display terminal 3800. In other words, the second program P2 may be a program stored in an external device. The control unit 23 may be configured to call the second program P2 via the network. According to this aspect, even if the second program is not installed, the first correction process can be performed via the second program in the form of a cloud service.

[0254] Specifically, the first program P1 should have functions such as a function to store an ID for automatically logging in to the cloud image sharpening service, an automatic sending function, an uploading function, a function to determine whether the conversion is complete and then automatically download the image when it is complete or to notify the user that it is available for download, and a function to play the downloaded image in synchronization with the image before conversion.

[0255] [Modification 5] Next, Modification 5 of the fourth embodiment will be described. The correction processes described above are various, and different correction processes may be performed in succession. Different correction processes may be performed on a single playback screen 4 at the same time, or different correction processes may be performed at different times. It is preferable that the different correction processes be determined according to needs and the scene.

[0256] In the former case, for example, when using an imaging device 3100 with a fisheye lens, the deck board of a moving object is often reflected near the center of the image data IM, and this portion is often unnecessary as information. Therefore, by not performing correction processing on this central portion, the overall processing load can be reduced. Furthermore, since the information in the uncorrected portion has poor visibility, some additional information 43 may be embedded in this position. For example, the original image data IM before correction may be embedded, or a recommended player may be presented.

[0257] In the latter case, for example, if a person suddenly appears from behind an object at the moment of a traffic accident, the focus should be on the person gradually coming forward from a dimly lit area, and correction processing should be performed so that the user can appropriately recognize the person. Different processing may be performed inside and outside the vehicle.

[0258] [Variation 6] Next, Variation 6 of the fourth embodiment will be described. A sharpening process may be performed to sharpen a portion of an image captured by an imaging device using a fisheye lens (for example, one or both of the imaging devices 3100 and 3200). The sharpening process is an example of a correction process for correcting the captured image. In particular, it is preferable to perform a sharpening process to sharpen a portion of an image captured by an imaging device using a circular fisheye lens (hereinafter referred to as a "circular fisheye image"). In this case, a processing pattern may be adopted in which the circular fisheye image is subjected to the sharpening process as is, and then a shape transformation (geometric transformation) is performed. In particular, the shape transformation is a process of projecting onto a rectangle or the like, in other words, a process of converting a planar image into a rectangular shape. It is preferable to use a conversion process. This can simplify the process in a configuration in which multiple locations are extracted from a circular fisheye video and displayed on a rectangular screen. Alternatively, when extracting a portion from a circular fisheye video and displaying the image, if there is a function that allows the extracted area to be changed during playback, it is possible to improve the responsiveness to area change operations. Furthermore, in cases where multiple extracted areas overlap (for example, when multiple rectangular areas overlap, such as in the shape of a "turn" or an "eye"), it is preferable to have a function that reduces the sharpening process for the overlapping areas. This can simplify the process by reducing the amount of overlapping sharpening process.

[0259] In a processing pattern in which a portion of a circular fisheye image is extracted and transformed into a shape before being sharpened, it is advisable not to perform sharpening processing on areas other than the extracted area. In this way, the range of sharpening processing is limited, thereby reducing the processing load.

[0260] A part of the circular fisheye image captured by the circular fisheye lens may be cut out, transformed into a shape (for example, projected into a rectangle), and then sharpened. This type of processing can be called double narrowing of the sharpening range.

[0261] The sharpening process should not be performed on devices such as low-performance PCs. Whether a device is a low-performance PC or the like can be determined automatically or manually by the user. Low performance means that the processing power of the device is below a predetermined specification, and in particular, the sharpening process should not be performed on devices such as PCs that are not equipped with a GPU.

[0262] The device may also be provided with a function to export the image after sharpening processing, or a function to upload the image for posting to social networking sites such as YouTube (registered trademark).

[0263] [Variation 7] Next, Variation 7 of Embodiment 4 will be described. In the above embodiment, the recorded image data IM is described as being of a moving vehicle, but the image data IM may be recorded as part of the parking monitoring function of the image capturing device 3100. In this case, the parking monitoring function of the image capturing device 3100 may perform a correction process that is different from that performed when the vehicle is moving.

[0264] [Fifth Embodiment] Next, a fifth embodiment will be described. The fifth embodiment may be implemented by the information processing system 3000 shown in Fig. 1. Fig. 45 is a functional block diagram realized by the control unit 33. The control unit 33 shown in Fig. 45 may be, for example, a control unit 3011 provided in the image capturing device 3100 mounted on the vehicle 3400. The storage unit 32 shown in Fig. 45 may be, for example, a memory 3112 provided in the image capturing device 3100 or the like.

[0265] The memory unit 32 stores various information defined above. This may be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the imaging device 3100 executed by the control unit 33, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to program calculations. Alternatively, a combination of these may be used. In particular, the memory unit 32 may store a third program P3 and a fourth program P4 as various programs related to the imaging device 3100 executed by the control unit 33.

[0266] The control unit 33 may function as an acquisition unit 331, a reception unit 332, a recording unit 333, a correction unit 334, and a display control unit 335 by executing various programs stored in the storage unit 32. In other words, information processing by software stored in the storage unit 32 may be specifically realized by the control unit 33, which is an example of hardware, and may be executed as each functional unit included in the control unit 33. Each functional unit related to the control unit 33 will be described below.

[0267] The acquisition unit 331 is configured to acquire various types of information.

[0268] The receiving unit 332 is configured to receive various types of information.

[0269] The recording unit 333 is configured to record data on a storage medium. For example, the recording unit 333 records video captured by a camera as image data IM. This will be described in more detail below.

[0270] The correction unit 334 is configured to perform a correction process.

[0271] The display control unit 335 is configured to control various visual information to be displayed on a display unit capable of displaying the same. Here, the visual information may be information itself generated in a manner that is visible to the user, such as a screen, an image, an icon, or text, or may be rendering information for displaying the screen, image, icon, or text. For example, the display control unit 335 may display video captured by a camera as a live view. This will be described in further detail later.

[0272] Next, an information processing method of the information processing system 3000 according to the fifth embodiment will be described. In the fifth embodiment, the control unit 33 may be configured to be able to call a pre-stored third program P3 and a fourth program P4 different from the third program P3. In particular, the third program P3 may be a player application recommended for playing back image data IM, and the fourth program P4 may be a general-purpose player application. FIG. 46 is an activity diagram showing the flow of information processing according to the fifth embodiment. Below, this information processing will be outlined in accordance with this activity diagram.

[0273] While the moving object is traveling, the following activities A301 to A305 are sequentially repeated. First, while the moving object is traveling, the recording unit 333 may record the video captured by the image capturing unit 3015 of the image capturing device 3100 as image data IM (activity A301). Specifically, the control unit 33 may read out a dedicated program stored in the storage unit 32, thereby causing the image data IM to be sequentially stored in the storage unit 32.

[0274] The image data IM may be, for example, moving image data having a predetermined frame rate. The frame rate of the image capturing device 3100 is not particularly limited, but may be, for example, 10 fps or more, preferably 30 fps or more, and more preferably 60 fps or more, specifically, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, or 250 fps, or may be within a range between any two of the values ​​exemplified here.

[0275] Next, the processing of activities A302 and A303 for displaying the image data IM on the display unit 3013 of the photographing device 3100, and the processing of activities A304 and A305 for later playing back the image data IM on the information display terminal 3800 are executed in parallel.

[0276] Specifically, the image data IM is recorded, and each frame is subjected to a first correction process (activity A302), and then the image is displayed as a live view (activity A303). In other words, the display control unit 335 displays the image captured by the camera as a live view. Here, the values ​​of the parameter PT for the recorded image data IM and the displayed image may be set to be different.

[0277] Fig. 47 is a schematic diagram showing aspects of image data IM displayed as a live view and image data IM recorded so that it can be played back later. As shown in Fig. 47, when the recording unit 333 records the image data IM, the recording unit 333 may record the image as image data IM without correcting the video, while the display control unit 335 may correct the video and display it as a live view. This aspect makes it possible to improve the visibility of the video viewed as a live view while preventing information loss due to processing of the image data to be recorded.

[0278] On the other hand, when image data IM displayed as a live view is recorded, the acquisition unit 331 may extract and acquire various parameters PT included in the image data IM (activity A304). Then, the recording unit 333 may record the parameters PT of the displayed video as a parameter log PL in chronological order together with the image data IM (activity A305). According to this embodiment, it is possible to save the visibility of the video viewed as a live view.

[0279] After the moving object has completed its travel, the image data IM may be played back via an information display terminal 3800 connected to the imaging device 3100 via a network. When the image data IM is played back on the information display terminal 3800, the reception unit 332 of the information display terminal 3800 receives a third input from the user for playing back the image data IM as a third reception step (activity A306). Subsequently, the third input may trigger the control unit 23 to read out the image data IM (activity A307).

[0280] Next, if the third program P3 is installed in the information display terminal 3800, the control unit 23 may read the parameter log PL as an operation based on the third program P3 (activity A308). Next, as an operation based on the third program P3, the correction unit 233 may refer to the values ​​of the parameter log PL and execute a first correction process to appropriately change the values ​​of each parameter of the original image data IM based on the values ​​of the parameter log PL (activity A309). Thereafter, as an operation based on the third program P3, the display control unit 235 may display the corrected image data IM (activity A311).

[0281] In other words, the display control unit 335 may read the image data IM and the parameter log PL based on the third input and display the corrected image data IM in a reproducible manner. According to this embodiment, when the user desires, a highly visible image can be reproduced on the spot, thereby realizing a device with high usability.

[0282] On the other hand, if the third program P3 is not installed on the information display terminal 3800, the control unit 23 may generate additional information 43 recommending playback in the third program P3 as an operation based on the fourth program P4 (activity A310). Thereafter, as an operation based on the fourth program P4, the display control unit 235 may display the additional information 43 superimposed on the image data IM in its recorded state (activity A311). In other words, when the image data IM is played in the fourth program P4, the display control unit 335 may display information encouraging playback in the third program P3. According to this configuration, even if the image data IM is played in an unintended program, playback in a preferred program can be induced, and appropriate information can be provided to the user.

[0283] Fig. 48 is a schematic diagram showing an aspect in which additional information 43 encouraging playback of the recommended third program P3 is superimposed on the playback screen 4. As shown in Fig. 48, the information encouraging playback may be displayed together with the playback screen of the image data IM. In this aspect, it is possible to provide the user with appropriate information that the current appearance is undesirable, but that there is no problem with the image data itself.

[0284] Alternatively, a QR code (registered trademark) encoding the URL of the third program P3 may be embedded and recorded along with the string of characters, or a video of a screen with these written on it may be recorded at the beginning of the playback video file.

[0285] To summarize the above, the photographing device 3100, which is an example of an information processing device according to this embodiment, includes a control unit 33. The control unit 33 includes the following units: A recording unit 333 records the image captured by the camera as image data IM; a display control unit 335 displays the image captured by the camera as a live view; Here, the parameters PT of the recorded image data IM and the displayed image are set to be different. According to this embodiment, by making the image viewed as a live view different from the image data to be recorded, it is possible to ensure both visibility and customizability.

[0286] In other words, various parameters PT related to image quality, such as parameters of the image sensor and ISP of the imaging device 3100 such as a drive recorder, can be recorded in association with the video data of the drive recorder, and when playing back on the viewer or main unit, these parameters can be used for playback.

[0287] [Variation 1] Next, Variation 1 of Embodiment 5 will be described. In this Variation 1, unlike in Embodiment 5, the image data IM to be recorded is image data IM that has been subjected to correction processing and has high visibility, similar to that used for live view. On the other hand, if it is desired to play back the original image data IM before correction later, image data IM may be generated by applying correction processing to the image data IM using an information display terminal 3800 or the like. Such correction processing is preferably an accurate inverse conversion processing, but correction processing that is assumed to be an inverse conversion may also be applied.

[0288] [Variation 2] Next, Variation 2 of Embodiment 5 will be described. In this Variation 2, parameters PT are recorded in two types. The parameters PT include parameters that remain constant throughout the entire time series of image data IM, and parameters that change on a frame-by-frame basis. By recording the former on an image data IM basis and recording the latter on an image data IM frame-by-frame basis, it is possible to reduce the processing load and perform correction processing efficiently.

[0289] In other words, the parameters PT are recorded separately as static ones that do not change during shooting, and dynamic ones that change depending on the video conditions during shooting, etc., and for example, one static one is recorded per file, and one dynamic one is recorded at least at the point of change, preferably across multiple frames such as all frames, and the viewer can read and process these separately at different times.

[0290] [Others] A program that causes a computer to function as a control unit of an information processing device, for example, the control unit 33 in the image capturing device 3100 or the control unit 23 in the information display terminal 3800, may be provided.

[0291] Since there is an issue of needing a mechanism to switch which of the different algorithms for fog, HDR, sharpening, etc. to apply to the currently playing video, the following aspects of correction processing may be adopted. 1. Obtain the position and direction of the sun from GPS and change or vary the correction content applied to the video. 2. Determine whether the video is in a city, a forest, etc. from GPS location information and map information and change the algorithm to be applied. 3. If an operation to turn on the fog lights is detected, apply the fog algorithm. 4. If the difference in contrast is small and there is white, determine that there is fog, and apply the fog removal algorithm. 5. If there is a lot of black, determine that there is darkness, and brighten the video. 6. Vary CLAHE, gamma, and contrast as a set.

[0292] The camera 3100 may have a function for writing out, in a separate file from the recorded video, video that has undergone image quality correction by the camera 3100, for video recorded by the camera 3100 or the like.

[0293] The drive recorder may also have a function for writing out video with corrected image quality by performing a predetermined operation during playback on the drive recorder itself.

[0294] While the image with corrected quality is being written, a message to not turn off the engine may be continuously displayed, and it is particularly recommended to repeat this message by voice.

[0295] The viewer may be provided with an operation unit, such as a radio button, check box, pull-down menu, slider, etc., that adjusts image quality parameters that can be operated by the user, and while playing back the recorded video, the image quality adjustment results according to the operation of the operation unit are displayed. When a fixation instruction is given to fix the operation unit in the displayed state, such as a check box that gives a fixation instruction, the operation of the operation unit may be prohibited from being changed, and the image quality adjustment results of the fixed operation unit may be displayed.

[0296] The viewer may be provided with an operation unit that allows the user to adjust image quality parameters, such as radio buttons, check boxes, pull-down menus, sliders, etc., and may have a function to record the user's operation of the operation unit in synchronization with the time of the video.The viewer may also have a function to adjust the image quality according to the recorded user operation of the operation unit and play it back.

[0297] An imaging device 3100 such as a drive recorder or an information display terminal 3800 on which a first program P1 serving as a viewer is installed may be provided with a function for writing out images for posting with location information deleted, and a function for writing out images for posting without deleting location information.

[0298] A selection function may be provided in which the video after correction of the recorded video is displayed as multiple streams with different correction states, and the user can select the stream to be exported or played back from the display screen of the multiple streams. For example, this can be achieved by opening multiple candidate video windows and playing them in chronological synchronization with the original video window, and allowing the user to select the best one from among them using a selection button or the like.

[0299] The relationship between the user's selection result or selection state and the information of the original video at that time may be used to determine image quality adjustment parameters, etc., for playing another video. Here, the relationship may be recorded, or may be learned by a learning model and then recorded.

[0300] When writing out the video, a function may be provided for selecting a spatial range for writing out the video using a rectangle or the like, and the image quality may be corrected only within the selected range before being re-encoded and written out.

[0301] For example, it would be good to have an export function for posting to social networking sites such as YouTube, and a function for uploading the original video and the processed video in a single file. Both videos can be streamed side by side, or a video stream in which the two videos are joined together in time sequence. In this case, it would be good to embed a mark indicating that each video is the "original video" and the "processed video" before exporting.

[0302] For images captured by a spherical or hemispherical camera, it is advisable to unfold the image, convert a portion of it into coordinates, and then perform processing on the displayed image. This reduces the processing load. Alternatively, for example, processing may be performed on a doughnut-shaped portion including the outermost periphery, while no processing or a different processing may be performed on the inner circular region of the doughnut.

[0303] Furthermore, the following aspects can also be provided.

[0304] In the information processing device, the screen includes an object, and the control unit receives an operation instruction for the object selected by a user, receives an instruction to display a processing result for the image corresponding to the operation instruction, and controls the information processing device to display a screen including the processing result for the image.

[0305] According to this aspect, any image processing corresponding to the manipulation of the object can be executed by the user's selection.

[0306] In the information processing device, the screen includes objects, which differ for each different processing result related to the image, and the control unit accepts an instruction to display the processing result related to the image corresponding to the object selected by a user, and controls the information processing device to display a screen including the processing result related to the image.

[0307] According to this aspect, any image processing corresponding to the object can be executed by user selection.

[0308] In the information processing device, when a first object and a second object are selected via the screen, the control unit accepts an instruction to display a first processing result for the image corresponding to the first object and an instruction to display a second processing result for the image corresponding to the second object, and controls the information processing device to display a screen including the first processing result for the image and the second processing result for the image.

[0309] According to this aspect, a plurality of arbitrary image processes can be executed simultaneously.

[0310] In the information processing device, the image processing related to the first object and the image processing related to the second object are performed on different areas of the image as targets of image processing.

[0311] According to this aspect, it is possible to simultaneously execute a plurality of arbitrary image processes on different regions.

[0312] In the information processing device, the object displayed on the screen is an object related to image processing relating to weather.

[0313] According to this aspect, it is possible to perform appropriate image processing in accordance with the weather.

[0314] In the information processing device, the control unit changes the object to be displayed on the screen based on the state of the image.

[0315] According to this aspect, it is possible to perform appropriate image processing based on the information within the image.

[0316] In the information processing device, the control unit controls to display, on the screen, an object that suggests image processing for the image based on road network information.

[0317] According to this aspect, it is possible to perform appropriate image processing based on road network information.

[0318] In the information processing device, the control unit controls the display of an object on the screen that suggests image processing for the image based on information from an acceleration sensor that detects the acceleration of a moving body having the shooting unit.

[0319] According to this aspect, it is possible to perform appropriate image processing based on information from the acceleration sensor of the moving body.

[0320] In the information processing device, the control unit controls to display an object suggesting image processing for the image on the screen based on position information of a moving body having the imaging unit.

[0321] According to this aspect, it is possible to perform appropriate image processing based on the position information of the moving body.

[0322] In the information processing device, the control unit controls the display of an object on the screen that suggests image processing for the image based on location information of a moving body having the photographing unit and weather information for the date and time the image was photographed.

[0323] According to this aspect, it is possible to perform appropriate image processing based on the location information of the moving body and the weather information for the date and time at that location.

[0324] In the information processing device, the control unit controls to display an object suggesting image processing for the image on the screen based on operation information of a moving object having the shooting unit.

[0325] According to this aspect, it is possible to perform appropriate image processing based on operation information of the moving object.

[0326] In the information processing device, the control unit controls to display, on the screen, an object suggesting image processing for the image based on a date and time when the image was captured.

[0327] According to this aspect, it is possible to perform appropriate image processing based on the date and time when the image was captured.

[0328] In the information processing device, the image processing on the image is image processing on a part of the image captured by the imaging unit.

[0329] According to this aspect, it is possible to perform image processing on a part of the captured image.

[0330] In the information processing device, the part of the image is a set area of ​​the entire image captured by the image capturing unit. The field is information processing equipment.

[0331] According to this aspect, it is possible to set an area where image processing is to be performed.

[0332] In the information processing device, the part of the images is an image taken during a set time period among images taken by the image taking unit.

[0333] According to this aspect, it is possible to perform appropriate image processing on a partial area of ​​an image captured during a set time period.

[0334] In the information processing device, the part of the image is an image captured within a range in which position information of a moving body having the image capturing unit is set.

[0335] According to this aspect, it is possible to perform appropriate image processing on a partial area of ​​an image captured by a moving object moving within a set position.

[0336] In the information processing device, the part of the image is an image showing the weather at the location of the moving body having the image capturing unit, the weather information being set for the date and time when the image was captured.

[0337] According to this aspect, it is possible to perform appropriate image processing on a partial area of ​​the captured image based on the weather at the location of the moving body.

[0338] In the information processing device, the part of the image is an image in which sensor information of a moving body having the imaging unit is within a set threshold.

[0339] According to this aspect, it is possible to perform appropriate image processing on a partial area of ​​an image captured based on sensor information of a moving body.

[0340] In the information processing device, the screen includes information on whether the range on which the image processing is to be performed has been selected automatically or manually.

[0341] According to this aspect, it is possible to identify whether the range of the executed image processing is set automatically or selected manually.

[0342] In the information processing device, the screen includes an image captured by the imaging unit and an image after one or more image processes.

[0343] According to this aspect, it is possible to display a comparison of before and after image processing.

[0344] In the information processing device, the control unit adjusts parameters of image processing based on learning information related to selection of the one or more images after image processing.

[0345] According to this aspect, the selected image processing result can be learned, and subsequent image processing can be performed more appropriately.

[0346] In the information processing device, the screen includes an image after image processing, and the control unit stores the image captured by the imaging unit in a storage area.

[0347] According to this aspect, the original image can be stored while the processed image is displayed on the screen.

[0348] In the information processing device, the screen includes information about what kind of image processing has been performed on the image captured by the imaging unit.

[0349] According to this aspect, it is possible to check what kind of image processing has been performed.

[0350] In the information processing device, the control unit stores information about what kind of image processing has been performed on the image captured by the imaging unit in a storage area as log information.

[0351] According to this aspect, it is possible to check what kind of image processing has been performed.

[0352] In the information processing device, the control unit converts the image after image processing into an image before image processing based on the log information and outputs the image to the screen.

[0353] According to this aspect, the image after image processing can be restored to the original image before image processing.

[0354] In the information processing device, the control unit changes an object included on the screen in accordance with an object detected from the image.

[0355] According to this aspect, the image processing method can be changed based on the object in the image.

[0356] In the information processing device, the screen includes an adjustment object that adjusts the degree of the image processing, and the control unit adjusts the degree of the image processing in response to an operation of the adjustment object.

[0357] According to this aspect, the degree of image processing such as brightness can be adjusted.

[0358] In the information processing device, the control unit stores the timing of the operation of the adjustment object and the timing of recording the image or the timing of playing back the image in association with each other.

[0359] According to this aspect, it becomes possible to know at what timing the degree of image processing has been adjusted.

[0360] In the information processing device, the screen includes a fixed instruction object that prohibits adjustment by the adjustment object, and the control unit prohibits adjustment by the adjustment object in response to an operation of the fixed instruction object.

[0361] According to this aspect, the degree of image processing can be fixed.

[0362] In the information processing device, the control unit adjusts the degree of image processing based on learning information about the degree of adjustment of the image processing in the past.

[0363] According to this aspect, the result of adjusting the degree of image processing can be learned, and subsequent image processing can be performed more appropriately.

[0364] In the information processing device, the screen includes an image output object, and the control unit deletes position information of the image and outputs the image in response to an operation of the image output object.

[0365] According to this aspect, it is possible to perform an operation to delete the location information linked to the image.

[0366] In the information processing device, when the image processed includes information that can identify an individual, the control unit requests execution of a masking process on the information that can identify an individual.

[0367] According to this aspect, it is possible to perform appropriate image processing so that information that can identify an individual is not included.

[0368] In the information processing device, the control unit executes image processing based on an operation instruction via the screen.

[0369] According to this aspect, the information processing device can execute image processing.

[0370] An image processing device capable of communicating with the information processing device, the image processing device having a control unit, the control unit executing image processing based on a request from the information processing device.

[0371] A program for causing a computer to function as a control unit of the information processing device.

[0372] Of course, this is not limiting.

[0373] The scope of the present invention is not limited to the structures explicitly described in the specification, but also includes combinations of various aspects of the present invention disclosed herein. The structures of the present invention that are sought to be patented are specified in the appended claims, but it is the intention of the present inventors to claim structures disclosed in this specification in the future, even if they are not currently specified in the claims.

[0374] The present invention is not limited to the configurations described in the above-described embodiments. The components of each of the above-described embodiments and variations may be arbitrarily selected and combined. Furthermore, any component of each embodiment or variation may be arbitrarily combined with any component described in the Summary of the Invention or any component embodying any component described in the Summary of the Invention. The present invention also intends to obtain rights to these configurations through amendments or divisional applications of the present application. Even if a description is made of "in the case of..." or "when...," it is not intended to describe a configuration limited to that case or time. Configurations that are not limited to those cases or times are also disclosed, and the present invention intends to obtain rights to them. Furthermore, any descriptions that specify an order are not limited to this order. Configurations in which some parts are deleted or the order is changed are also disclosed, and the present invention intends to obtain rights to them.

[0375] Furthermore, we intend to obtain rights to the overall design or partial design by converting the application to a design registration. The drawings depict the entire device in solid lines, but they also include partial designs claimed for parts of the device. For example, a partial design can be a partial design for a part of the device, or a part of that part. A partial design can be a part of the device, or a part of that part. We intend to obtain rights not only for the overall design, but also for partial designs in which any part of the solid line portion of the drawing is drawn as a broken line. Furthermore, all of the modules, components, and parts inside the device's casing, shown in the drawings, are subject to independent commerce, and we intend to similarly obtain rights by converting the application to a design registration. [Explanation of symbols]

[0376] 11: Control unit 22: Memory unit 23: Control unit 231: Acquisition unit 232: Reception unit 233: Correction unit 234: Release unit 235: Display control unit 32: Memory unit 33: Control unit 331: Acquisition unit 332: Reception unit 333: Recording unit 334: Correction unit 335: Display control unit 4: Playback screen 41: Correction area 42: Non-correction area 43: Additional information 5: Seek bar 51: Correction time IF1: Specific information IF2: Road network information IF3: Topographic information IF4: Operation information IM: Image data P1: First program P2: Second program P3: Third program P4: Fourth program PL: Parameter log PT: Parameter R: Region of interest 3000: Information processing system 3011: Control unit 3013: Display unit 3014: Audio output unit 3015: Imaging unit 3111: Processor 3100: Imaging device 3200: Imaging device 3400: Vehicle 3500: Recording medium 3700: Server 3710: Control unit 3711: Image processing unit 3712: Communication control unit 3713: Memory control unit 3720: Memory unit 3730: Communication unit 3800: Information display terminal 3810: Control unit 3811: Display control unit 3812: Communication control unit 3813: Memory control unit 3820: Memory unit 3830: Communication unit 3840: Input unit 3850: Output unit 3900: Network 4100: Playback window 4101: File button 4102: Display button 4103: Play button 4104: Tool button 4105: Information button 4110: Video display area 4120: Acceleration graph display area 4130: Map image display area 4131: Current location icon 4140: Fog compensation button 4141: HDR button 4142: Sharpness button 4143: Other processing buttons 4144: Partial processing button 4145: Stream display button 4146: Operation buttons 4150: Date and time display area 4160: Vehicle status display area 4170: Speed ​​graph display area 4180: Playlist display area 4200: Playback window 4210: Video display area 4240: Fog compensation button 4300: Playback window 4310: Video display area 4311: Fog compensation area 4312: Processing details display area 4313: Sharpness area 4314: Processing details display area 4340: Fog correction button 4342: Sharpness button 4344: Partial processing button 4400: Playback window 4440: Sunny correction button 4441: Cloud correction button 4442: Rain correction button 4443: Fog correction button 4500: Playback window 4510: Video display area 4540: Fog correction button 4600: Playback window 4610: Video display area 4630: Map image display area 4631: Current location icon 4640: Tunnel correction button 4700: Playback window 4720: Acceleration graph display area 4721: Acceleration display 4740: Acceleration correction button 4770: Speed ​​graph display area 4800: Playback window 4810: Video display area 4830: Map image display area 4831: Current location icon4832: Processing area 4840: Fog compensation button 4900: Playback window 4910: Video display area 4911: Weather information 4941: Rain compensation button 5000: Playback window 5010: Video display area 5030: Map image display area 5031: Current location icon 5040: Brake compensation button 5070: Speed ​​graph display area 5100: Playback window 5110: Video display area 5140: Night compensation button 5150: Date and time display area 5200: Playback window 5210: Video display area 5211: Fog compensation area 5240: Fog compensation button 5300: Playback window 5310: Video display area 5311: Night compensation area 5340: Night compensation button 5350: Date and time display area 5400: Playback window 5410: Video display area 5411: Fog compensation area 5430: Map image display area 5431: Current location icon 5432: Processing area 5440: Fog correction button 5500: Playback window 5510: Video display area 5511: Fog correction area 5512: Weather information 5540: Fog correction button 5541: Rain correction button 5600: Playback window 5610: Video display area 5611: Acceleration correction area 5620: Acceleration graph display area 5621: Acceleration display area 5640: Acceleration correction button 5700: Playback window 5710:Video display area 5711:Automatic fog compensation area 5712:Processing details display area 5713:Manual fog compensation area 5714:Processing details display area 5740:Fog compensation button 5745:Stream display button 5800:Playback window 5810:Video display area 5811:Original image display area 5812:Original display area 5813:Processing 1 result display area 5814:Processing 1 details display area 5815:Processing 2 result display area 5816:Processing 2 details display area 5817:Processing 3 result display area 5818:Processing 3 details display area 5846:Operation buttons 5900:Playback window 5910:Video display area 5911:Processing details display area 5940:Fog compensation button 6000:Playback window 6080:Playlist display area 6081:Playlist 6100: Playback window 6110: Video display area 6111: Detection information display area 6140: Fog compensation button 6200: Playback window 6290: Image processing adjustment area 6291: Hue adjustment object 6292: Brightness adjustment object 6293: Contrast adjustment object 6294: Saturation adjustment object 6295: Gamma adjustment object 6296: Sigma adjustment object 6297: Radius adjustment object 6298: Variation adjustment object 6300: Playback window 6390: Image processing adjustment area 6391: Image adjustment fixed object 6392: Sharpness adjustment fixed object 6393: Banding noise reduction adjustment fixed object 6394: Film grain adjustment fixed object 6400: Playback window 6401: File button 6410: Video display area 6500: Playback window 6510: Video display area 6511: Other vehicle 6512: License plate information 6540: Personal information processing button 6541: HDR button 6600: Information processing system 6800: Information processing terminal 6811: Display control unit 6812: Communication control unit 6813: Memory control unit 6814: Image processing unit

Claims

1. A system for correcting image data generated by an imaging device of a moving object, comprising: a function of acquiring specific information, which is information not directly extracted from the image data at the time the image data is generated, and which is information relating to the state or situation of the moving object; a function of executing a correction process on the image data based on the specific information; a function of displaying the image data after the correction process has been performed; Equipped with A system characterized by:

2. The function of executing the correction process has a function of executing the correction process on the image data while varying a correction range, which is a range to be corrected, based on the specific information.

2. The system of claim 1.

3. The specific information is at least one of the position, direction, speed, acceleration, time, time zone, weather, running condition, and pedal work of the moving object.

3. The system according to claim 1 or 2.

4. the specific information is road network information corresponding to the location of the mobile object, The function of executing the correction process includes a function of executing the correction process on the image data based on the road network information.

4. The system according to claim 1, wherein the first and second sensors are connected to each other.

5. the specific information is topographical information corresponding to the position of the moving object, The function of executing the correction process includes a function of executing the correction process on the image data based on the topographical information.

5. The system according to claim 1, wherein the first and second sensors are connected to a first power supply.

6. the specific information is the position and orientation of the moving object; The function of executing the correction process includes a function of executing a correction process on the image data based on the position of the sun estimated from the position and the orientation. The system according to any one of claims 1 to 5.

7. A program for causing a computer to realize the functions of the system according to any one of claims 1 to 6.

Images