Systems, information processing methods, programs, etc.

Abstract

To provide a system and the like which are superior to conventional ones, with which it is possible to apply appropriate image processing that is based on the state or condition of a moving body.SOLUTION: A system includes a control unit. The control unit is constituted so as to execute each of the following steps. In an acquisition step, specific information pertaining to the state or condition of a moving body is acquired. In a first correction step, first correction processing is executed, on the basis of the specific information, upon image data that is generated by an image capturing device that the moving body has during the time corresponding to the specific information. In a display control step, the image data upon which the first correction processing is executed is reproducibly displayed.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and the like. [Background technology]

[0002] A technique has been disclosed in which, when a user selects an indoor emphasis button, brightness correction within a position is performed on video data representing a dark image displayed in an image display area (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-129530 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology of Patent Document 1 only provides buttons on the screen for performing image processing on video data, and is unable to perform image processing as desired by users in a variety of different situations.

[0005] In view of the above circumstances, the present invention provides a system capable of performing appropriate image processing based on the state or situation of a moving object, thereby providing a system that is superior to conventional systems.

[0006] The object of the present invention is not limited to this, and the applicant intends to obtain rights for configurations that aim to obtain effects from parts of the configurations disclosed in this specification and drawings, etc., by filing a divisional application, amendment, etc. For example, this specification discloses problems in which the parts described as "can" and "is possible" in this specification are read as "is the problem." The problems are described as being independent, and the applicant intends to obtain rights for the configurations for solving each problem separately by filing a divisional application, amendment, etc. Even if the problems are implicitly understood from the description in the specification, the applicant intends to include part of the configurations described in this specification in the scope of the patent claim by amendment or divisional application. In addition, the applicant has disclosed a configuration that solves a problem that combines these independent problems, and intends to obtain rights for it. [Means for solving the problem]

[0007] (1) According to one aspect of the present invention, a system is provided. The system may include a control unit. The control unit may be configured to execute the following steps. In the acquisition step, specific information regarding a state or situation of the moving object may be acquired. In the first correction step, a first correction process may be executed on image data generated by an image capture device possessed by the moving object at a time corresponding to the specific information, based on the specific information. In the display control step, the image data on which the first correction process has been executed may be displayed in a reproducible manner.

[0008] In this way, the user can view an image reproduced from image data after the first correction process has been performed on image data generated by the image capture device at a time corresponding to the state or situation of the moving object. In particular, the correction process for the image data may be appropriate image processing. The image processing may be correction processing for correcting the image data to be easier for the user to view than image data generated by an image capture device possessed by the moving object. In particular, the correction process may be performed constantly. According to this aspect, since appropriate image processing can be performed based on the state or situation of the moving object, it is possible to realize a system capable of presenting image data that is easy for the user to view constantly, regardless of the situation or scene.

[0009] For example, it may be provided in the following aspects:

[0010] (2) In the above system, in the first correction step, a correction range, which is a range to be corrected, is varied based on the specific information, and the first correction process is performed on the image data.

[0011] According to this aspect, since it is not necessary to correct the entire range, the processing load can be reduced. Also, since the correction is applied to a preferred range, the user's attention can be directed to the preferred range regardless of the spatial range or the temporal range.

[0012] (3) In the system, the correction range is a region of interest set in a frame of the image data.

[0013] According to this embodiment, it is possible to narrow down the spatial area where it is originally desired to improve visibility and apply correction, so that the user's attention can be directed to the desired range on a frame-by-frame basis.

[0014] (4) In the system, the control unit is configured to further execute the following steps: a first reception step, receiving a first input from a user; and a release step, releasing the first correction processing on the image data based on the first input.

[0015] According to this aspect, when the correction process presented by the system is not necessary, the user can disable it by inputting their intention, thereby improving visibility, and an appropriate image can be presented depending on the situation or scene.

[0016] (5) In the above system, the control unit is configured to further execute the following steps: in a second reception step, receiving a second input from a user; and in a second correction step, performing a second correction process on the image data based on the second input.

[0017] According to this aspect, in addition to the correction processing presented by the device, it is possible to improve visibility by adding corrections at the user's discretion through input, and it is possible to present images that match the user's preferences depending on the situation or scene.

[0018] (6) In the above system, in the display control step, the image data is displayed so as to be reproducible in a manner in which the first correction process and the second correction process can be distinguished.

[0019] According to this embodiment, the user can understand the types of correction processing, which is user-friendly.

[0020] (7) In the above system, the specific information is at least one of the position, orientation, speed, acceleration, time, time zone, weather, driving conditions, and pedal work of the moving object.

[0021] According to this aspect, at least one of the specific moving body conditions, namely, position, speed, acceleration, running state, and pedal work, is taken into consideration, and more appropriate image processing can be performed based on this information.

[0022] (8) In the above system, the control unit is configured to further execute an acquisition step, wherein the specific information is the position of the moving body, the acquisition step acquires road network information, and the first correction step executes a first correction process on the image data based on the position and the road network information.

[0023] According to this aspect, it is possible to perform appropriate image processing according to the location and the road conditions at that time. For example, by performing appropriate image processing according to the traffic volume obtained from the road network information, it is possible to present an image that is easier for the user to view.

[0024] (9) In the above system, the control unit is configured to further execute an acquisition step, wherein the specific information is the position of the moving body, and in the acquisition step, topographical information is acquired, and in the first correction step, a first correction process is executed on the image data based on the position and the topographical information.

[0025] According to this aspect, it is possible to perform appropriate image processing according to the location and the terrain, environment, etc. For example, by performing different image processing on a downtown area and a forest, it is possible to present an image that is easier for the user to view.

[0026] (10) In the above system, the specific information is the position and orientation of the moving body, and in the first correction step, a first correction process is performed on the image data based on the position of the sun estimated from the position and orientation.

[0027] According to this embodiment, it is possible to perform appropriate image processing taking into account the position of the sun. For example, in a situation where the setting sun is shining in, the brightness of the area that is originally intended to be viewed tends to be reduced due to backlighting, so by performing image processing to correct this, it is possible to present an image that is easy for the user to view.

[0028] (11) In the above system, the first correction step includes a preparation step and a processing step, the control unit is configured to be able to call up a first program stored in advance and a second program different from the first program, the first program executes the preparation step, in which a parameter included in the image data is extracted and the value of the extracted parameter is passed to the second program, the second program executes the processing step and the display control step, in which the first correction process is performed on the parameter in the processing step to change its value, and in the display control step, the image data based on the value of the parameter changed in the processing step is displayed in a reproducible manner.

[0029] According to this aspect, a separate program specialized for correction processing can be used, which contributes to reducing calculation costs and improving image processing performance. In particular, by passing the parameter values ​​to the second program without duplicating the original image data, the second program suitable for image processing can read the parameter values ​​and reproduce image data that has been subjected to image processing according to the parameter values.

[0030] (12) In the above system, the parameters include at least one of hue, brightness, saturation, contrast, gamma, sharpness, playback start time, playback end time, and whether or not to loop.

[0031] According to this embodiment, the parameters such as hue, brightness, saturation, contrast, gamma, sharpness, playback start time, playback end time, and whether or not to loop are common parameters that image data has, so that a more general-purpose image processing program can be used as the second program, thereby realizing a system with high usability that is not limited to dedicated software.

[0032] (13) In the above system, in the preparation step, operation information regarding the operation of the second program is passed to the second program, and in the display control step, at least a portion of the functions of the second program is restricted based on the operation information, thereby displaying the image data in a reproducible manner.

[0033] According to this aspect, the second program can be used solely for correction and playback, and the user does not need to use unnecessary functions of the second program, thereby preventing or eliminating problems such as incorrect operation or the user finding the program difficult to use due to the large number of functions.

[0034] (14) In the above system, the first program is configured to execute the display control step, and in the display control step, if the second program is not installed, a screen recommending the installation of the second program is displayed.

[0035] According to this aspect, even if the second program is not installed, the user can be informed of a recommended second program, and as a result, the user can install the second program on his or her computer, and thereby reproduce the image data that has been appropriately subjected to the first correction process.

[0036] (15) In the above system, the second program is a program stored in an external device, and the control unit is configured to call the second program via a network.

[0037] According to this aspect, even if the second program is not installed, the first correction process can be performed via the second program. For example, even if the capacity of the computer being used is limited or a user does not have the authority to install a new program, the first correction process can be performed appropriately.

[0038] (16) A system comprising a control unit, the control unit being configured to execute the following steps: in a recording step, an image captured by a camera is recorded as image data; in a display control step, the image captured by the camera is displayed as a live view; and parameters of the recorded image data and the displayed image are set to be different.

[0039] According to this aspect, the image viewed as a live view and the image data to be recorded are different, so that both visibility and customizability can be ensured.

[0040] (17) In the above system, in the recording step, the image is recorded as the image data without correction, and in the display control step, the image is corrected and displayed as the live view.

[0041] According to this aspect, it is possible to improve the visibility of the image viewed as a live view, while preventing information loss due to processing of the image data to be recorded.

[0042] (18) In the above system, in the recording step, parameters of the displayed image are recorded in chronological order together with the image data as a parameter log.

[0043] According to this aspect, the visibility of the image viewed as a live view can be stored as a record, and this information can be used to reproduce the visibility later when playing back the image data.

[0044] (19) In the above system, the control unit is configured to further execute a third reception step, in which a third input from a user is received in the third reception step, and in the display control step, the image data and the parameter log are read out based on the third input, and the corrected image data is displayed in a reproducible manner.

[0045] According to such an embodiment, when the user wishes, it is possible to instantly play back video with high visibility like that of a live view, thereby realizing a system with high usability.

[0046] (20) In the above system, the control unit is configured to be able to call up a pre-stored third program and a fourth program different from the third program, and in the display control step, when the image data is played back using the fourth program, information encouraging playback using the third program is displayed.

[0047] According to such an embodiment, even if an unintended program is played back, it is possible to induce playback of a preferred program, and appropriate information can be provided to the user.

[0048] (21) In the above system, the information encouraging playback is displayed together with a playback screen of the image data.

[0049] According to this embodiment, it is possible to provide the user with appropriate information that the current appearance is undesirable, but that there is no problem with the image data itself.

[0050] (22) An information processing method comprising each of the steps of the system.

[0051] According to such a method, the functionality of the above-described system can be realized and provided in the form of a method.

[0052] (23) A program that causes a computer to function as a control unit of the system.

[0053] According to this embodiment, the functions of the above-mentioned system can be realized and provided in the form of a program. Of course, this is not the case.

[0054] The inventions shown in (1) to (23) above can be arbitrarily combined. For example, at least a part of the configuration of at least one of the inventions after (2) may be added to all or a part of the configuration of the invention shown in (1). In particular, it is preferable to add at least a part of the configuration of at least one of the inventions after (2) to the invention shown in (1). In addition, any configuration may be extracted from the inventions shown in (1) to (23) and the extracted configurations may be combined. The applicant of this application intends to obtain rights to the inventions including these configurations. In addition, even if there is a description such as "in the case of" or "when", it is not intended to describe the configuration as being limited to that case or time. These are examples of better configurations, and the applicant intends to obtain rights to configurations other than these cases or times. In addition, the parts described in order are not limited to this order. Configurations in which some parts have been deleted or the order has been changed are also disclosed, and the applicant intends to obtain rights. Effect of the Invention

[0055] According to the present invention, appropriate image processing can be performed based on the state or situation of a moving object, thereby realizing a system that can consistently present image data that is easy for the user to view, regardless of the situation or scene.

[0056] The effects of the invention of this application are not limited to this, and effects achieved from the configuration disclosed in the specification and drawings are also disclosed, and the applicant intends to obtain rights to the configuration achieving said effects through divisional applications, amendments, etc. For example, in this specification, statements such as "can" and "is possible" are statements that clearly indicate the effects achieved, and there are also parts that show effects even without statements such as "can" and "is possible." Furthermore, there are effects that can be understood from the configuration even without such statements. [Brief description of the drawings]

[0057] [Figure 1]FIG. 1 is a diagram illustrating an example of a system configuration of an information processing system according to the first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating the configuration of the vehicle according to the first embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of the external configuration of the imaging device. [Figure 4] FIG. 4 is a diagram showing the configuration of the bracket. [Diagram 5] FIG. 5 is a block diagram showing the electrical configuration of the imaging device. [Figure 6] FIG. 6 is a diagram illustrating an example of a hardware configuration of the server according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a hardware configuration of the information display terminal according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a functional configuration of the server according to the first embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a functional configuration of the information display terminal according to the first embodiment. [Figure 10] FIG. 10 is an activity diagram showing the flow of information processing of the server according to the first embodiment. [Figure 11] FIG. 11 is an activity diagram showing the flow of information processing of the information display terminal according to the first embodiment. [Figure 12] FIG. 12 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit according to the first embodiment. [Figure 13] FIG. 13 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit according to the first embodiment. [Figure 14] FIG. 14 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit of the first modified example of the first embodiment. [Figure 15] FIG. 15 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit in the second modification of the first embodiment. [Figure 16] FIG. 16 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit in the third modification of the first embodiment. [Figure 17]FIG. 17 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit in the fourth modified example of the first embodiment. [Figure 18] FIG. 18 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit in the fifth modified example of the first embodiment. [Figure 19] FIG. 19 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit in the sixth modification of the first embodiment. [Figure 20] FIG. 20 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit in the seventh modification of the first embodiment. [Figure 21] FIG. 21 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit in the eighth modification of the first embodiment. [Figure 22] FIG. 22 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit in the ninth modification of the first embodiment. [Figure 23] FIG. 23 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the tenth modification of the first embodiment. [Figure 24] FIG. 24 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the eleventh modification of the first embodiment. [Diagram 25] FIG. 25 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the twelfth modification of the first embodiment. [Figure 26] FIG. 26 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the thirteenth modification of the first embodiment. [Figure 27] FIG. 27 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the fourteenth modification of the first embodiment. [Figure 28] FIG. 28 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the fifteenth modification of the first embodiment. [Figure 29] FIG. 29 is a diagram showing an example of a screen displayed by the display control unit in the sixteenth modification of the first embodiment. [Diagram 30] FIG. 30 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit in the seventeenth modification of the first embodiment. [Diagram 31] FIG. 31 is a diagram showing an example of a screen displayed by the display control unit in the eighteenth modification of the first embodiment. [Diagram 32] FIG. 32 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit in the nineteenth modification of the first embodiment. [Diagram 33] FIG. 33 is a diagram showing an example of a screen displayed by the display control unit in the twentieth modification of the first embodiment. [Diagram 34] FIG. 34 is a diagram showing an example of a screen displayed by the display control unit in the twenty-first modification of the first embodiment. [Diagram 35] FIG. 35 is a diagram showing an example of a screen displayed by the display control unit in the twenty-second modification of the first embodiment. [Diagram 36] FIG. 36 is a diagram showing an example of a screen displayed by the display control unit in the twenty-third modification of the first embodiment. [Figure 37] FIG. 37 is a schematic diagram showing a system configuration of an information processing system according to a modified example of the second embodiment. [Figure 38] FIG. 38 is a schematic diagram illustrating a functional configuration of an information processing terminal according to a modification of the second embodiment. [Figure 39] FIG. 39 is a functional block diagram realized by the storage unit 22 and the control unit 23. As shown in FIG. [Diagram 40] FIG. 40 is an activity diagram showing the flow of information processing according to the fourth embodiment. [Diagram 41] FIG. 41 is a schematic diagram showing an example of a spatial correction range of the first correction process. [Diagram 42] FIG. 42 is a schematic diagram showing an example of a time-wise correction range of the first correction process. [Diagram 43] FIG. 43 is a schematic diagram showing additional information 43 added to the reproduced image 4 in the case where the first correction process and the second correction process are executed. [Diagram 44] FIG. 44 is an activity diagram showing the flow of information processing according to the third modification of the fourth embodiment. [Diagram 45] FIG. 45 is a functional block diagram realized by the control unit 33. As shown in FIG. [Figure 46]FIG. 46 is an activity diagram showing the flow of information processing according to the fifth embodiment. [Figure 47] FIG. 47 is a schematic diagram showing the state of image data IM displayed as a live view and image data IM recorded so as to be playable later. [Figure 48] FIG. 48 is a schematic diagram showing a state in which additional information 43, which encourages playback of the recommended third program P3, is superimposed on the playback screen 4. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0058] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment shown below is one embodiment of the present invention, and the contents of the present invention should not be interpreted as being limited based on the following description.

[0059] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0060] In addition, in this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, in this embodiment, various information is handled, and this information is represented, for example, by physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit collection consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation can be performed on the circuit in the broad sense.

[0061] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0062] [Embodiment 1] [System configuration of information processing system] An example of the information processing system 3000 will be described below. FIG. 1 is a diagram showing an example of a system configuration of the information processing system 3000 according to the first embodiment. As shown in FIG. 1, the information processing system 3000 includes a vehicle 3400, a server 3700, a plurality of information display terminals 3800A, 3800B, and 3800C, and a network 3900. The vehicle 3400 is configured to be able to communicate with the server 3700 via the network 3900. More specifically, the vehicle 3400 transmits information held by the vehicle 3400 to the server 3700 via the network 3900. In addition, the plurality of information display terminals 3800A, 3800B, and 3800C are configured to be able to communicate with the server 3700 via the network 3900. More specifically, the plurality of information display terminals 3800A, 3800B, and 3800C acquire information in the server 3700 via the network 3900. It should be noted that the information display terminals 3800A, 3800B, and 3800C may be any device that has a role as a UI (User Interface), and specific examples include a personal computer, a smartphone, and a tablet.

[0063] [Overall configuration of vehicle 3400] FIG. 2 is a diagram for explaining the configuration of the vehicle 3400 of the first embodiment. FIG. 2 shows a schematic diagram of the vehicle 3400 seen from the side. The vehicle 3400 has an image capturing device 3100 as a first image capturing device and an image capturing device 3200 as a second image capturing device. The vehicle 3400 is, for example, an internal combustion engine vehicle having an engine as a drive source, a hybrid vehicle having an engine and a running motor as drive sources, an electric vehicle having a running motor as a drive source, etc. The vehicle 3400 is, for example, a four-wheeled automobile, but may be any vehicle in which the image capturing device 3100 and the image capturing device 3200 can be placed. The vehicle may be, for example, a large transport vehicle with four or more wheels such as an automobile, a bus, or a truck, a two-wheeled vehicle such as a motorcycle or a bicycle, or other vehicles. The vehicle may be, for example, a transportation vehicle such as a train, a monorail, or a linear motor car.

[0064] The photographing device 3100 is a front camera disposed on the front side of the vehicle 3400. For example, the photographing device 3100 is attached to a predetermined position on the front side of the cabin of the vehicle 3400, and photographs the front of the vehicle 3400 through the windshield as the photographing direction. The photographing device 3100 is a drive recorder. More specifically, the photographing device 3100 has a function of photographing, a function of recording image data showing the photographed image, and a function of recording image data acquired from the photographing device 200.

[0065] The photographing device 3200 is a rear camera disposed on the rear side of the vehicle 3400. For example, the photographing device 3200 is attached to a predetermined position on the rear side of the cabin of the vehicle 3400, and photographs the rear of the vehicle through the rear window. The photographing device 3200 has a function of photographing and a function of outputting image data indicating the photographed image to the photographing device 3100.

[0066] The photographing device 3100 and the photographing device 3200 are connected via a cable 3300. The cable 3300 is a wired communication path connecting the photographing device 3100 and the photographing device 3200. The cable 3300 has, for example, a power line for supplying operating power from the photographing device 3100 to the photographing device 3200, and a signal line for transmitting various signals between the photographing device 3100 and the photographing device 3200. The photographing device 3200 operates by receiving power supplied from the photographing device 3100 via the cable 3300. Note that the photographing device 3100 and the photographing device 3200 may be connected by a wireless communication path of Wi-Fi (registered trademark), Bluetooth (registered trademark), or other standards, instead of a wired communication path. The photographing device 3100 may be used without being connected to the photographing device 200 by communication.

[0067] [External configuration of the imaging device 3100] FIG. 3 is a diagram showing an example of the external configuration of the imaging device 3100. FIG. 3(A) is a diagram showing the imaging device 3100 as viewed from the upper right diagonal direction on the front side. FIG. 3(B) is a diagram showing the imaging device 3100 as viewed from the upper right diagonal direction on the rear side. The imaging device 3100 has a housing 3101. The housing 3101 is a rectangular parallelepiped shape that is longer in the left-right direction than in the up-down direction and has a relatively small thickness. The housing 3101 has a top surface 31011 that faces upward when attached to the vehicle 3400, a first side surface 31012, a second side surface 31013, a third side surface 31014 located on the opposite side to the second side surface 31013, and a fourth side surface 31015 located on the opposite side to the first side surface 31012.

[0068] The upper surface 31011 is provided with a joint rail 3102 and a camera jack 3191. The joint rail 3102 is detachably attached to a bracket (for example, a bracket 3600 described later) for mounting the image capturing device 3100 to a predetermined mounting position of the vehicle 3400. The mounting position may be, for example, the windshield of the vehicle 3400 (for example, near the upper end of the windshield), or the rearview mirror or the ceiling of the vehicle 3400. When the image capturing device 3100 is mounted on the vehicle 3400, the first side surface 31012 faces the front side of the vehicle 3400. At this time, the second side surface 31013 faces the right side as viewed from the rear side of the vehicle 3400. The third side surface 31014 faces the left side as viewed from the rear side of the vehicle 3400. The fourth side surface 31015 faces the rear side of the vehicle 3400.

[0069] The camera jack 3191 is a terminal to which one end of the cable 3300 is connected. The camera jack 3191 is compatible with, for example, the USB Type C standard, and may be a terminal through which the photographing device 3100 communicates with the photographing device 3200 according to the Ethernet standard.

[0070] The first side surface 31012 is provided with a photographing lens 3151, a sound output hole 3103, and a microphone hole 3104. The photographing lens 3151 is a lens for collecting light that is included in a photographing section (a photographing section 3015 described later) included in the photographing device 3100. The sound output hole 3103 is provided above the photographing lens 3151, and is a hole that transmits sound output from a sound output section (a sound output section 3014 described later) included in the photographing device 3100 from inside to outside the housing 3101. The microphone hole 3104 is provided below the photographing lens 3151, and is a hole that transmits sound from the outside from the outside to the inside of the housing 3101. The sound that has transmitted into the inside of the housing 3101 is input to a microphone (a microphone 3121 described later) included in the photographing device 3100.

[0071] An event record button 3122 is provided on the second side surface 31013. The event record button 3122 is an operation means for instructing the start or end of recording (video recording) of an image captured by the image capturing unit 3015. When the event record button 3122 is operated by a user while event recording is not being performed, the image capturing device 3100 starts event recording. Event recording will be described in detail later. The event record button 3122 is provided on the second side surface 31013 facing the driver's seat so that it is easy for the driver of a right-hand drive vehicle 3400 to operate.

[0072] A terminal 3192 and a storage medium insertion port 3181 are provided on the third side surface 31014. The terminal 3192 is a terminal for receiving power supply from an external device. The terminal 3192 is, for example, a DC jack. A connector on one end of a power cord (for example, a cigarette lighter plug cord) is connected to the terminal 3192. The connector on the other end of the power cord is connected to, for example, a power supply terminal (for example, a cigarette lighter socket) provided on the vehicle 3400 side.

[0073] The terminal 3192 may be connected to an OBDII adapter that can be connected to an OBDII ("II" is the Roman numeral for "2") connector of the vehicle 3400. The OBDII connector is also called a fault diagnosis connector, and is a terminal that is connected to an ECU (Engine Control Unit) of the vehicle and outputs various types of vehicle information at predetermined intervals (e.g., every 0.5 seconds). By connecting the terminal 3192 to the OBDII connector using the OBDII adapter, the image capturing device 3100 can receive power for operation and acquire vehicle information.

[0074] The vehicle information is information related to the state of the vehicle 3400. The vehicle information may be, for example, at least one of the following: the speed of the vehicle 3400 (vehicle speed), engine speed, engine load factor, throttle degree, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air flow rate (MAF), injection open time, engine coolant temperature (coolant temperature), air temperature taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of remaining fuel in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on / off)), brake opening, steering wheel rotation angle, gear position, and door open / closed state information.

[0075] The storage medium insertion slot 3181 is an insertion slot for inserting the storage medium 3500 as an external storage means into the inside of the image capturing device 3100. The storage medium 3500 is a storage medium in which images captured by the image capturing device 3100 or the image capturing device 3200 are recorded, such as an SD card. The SD card may have any shape, such as an SD memory card, a miniSD card, or a microSD card. The storage medium 3500 may further store a viewer (e.g., a dedicated viewer) program for playing back the stored images on an information display terminal such as a personal computer.

[0076] An operation unit 3123, a display surface 3131, and a light emitting unit 3021 are provided on the fourth side surface 31015. The operation unit 3123 has a first button 31231, a second button 31232, a third button 31233, and a fourth button 31234. The first button 31231, the second button 31232, the third button 31233, and the fourth button 31234 are arranged vertically along one side of the right side of the display surface 3131. The functions assigned to each of these buttons include, for example, the following functions.

[0077] The first button 31231 functions as a button for switching images when pressed for a long time, and functions as a button for instructing formatting of the storage medium 3500 when pressed for a short time. The image displayed on the display surface 3131 is, for example, one or both of an image currently being photographed by the photographing device 3100 and an image currently being photographed by the photographing device 3200. Formatting the storage medium 3500 is to initialize the storage medium 3500, and is understood as, for example, at least one of erasing data such as images stored in the storage medium 3500, writing setting information indicating the contents of operation settings to the storage medium 3500 so that the photographing device 3100 can use the storage medium 3500 (for example, so that images can be recorded and read), and putting the storage medium 3500 into a specific file state.

[0078] The second button 31232 is a button for displaying a selection screen for selecting an image to be played back by the image capturing device 3100. The third button 31233 is a button for displaying a menu related to settings of the image capturing device 3100 and the image capturing device 3200. The fourth button 31234 is a button for instructing the start and stop of image recording. For example, when the fourth button 31234 is pressed briefly during recording by a continuous recording function described later, the recording is paused. When the fourth button 31234 is pressed briefly during the pause, the recording of images by the continuous recording function is resumed. When the fourth button 31234 is pressed for a long time, the frame rate at which images are recorded can be changed.

[0079] The display surface 3131 is an area where an image is displayed by a display unit (display unit 3013 described later) of the imaging device 3100. The display surface 3131 is, for example, a rectangular or square area. A touch sensor 3124 for detecting a touch operation by a user is provided on top of the display surface 3131.

[0080] The light emitting portion 3021 is provided above the first button 31231, and emits light in a predetermined color.

[0081] The image capturing device 3200 may also have a function as a drive recorder, and may have, for example, the same configuration as the image capturing device 3100. In addition, as an image capturing device connected to the image capturing device 3100 through communication, one or more image capturing devices capturing images in other directions may be used instead of or in addition to the image capturing device 3200. The other directions include the right rear diagonal direction, the left rear diagonal direction, the vehicle width direction (sideways), and the like.

[0082] [Bracket 3600 Configuration] Fig. 4 is a diagram showing the configuration of the bracket 3600. Fig. 4(A) is a diagram showing the bracket 3600 as viewed from the front, diagonally upward right direction. Fig. 4(B) is a diagram showing the bracket 3600 as viewed from the side. Fig. 4(C) is a diagram showing the bracket 3600 as viewed from the front, diagonally downward right direction. The bracket 3600 is an example of a mounting member for mounting the image capturing device 3100 to the vehicle 3400.

[0083] Bracket 3600 is a mounting member that uses a ball joint mechanism. In bracket 3600, flat base portion 3610 has mounting surface 3611 to be attached to the windshield of vehicle 3400. Base portion 3610 is inclined at a predetermined angle with respect to the support of ball stud 3620. An adhesive member such as double-sided tape is attached to mounting surface 3611, and bracket 3600 is attached to the windshield of vehicle 3400 via the adhesive member.

[0084] The ball stud 3620 is a portion that stands up from the surface of the base portion 3610 opposite to the mounting surface 3611. The ball portion 3621 of the ball stud 3620 is attached to the socket portion 3630. The nut 3640 is detachably attached to the periphery of the socket portion 3630. A thread groove is formed on the outer periphery of the socket portion 3630. The nut 3640 that fits into the thread groove is attached to the outer periphery of the socket portion 3630. With the ball portion 3621 of the ball stud 3620 attached in the socket portion 3630, before the nut 3640 is tightened, the socket portion 3630 can rotate in a desired direction along the circumferential surface of the ball portion 3621 to change the attitude and position of the base portion 3650. When the nut 3640 is tightened, the attitude and position of the base portion 3650 are fixed.

[0085] The base portion 3650 is a part formed as a pair with the socket portion 3630 and is used for mounting on the imaging device 3100. The base portion 3650 has a pair of guide rails 3651 on its underside. The pair of guide rails 3651 are configured to be slidable along the joint rails 3102 of the imaging device 3100. A claw-shaped tip portion 3652 is provided at the tip portion of the base portion 3650. The tip portion 3652 is hooked onto the joint rails 3102 near the end portion on the front side of the imaging device 3100, whereby the bracket 3600 is mounted on the imaging device 3100.

[0086] [Electrical configuration of the imaging device 3100] FIG. 5 is a block diagram showing an electrical configuration of the photographing device 3100. The control unit 3011 controls each unit of the photographing device 3100. The control unit 3011 is, for example, a computer including a processor 3111 and a memory 3112. The processor 3111 has, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA). The memory 3112 is, for example, a main storage device having a random access memory (RAM) and a read only memory (ROM). The processor 3111 temporarily stores a program read from the ROM of the memory 3112 in the RAM. The RAM of the memory 3112 provides a working area for the processor 3111. The processor 3111 performs various controls by performing arithmetic processing while temporarily storing data generated during the execution of the program in the RAM. The control unit 3011 further includes a clock unit 3113 that measures time. The timekeeping unit 3113 is, for example, a real-time clock. The timekeeping unit 3113 may be mounted on a motherboard of the processor 3111, or may be attached externally to the processor 3111.

[0087] The input unit 3012 accepts input of information from a user. The input unit 3012 has, for example, the above-mentioned microphone 3121, event record button 3122, operation unit 3123, and touch sensor 3124. The microphone 3121 converts sound incident via the microphone hole 3104 or the like into an electric signal. The microphone 3121 is, for example, a condenser microphone. The touch sensor 3124 detects a position touched by the user on the display surface 3131. The touch sensor 3124 is, for example, of a capacitive type.

[0088] The display unit 3013 displays an image on a display surface 3131. The display unit 3013 is, for example, a liquid crystal display (LCD).

[0089] The audio output unit 3014 outputs audio. Examples of the audio include notification sounds, background music, audio messages, etc. The audio output unit 3014 includes, for example, an audio processing circuit and a speaker.

[0090] The photographing unit 3015 photographs and generates image data obtained by photographing. The photographing unit 3015 has, for example, a photographing lens 3151 and a photographing element that photographs light collected by the photographing lens 3151. The photographing element is, for example, a CMOS (Complementary MOS) or a CCD (Charge Coupled Device). The photographing unit 3015 generates a color (multi-color) image consisting of, for example, red (R), green (G), and blue (B) color components.

[0091] The communication unit 3016 communicates with an external device. The communication unit 3016 has a communication circuit for wirelessly communicating with an external device by, for example, Wi-Fi (registered trademark), Bluetooth (registered trademark), or other wireless LAN (Local Area Network) communication or short-distance wireless communication. The communication unit 3016 may have a communication circuit for performing communication in compliance with standards of mobile communication systems such as LTE (Long Term Evolution), 4G, and 5G.

[0092] The sensor unit 3017 has various sensors. For example, the sensor unit 3017 has at least one of an acceleration sensor, a gyro sensor, an air pressure sensor, and an illuminance sensor. The acceleration sensor is, for example, a three-axis acceleration sensor that measures the acceleration of the vehicle in the forward / backward, left / right, and up / down directions. The gyro sensor is a sensor that measures the inclination of the image capturing device 3100. The acceleration sensor and the gyro sensor may be used to estimate the position of the vehicle 3400 by autonomous navigation, for example, when a signal from a GNSS satellite cannot be received. The air pressure sensor measures air pressure. The air pressure sensor is used, for example, to detect the difference in elevation and determine whether the vehicle is on a highway or an ordinary road. The illuminance sensor is a sensor that detects illuminance indicating the brightness inside the vehicle cabin, which is the periphery of the image capturing device 3100. The illuminance sensor is used, for example, to adjust the brightness of the display of the display unit 3013.

[0093] The reader / writer 3018 functions as a medium holding unit that holds the storage medium 3500 inserted into the inside of the imaging device 3100 through the storage medium insertion port 3181. The reader / writer 3018 writes data to the storage medium 3500 and reads data from the storage medium 3500. The reader / writer 3018 may hold only one storage medium 3500, but may also be configured to be capable of holding two or more storage media 3500 simultaneously.

[0094] The terminal unit 3019 has a terminal for electrically connecting to an external device. The terminal unit 3019 has the above-mentioned camera jack 3191 and terminal 3192. An external battery may be used as a device connected to the terminal unit 3019 so that the image capturing device 3100 and the image capturing device 3200 can operate even without power supply from the vehicle 3400 side. The device connected to the terminal unit 3019 may be, for example, a device having a function of supporting safe driving of a user. Examples of such devices include a device having a function of capturing an image of a driver (e.g., a face) and detecting and notifying the driver's state, such as distracted driving and drowsy driving, and a device having a function of detecting and notifying an obstacle around the vehicle 3400 (e.g., a device used for vehicle detection for a forward vehicle collision warning system (FCWS)). The device connected to the terminal unit 3019 may also be an in-vehicle device such as a radar detector, a laser detector, a car navigation device, or a display device.

[0095] The position information acquisition unit 3020 acquires position information indicating the position of the image capturing device 3100 (more specifically, the current position). The position of the image capturing device 3100 can be regarded as the same as the position of the vehicle 3400 in which the image capturing device 3100 is placed, the position of the image capturing device 3200 placed in the vehicle 3400, and the positions of the driver and other people riding in the vehicle 3400. The position information acquisition unit 3020 acquires position information (latitude information and longitude information) of the image capturing device 3100 based on a signal from a Global Positioning System (GPS), which is one of the Global Navigation Satellite Systems (GNSS). The position information acquisition unit 3020 may also use Michibiki as a Quasi-Zenith Satellite System (QZSS).

[0096] The light emitting section 3021 emits light of a predetermined color and includes, for example, a light emitting diode.

[0097] The power supply control unit 3022 controls the supply of power to each unit of the photographing device 3100 and the photographing device 3200. The power supply control unit 3022 has, for example, a power switch and a power supply control circuit. The power supply control unit 3022 supplies power supplied from the vehicle 3400 via the terminal unit 3019 to each unit of the photographing device 3100 and the photographing device 3200. The power supply control unit 3022 may further have a secondary battery, a button battery, or an electric double layer capacitor (also called a supercapacitor) as a power storage means.

[0098] The photographing device 3100 may further include an auxiliary storage device such as a flash memory (e.g., eMMC, SSD) as an internal storage means. As the auxiliary storage device, various storage media such as an optical storage medium, a magnetic storage medium, and a semiconductor storage medium can be used.

[0099] [Image recording function of the imaging device 3100] The photographing device 3100 has one or more of the following functions as an image recording function. The image recording function is a function for recording an image photographed by the photographing device 3100 as image data in a predetermined file format. The image data may be image data in a moving picture format, such as an MPEG (Moving Picture Experts Group) format (e.g., MPEG2, MPEG4), AVI, MOV, WMV, etc. The image data includes information on the photographing date and time, the photographing location, etc. as Exif (Exchangeable image file format) information.

[0100] [Continuous recording function] The continuous recording function (also called the continuous video recording function) is a function that continuously (i.e., constantly) records images captured by one or both of the image capturing devices 3100 and 3200 while the image capturing device 3100 is in operation. When the control unit 11 is executing the continuous recording function, it stores video data captured from the start to the stop of the drive source of the vehicle 3400. The control unit 3011 detects the start of the drive source, for example, by the power supply (e.g., accessory power supply) of the vehicle 3400 being turned on. The control unit 3011 detects the stop of the drive source, for example, by the power supply of the vehicle 3400 being turned off.

[0101] [Event recording function] The event recording function is a function for recording an image captured by one or both of the image capturing devices 3100 and 3200 in response to the occurrence of a specific event. The event is an occurrence for which an image captured by the image capturing device 3100 or the image capturing device 3200 should be recorded, such as when the user performs an operation such as abrupt steering or sudden braking while the vehicle 3400 is traveling, or when the vehicle 3400 collides with another object. The control unit 3011 determines that an event has occurred, for example, based on the acceleration measured by the acceleration sensor of the sensor unit 3017. More specifically, the control unit 3011 determines that an event has occurred when the measured acceleration is equal to or greater than a predetermined threshold value or shows a predetermined change over time. The conditions for determining the occurrence of an event are not limited to the above. The control unit 3011 may determine that an event has occurred based on the vehicle information, for example, when the state of the vehicle 3400, such as the vehicle speed or steering state, satisfies a predetermined condition. The control unit 3011 may analyze an image captured by the image capturing unit 3015 or the image capturing device 3200, and determine that an event has occurred when dangerous driving (e.g., tailgating, approaching, or abnormal approach) of the vehicle 3400 or another vehicle is detected. The control unit 3011 also determines that an event has occurred when the event record button 3122 is operated.

[0102] When the control unit 3011 determines that an event has occurred, it records images captured during a predetermined period (hereinafter referred to as "event recording period") before and after the occurrence of the event in the storage medium 3500. For example, the control unit 3011 may temporarily record images captured by the image capturing unit 3015 and the image capturing device 3200 in the memory 3112 (for example, RAM), and when it determines that an event has occurred, it may record images during the event recording period read from the memory 3112 in the storage medium 3500. For example, the control unit 3011 may store images for a total of 40 seconds, including 20 seconds before the occurrence of the event and 20 seconds after the occurrence of the event, as one file. The event recording period is an example, and may be different depending on the type of event, or may be changeable by the user. The control unit 3011 may record images consisting of multiple files in the storage medium 3500 for the occurrence of one event. The control unit 3011 may further record in the storage medium 3500 values ​​measured by the sensor unit 3017 during the event recording period (e.g., acceleration in each of the three axial directions) and position information acquired by the position information acquisition unit 3020 in association with the image.

[0103] [Parking monitoring function] The parking monitoring function is a function for recording images captured by one or both of the image capturing devices 3100 and 3200 while the vehicle 3400 is parked. The parking monitoring function is a function for monitoring the inside of the parked vehicle 3400 or the outside around the vehicle 3400. When the power supply of the vehicle 3400 is off, the control unit 3011 receives power from an external battery and records images in the storage medium 3500. Regarding whether the vehicle 3400 is parked, the control unit 3011 determines whether the vehicle 3400 is parked based on one or more of the following: the power supply of the vehicle 3400 is off; the power source of the vehicle 3400 is off; the power supply from the external battery has started; the vehicle speed is 0 km / h or a predetermined speed or less; and the location information acquired by the location information acquisition unit 3020 is predetermined location information (for example, location information of a home, a workplace, or a parking lot).

[0104] The parking monitoring function may have a time lapse mode and a moving object detection mode. More specifically, when the time lapse mode is selected by the user, the control unit 3011 records images at a frame rate lower than that of other image recording functions such as the continuous recording function and the event recording function. For example, the frame rate of the other image recording functions is 20 to 30 frames / second, whereas the frame rate of the time lapse mode is 1 frame / second. The moving object detection mode is a mode for recording images in response to detection of a moving object. More specifically, when the control unit 3011 detects a moving object from a change in the images captured by the image capturing device 3100 and the image capturing device 3200 in the case where the moving object detection mode is selected by the user, the control unit 3011 records images captured in a predetermined period before and after the detection in the storage medium 3500. The frame rate may be the same as that of the continuous recording function and the event recording function.

[0105] Note that a photographing device that photographs a celestial spherical image such as a full celestial sphere or a hemisphere may be used as the photographing device 3100 or 3200. Also, a photographing device that does not have a display unit 3013 may be used as the photographing device 3100. Also, the housing of the photographing device 3100 does not have to be rectangular parallelepiped-shaped, and may be, for example, cylindrical, or the like.

[0106] [Server 3700 hardware configuration] The hardware configuration of the server 3700 will be described below with reference to FIG.

[0107] 6 is a diagram showing an example of a hardware configuration of a server 3700 according to the first embodiment. In the first embodiment, the server 3700 serves as an image processing device. The server 3700 includes, as its hardware configuration, a control unit 3710, a storage unit 3720, and a communication unit 3730. The control unit 3710, the storage unit 3720, and the communication unit 3730 are electrically connected to each other via a communication bus.

[0108] The control unit 3710 is configured with a microcomputer including a CPU and a semiconductor memory, and controls the overall operation of the server 3700. In particular, the control unit 3710 realizes various functions related to the server 3700 by reading and executing a predetermined program stored in the storage unit 3720. That is, the control unit 3710 executes processing based on a predetermined program stored in the storage unit 3720, thereby realizing the functional configuration shown in FIG. 8 described later and the processing of the activity diagram shown in FIG. 10 described later. Note that the control unit 3710 is not limited to being single. The server 3700 may be configured to include multiple control units 3710.

[0109] The storage unit 3720 is configured with any one of a hard disk drive (HDD), a random access memory (RAM), a read only memory (ROM), and a solid state drive (SSD), or an arbitrary combination of these. The storage unit 3720 stores image information sent from the vehicle 3400, information other than images sent from the vehicle 3400, various data related to various programs including a part of the image processing program according to the first embodiment, data used when the control unit 3710 executes processing based on the program, and the like.

[0110] The communication unit 3730 is a so-called network interface that is communicatively connected to the vehicle 3400 and each of the information display terminals 3800A, 3800B, and 3800C via a network 3900 such as the Internet or a LAN. In the first embodiment, the communication unit 3730 of the server 3700 that performs image processing is an image processing device that can communicate with the communication unit 3830 of the information display terminal 3800. Main information received by the server 3700 via the communication unit 3730 includes image information captured by the imaging device 3100 of the vehicle 3400, image information captured by the imaging device 3200, information on the status of the vehicle 3400 (the speed (vehicle speed) of the vehicle 3400, acceleration, engine RPM, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air flow rate (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of remaining fuel in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on / off)), brake opening, steering wheel rotation steering angle, gear position, and door open / closed status information, position information, etc.), date and time information, air pressure, illuminance, image processing instructions, image download request instructions, request instructions for information other than images, etc. The main information transmitted by the server 3700 via the communication unit 3730 includes image information before image processing is performed, image information after image processing is performed, information related to the image processing method, information related to the state of the vehicle 3400 (the speed of the vehicle 3400 (vehicle speed), acceleration, engine RPM, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air flow rate (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of remaining fuel in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on / off)), brake opening, steering wheel rotation steering angle, gear position, and information on the door open / closed state, location information, etc.), date and time information, atmospheric pressure, illuminance, etc.

[0111] [Hardware configuration of the information display terminal 3800] Next, the hardware configuration of the information display terminal 3800 will be described with reference to FIG.

[0112] 7 is a diagram showing an example of a hardware configuration of an information display terminal 3800 according to embodiment 1. The information display terminal 3800 includes, as a hardware configuration, a control unit 3810, a storage unit 3820, a communication unit 3830, an input unit 3840, and an output unit 3850. The control unit 3810, the storage unit 3820, the communication unit 3830, the input unit 3840, and the output unit 3850 are electrically connected to each other via a communication bus.

[0113] The control unit 3810 is configured with a microcomputer including a CPU and a semiconductor memory, and controls the overall operation of the information display terminal 3800. In particular, the control unit 3810 realizes various functions related to the information display terminal 3800 by reading and executing a predetermined program stored in the storage unit 3820. That is, the control unit 3810 executes processing based on a predetermined program stored in the storage unit 3820, thereby realizing the functional configuration shown in FIG. 9 described later and the processing of the activity diagram shown in FIG. 11 described later. Note that the control unit 3810 is not limited to being single. The information display terminal 3800 may be configured to include multiple control units 3810.

[0114] The storage unit 3820 is configured with any one of an HDD, an SSD, a RAM, and a ROM, or any combination of these. The storage unit 3820 stores a program for exchanging data related to image processing with the server 3700, data used when the control unit 3810 executes processing based on the program, and the like.

[0115] The communication unit 3830 is a so-called network interface that is communicatively connected to a network 3900 in order to transmit and receive various data between the information display terminal 3800 and the server 3700. That is, main information received by the information display terminal 3800 via the communication unit 3830 includes image information before image processing is performed, image information after image processing is performed, information related to the image processing method, information related to the state of the vehicle 3400 (such as the speed of the vehicle 3400 (vehicle speed), acceleration, engine RPM, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air flow rate (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of remaining fuel in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on / off)), brake opening, steering wheel rotation steering angle, gear position, and information on the door open / closed state), date and time information, location information, air pressure, and illuminance. Main information transmitted by information display terminal 3800 via communication unit 3830 includes an instruction for image processing, an instruction requesting downloading of an image, an instruction requesting information other than an image, and the like.

[0116] The input unit 3840 accepts operations to the information display terminal 3800. Examples of the input unit 3840 include a mouse, a keyboard, a touch panel, a microphone, and buttons.

[0117] The output unit 3850 outputs the result of an operation on the input unit 3840. Examples of the output unit 3850 include a display, a speaker, a light, and a button.

[0118] [Server 3700 functional configuration] Hereinafter, the functional configuration of the control unit 3710 of the server 3700 will be described with reference to FIG.

[0119] Fig. 8 is a diagram showing an example of the functional configuration of the server 3700 according to embodiment 1. As shown in Fig. 8, the server 3700 includes an image processing unit 3711, a communication control unit 3712, and a storage control unit 3713 as its functional configuration.

[0120] The image processing unit 3711 executes image processing on an image in the storage unit 3720 or an image transmitted from the information display terminal 3800 based on a request from the information display terminal 3800. This allows the server 3700, which can communicate with the information display terminal 3800, to execute image processing. In the first embodiment, the image processing unit 3711 is described as executing various image processing based on a request from the information display terminal 3800. However, the server 3700 may have an image processing unit for each image processing. In addition, instead of the software of the server 3700 executing the image processing, dedicated hardware (for example, a GPU, etc.) of the server 3700 may execute the image processing. When the dedicated hardware executes the image processing, one piece of hardware of the server 3700 may be configured to execute multiple image processing, or multiple pieces of hardware of the server 3700 corresponding to each of the multiple image processing may be configured to execute the image processing.

[0121] The communication control unit 3712 controls the exchange of information with the vehicle 3400 and the information display terminal 3800 via the network 3900 .

[0122] The memory control unit 3713 stores information in the memory unit 3720 and acquires the stored information.

[0123] [Functional configuration of the information display terminal 3800] Hereinafter, the functional configuration of the control unit 3810 of the information display terminal 3800 will be described with reference to FIG.

[0124] Fig. 9 is a diagram showing an example of the functional configuration of the information display terminal 3800 according to embodiment 1. As shown in Fig. 9, the information display terminal 3800 includes a display control unit 3811, a communication control unit 3812, and a storage control unit 3813 as the functional configuration.

[0125] The display control unit 3811 controls information to be output to the output unit 3850 of the information display terminal 3800. More specifically, the display control unit 3811 controls so as to display a screen including information related to an image.

[0126] The communication control unit 3812 controls the exchange of information with the server 3700 via the network 3900. More specifically, the communication control unit 3812 acquires, from the server 3700, images captured by the photographing device 3100, images captured by the photographing device 3200, etc.

[0127] The memory control unit 3813 stores information in the memory unit 3820 and acquires the stored information.

[0128] [Outline of information processing] The flow of information processing in the information processing system 3000 according to the first embodiment will be described with reference to FIGS.

[0129] [Server 3700 information processing] First, a description will be given of the flow of information processing by the server 3700. Fig. 10 is an activity diagram showing the flow of information processing by the server 3700 according to embodiment 1. Below, an overview of the flow of information processing will be given based on this activity diagram.

[0130] In A4001, the communication control unit 3712 waits to receive information from the vehicle 3400. When the communication control unit 3712 determines that information has been received from the vehicle 3400, the communication control unit 3712 advances the process to A4002. The following description will be given assuming that the communication control unit 3712 receives, as information from the vehicle 3400, identification information for identifying the vehicle 3400, image information captured by the image capturing device 3100, image information captured by the image capturing device 3200, vehicle information of the vehicle 3400, and other information (for example, time information when the image was captured, etc.).

[0131] In A4002, the storage control unit 3713 stores the received image information, vehicle information, and other information in the storage unit 3720. More specifically, the storage control unit 3713 stores the image information, vehicle information, and other information in a predetermined area of ​​the storage unit 3720 in association with the identification information for identifying the received vehicle 3400.

[0132] In A4003, the image processing unit 3711 analyzes the image information, the vehicle information, and other information. In the first embodiment, the image processing unit 3711 analyzes the acquired image information to determine, for example, that the vehicle 3400 is traveling in fog from 07:22:10 on April 29, 2021 to 07:24:18 on April 29, 2021. The image processing unit 3711 associates the analysis result for each time information with the identification information for identifying the vehicle 3400, and stores the result in a predetermined area of ​​the storage unit 3720 together with the image information, the vehicle information, and other information. In this embodiment, the server 3700 is assumed to analyze the image information. However, the information display terminal 3800 that receives the image information may analyze the image information.

[0133] In A4004, the communication control unit 3712 transmits image information, vehicle information, analysis results, and other information to the information display terminal 3800 based on an instruction from the information display terminal 3800. More specifically, the communication control unit 3712 transmits to the information display terminal 3800 image information, vehicle information, and other information relating to the vehicle identified by the identification information instructed by the information display terminal 3800, from the specified date and time information onward.

[0134] In A4005, the communications control unit 3712 waits to receive an image processing request from the information display terminal 3800. When the communications control unit 3712 determines that an image processing request has been received from the information display terminal 3800, the communications control unit 3712 advances the process to A4006. For example, the communications control unit 3712 receives a request to perform fog correction for the entire range of an image as the image processing request from the information display terminal 3800.

[0135] In A4006, the image processing unit 3711 executes image processing on the image based on the image processing request. For example, the image processing unit 3711 performs defogging on the entire image based on the image processing request. Defogging is image processing that adjusts the brightness of the image to improve poor visibility due to fog.

[0136] In A4007, the communication control unit 3712 transmits the result of the image processing in the image processing unit 3711 to the information display terminal 3800.

[0137] [Information processing by the information display terminal 3800] Next, a description will be given of the flow of information processing by the information display terminal 3800. Fig. 11 is an activity diagram showing the flow of information processing by the information display terminal 3800 according to embodiment 1. Below, the flow of information processing will be outlined with reference to this activity diagram.

[0138] In A4011, the display control unit 3811 displays a launch button for the external video player on the viewer. When the launch button on the viewer is selected, the communication control unit 3812 launches the external video player in the server 3700. At this time, the communication control unit 3812 transmits to the server 3700, as a "command line argument", the file name of the video, video effect settings, parameters for disabling at least a part of the operation unit of the external video player, and the like. At this time, the communication control unit 3812 may also transmit the playback start time, playback end time, whether or not to loop, and the like. Thereafter, the communication control unit 3812 controls the external video player by sending a control command. At this time, the communication control unit 3812 may transmit an instruction for disabling at least a part of the operation unit of the external video player.

[0139] At A4012, the communication control unit 3812 waits to receive information from the server 3700. When the communication control unit 3812 determines that information from the server 3700 has been received, the communication control unit 3812 advances the process to A4013. The following description is given assuming that the communication control unit 3812 receives, as information from the server 3700, image information, vehicle information, analysis information analyzed by the image processing unit 3711, and other information. More specifically, in the first embodiment, the following description is given assuming that the communication control unit 3812 receives, from the server 3700, image information, acceleration information, vehicle speed information, position information, analysis information, date and time information, and the like.

[0140] In A4013, the display control unit 3811 displays the received image information, vehicle information, analysis information, and other information. More specifically, in the first embodiment, the display control unit 3811 displays image information, acceleration information, vehicle speed information, position information, analysis information, date and time information, and the like.

[0141] In A4014, the display control unit 3811 determines whether or not a range designation for image processing has been accepted. If the display control unit 3811 determines that a range designation for image processing has been accepted, the process proceeds to A4015. If the display control unit 3811 determines that a range designation for image processing has not been accepted, the process proceeds to A4016. In the following description, it is assumed that when no range designation for image processing is received, the entire range of the image is subject to image processing. That is, although omitted in Fig. 11 for simplification, when no range designation for image processing is received, "the entire image" is held as the range information. In the first embodiment, it is assumed that "the entire image" is held as the range information.

[0142] In A4015, the communication control unit 3812 holds the range information of the image processing selected by the operator of the information display terminal 3800 via a screen or the like.

[0143] In A4016, the display control unit 3811 waits for receipt of an instruction to execute image processing selected from among multiple image processing from the user via the screen. In the first embodiment, the display control unit 3811 will be described on the assumption that it has received an instruction to execute fog correction from among multiple image processing from the user via the screen.

[0144] When a user issues an instruction to execute image processing, in A4017, the communication control unit 3812 transmits an instruction to execute image processing to the server 3700. In the first embodiment, the communication control unit 3812 transmits an instruction to execute fog correction for the image to the server 3700.

[0145] In A4018, the communication control unit 3812 waits to receive an image resulting from processing by the server 3700. When the communication control unit 3812 receives the image, it advances to processing A4019. In the first embodiment, the communication control unit 3812 receives the image after fog correction processed by the server 3700.

[0146] In A4019, the display control unit 3811 displays on the screen the information acquired from the server 3700. In the first embodiment, the display control unit 3811 displays on the screen the image after fog correction acquired from the server 3700.

[0147] In this embodiment, the external video player in the server 3700 is controlled from a PC Browser (PC application) which is a viewer on the information display terminal 3800. Examples of the external video player include free software such as VLC, a cloud image processing service, and an image sharpening service. When installing the viewer of this embodiment, the display control unit 3811 asks whether to also install the external video player. When there is an instruction to install the external video player, the storage control unit 3813 installs the external video player. When installing the external video player or immediately after starting it, the display control unit 3811 displays a guide to a button for starting the external video player and a note when using the external video player. The display control unit 3811 may display, as a note when using the external player, "When a button on the external video player is operated, it may be different from the display state of the viewer" and "When the display state of the viewer is different, it is recommended to close the external video player and start the external video player from the viewer again."

[0148] When a cloud-based system is used, the storage control unit 3813 may be configured to have a function of storing an ID or the like for automatically logging in to the cloud image sharpening service in the storage unit 3820. The communication control unit 3812 may also be configured to have a function of automatically sending information, a function of uploading, and a function of determining whether conversion is complete and automatically downloading when conversion is complete or notifying that downloading is possible. Furthermore, the display control unit 3811 may have a function of playing the downloaded image in synchronization with the image before conversion.

[0149] Furthermore, the display control unit 3811 controls to display that when playing with a non-recommended viewer, the image can be played beautifully when played with a dedicated viewer. When a dark image is recorded, it is corrected and displayed beautifully when played with a dedicated viewer, but when played with a standard player for an OS such as Windows (registered trademark), the played image may be dark. Therefore, for example, the display control unit 3811 controls to provide black band areas above and below the image when playing with a non-recommended viewer, and to display "This image will be played beautifully when played with a dedicated viewer" in the black band areas. At this time, the display control unit 3811 may display a code (for example, a QR code (registered trademark)) that refers to a URL where the viewer can be installed together with the character string. As another form, the display control unit 3811 may display the character string "This image will be played beautifully when played with a dedicated viewer" at the beginning of the image when playing with a non-recommended viewer, or may display a code (for example, a QR code (registered trademark)) that refers to a URL where the viewer can be installed.

[0150] [Details of information processing] Next, the above-mentioned information processing will be described in detail.

[0151] FIG. 12 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of the first embodiment. The display control unit 3811 controls the display of information in the server 3700 on the viewer 4100. The viewer 4100 is configured to be able to call various functions and display information. More specifically, the viewer 4100 includes a file button 4101, a display button 4102, a play button 4103, a tool button 4104, and an information button 4105, and these buttons enable calling and referring to various functions. The viewer 4100 also includes a video display area 4110, an acceleration graph display area 4120, a map image display area 4130, a date and time display area 4150, a vehicle state display area 4160, and a speed graph display area 4170, and these areas enable referring to the image being played and information related to the image. Furthermore, the viewer 4100 includes a video display area 4110, a fog correction button 4140, an HDR button 4141, a sharpening button 4142, an other processing button 4143, a partial processing button 4144, a stream display button 4145, an operation button 4146, and a playlist display area 4180, and in response to operations of these buttons, the display control unit 3811 requests the server 3700 to process images and play back images. That is, the viewer 4100 is configured to be able to accept a plurality of operation instructions from the user. At this time, each of the plurality of operation instructions instructs the display of a different processing result related to the image.

[0152] The file button 4101 is configured to be able to execute various file operation-related processes. More specifically, when the file button 4101 is selected, the display control unit 3811 displays file-related processes such as "view file", "save over", and "save as" in a pull-down format on the screen, and when any file-related process is selected, a request is made to the control unit 3810 etc. to execute the selected process.

[0153] The display button 4102 makes it possible to execute various processes related to the display of the video display area 4110. More specifically, when the display button 4102 is selected, the display control unit 3811 displays display-related processes such as "display side by side," "display in a row," and "minimize" in a pull-down format, and when one of the display-related processes is selected, the selected process is executed.

[0154] The play button 4103 makes it possible to execute various processes related to the playback of video. More specifically, when the play button 4103 is selected, the display control unit 3811 displays processes related to playback, such as "play next playlist," "play previous playlist," "play from next image processing," and "play from previous image processing," in a pull-down format, and by selecting one of the display-related processes, the selected process is executed.

[0155] Various functions can be called up by the tool button 4104. More specifically, when the tool button 4104 is selected, the display control unit 3811 displays a pull-down menu of options such as "Macro" and "User Settings," and when one of the processes is selected, the display control unit 3811 requests the control unit 3810 to execute the selected process.

[0156] Various information can be referred to by the information button 4105. More specifically, when the tool button 4104 is selected, the display control unit 3811 displays buttons relating to information such as "Version Information" in a pull-down format, and by selecting one of the buttons, the selected information is displayed so that the user can refer to it.

[0157] The video display area 4110 is an area where images captured by the imaging device 3100 and the imaging device 3200 and images processed by the image processing unit 3711 are displayed. The display format of the video display area 4110 can be changed arbitrarily in response to a user instruction. More specifically, the video display area 4110 may display a plurality of images after image processing using different methods in accordance with a user instruction, or may display an image after image processing and an image before image processing in a side-by-side manner. Furthermore, the video display area 4110 may display any information, such as an acceleration graph display area 4120.

[0158] The acceleration graph display area 4120 is an area where the acceleration of the vehicle that captured the video is displayed in three dimensions. In the acceleration graph display area 4120, an X axis is formed in the depth direction, a Y axis is formed in the horizontal direction, and a Z axis is formed in the vertical direction. Here, the X axis is an axis indicating the acceleration in the front and rear of the vehicle, the Y axis is an axis indicating the acceleration in the lateral direction of the vehicle, and the Z axis is an axis indicating the acceleration in the vertical direction of the vehicle. Acceleration in the front of the vehicle (positive acceleration) is graphed in the upward direction on the X axis, and acceleration in the rear of the vehicle (negative acceleration) is graphed in the downward direction on the X axis. Acceleration in the right direction toward the vehicle is graphed in the upward direction on the Y axis, and acceleration in the left direction toward the vehicle is graphed in the downward direction on the Y axis. Acceleration in the upward direction of the vehicle is graphed in the left direction on the Z axis, and acceleration in the downward direction of the vehicle is graphed in the right direction on the Z axis.

[0159] The map image display area 4130 is an area where an image of a map in the vicinity of the shooting location of the video displayed in the video display area 4110 is displayed. Data showing the vehicle's position is transmitted from the information display terminal 3800 to a map server (not shown) existing on the network 3900, and data showing a map in the vicinity of the vehicle's position is transmitted from this map server to the information display terminal 3800, whereby an image of the map is displayed in the map image display area 4130. At this time, a current location icon 4131 is displayed on the map image display area 4130.

[0160] The viewer 4100 includes a defog button 4140. When the defog button 4140 is selected, the image processor 3711 performs defog on the image displayed in the video display area 4110. More specifically, when the defog button 4140 is selected, the communication controller 3812 of the information display terminal 3800 instructs the server 3700 to perform defog on the image. The server 3700 receives the instruction and performs defog on the image with the image processor 3711. The communication controller 3712 of the server 3700 transmits the image that has been defogged to the information display terminal 3800. The display controller 3811 controls the video display area 4110 to display the image that has been defogged. In other words, the video can be defogged by selecting a button in the PC Browser (PC application). In addition, when the defog button 4140 is operated again while defog is being performed, the image processor 3711 can control the defog to be released.

[0161] The display control unit 3811 can also control the display to adjust the degree of mist correction according to the number of times the mist correction button 4140 is selected or the selection time. More specifically, the display control unit 3811 controls the display to increase the degree of mist correction according to the number of times the mist correction button 4140 is pressed, or controls the display to increase the degree of mist correction according to the time the mist correction button 4140 is pressed. That is, the display control unit 3811 receives an operation instruction for the mist correction button 4140 selected by the user, receives an instruction to display the processing results related to the image corresponding to the operation instruction, and controls the display to display the video display area 4110 including the processing results related to the image. The mist correction button 4140 is an example of an object. The operation of selecting the mist correction button 4140 is an example of an operation of an object.

[0162] The viewer 4100 includes an HDR (High Dynamic Range) button 4141. When the HDR button 4141 is selected, the image processing unit 3711 performs HDR processing on the image displayed in the video display area 4110. Here, the HDR processing is a process of expressing an image without blown-out highlights or crushed shadows by synthesizing a plurality of photos with different brightnesses. More specifically, when the HDR button 4141 is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform HDR processing on the image. The server 3700 that has received the instruction synthesizes a plurality of photos with different brightnesses and performs HDR processing by the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the image that has been subjected to HDR processing to the information display terminal 3800. The display control unit 3811 controls the video display area 4110 including the image that has been subjected to HDR processing to be displayed.

[0163] Furthermore, the display control unit 3811 receives an instruction to display the processing result related to the image corresponding to the button selected by the user, and controls to display a screen including the processing result related to the image. More specifically, the image processing unit 3711 receives an instruction to execute fog correction by selecting the fog correction button 4140, receives an instruction to execute HDR processing by selecting the HDR button 4141, and controls to display the processing result in the video display area 4110. Here, like the fog correction button 4140 and the HDR button 4141, the buttons related to image processing are different for different processing results related to the image. The fog correction button 4140 and the HDR button 4141 are examples of objects. This allows any image processing corresponding to the button to be executed by the user's selection. Furthermore, when multiple buttons are displayed, the display control unit 3811 can control not to display these at the same time. More specifically, the display control unit 3811 can control not to display the fog correction button 4140 and the HDR button 4141 at the same time.

[0164] Furthermore, at this time, when the fog correction button 4140 and the HDR button 4141 are selected via the screen, the display control unit 3811 may receive an instruction to display a first processing result related to the image, which corresponds to the fog correction button 4140, and an instruction to display a second processing result related to the image, which corresponds to the HDR button 4141, and control to display a screen including the first processing result related to the image and the second processing result related to the image. More specifically, when the fog correction button 4140 and the HDR button 4141 are selected via the viewer 4100, the display control unit 3811 receives an instruction to display a processing result of fog correction related to the image, which corresponds to the fog correction button 4140, and an instruction to display a processing result of HDR processing related to the image, which corresponds to the HDR button 4141, and controls to display a screen including the processing result of fog correction related to the image and the processing result of HDR processing related to the image. This makes it possible to simultaneously execute a plurality of arbitrary image processes. The fog correction button 4140 is an example of a first object. The HDR button 4141 is an example of a second object.

[0165] The viewer 4100 includes a sharpening button 4142. When the sharpening button 4142 is selected, the image processing unit 3711 performs sharpening processing on the image displayed in the image display area 4110. More specifically, when the sharpening button 4142 is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform sharpening processing. The server 3700, which has received the instruction, sharpens the image using the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the image that has been sharpened to the information display terminal 3800. The display control unit 3811 controls the image display area 4110 to display the image that has been sharpened.

[0166] The viewer 4100 includes an other processing button 4143. When the other processing button 4143 is selected, the display control unit 3811 controls the viewer 4100 to display various image processing options. Here, examples of the image processing options include an image processing adjustment area 6290, which will be described later. When this image processing option is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform image processing. The server 3700 that receives the instruction performs image processing by the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the image that has been subjected to image processing to the information display terminal 3800. The display control unit 3811 controls the display of the video display area 4110 including the image that has been subjected to image processing.

[0167] The viewer 4100 includes a partial processing button 4144. When the partial processing button 4144 is selected, the display control unit 3811 controls the display so that a part of the video display area 4110 can be selected. When this image processing option is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform image processing on a part of the image. The server 3700 that receives the instruction performs image processing on a part of the image by the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the image that has been subjected to image processing to the information display terminal 3800. The display control unit 3811 controls so as to display the video display area 4110 including the image that has been subjected to image processing on a part of the image. Here, the display control unit 3811 may control so as to use the image before image processing and the image after image processing to display the part of the image in the video display area 4110 including the image that has been subjected to image processing. Note that the method for performing partial processing may be another embodiment. For example, the display control unit 3811 accepts the designation of a correction target range for the video display area 4110. The display control unit 3811 further provides a range registration checkbox. The display control unit 3811 may set the designated range as the correction target range while the correction button is pressed with the range registration checkbox checked. At this time, the display control unit 3811 may provide a button to temporarily remove the correction while the range remains in a corrected state.

[0168] The viewer 4100 includes a stream display button 4145. When the stream display button 4145 is selected, the display control unit 3811 performs control so that a plurality of images are displayed side-by-side in the video display area 4110.

[0169] Each button displayed on the viewer 4100 may be in another form. For example, each button displayed on the viewer 4100 may be a radio button, a check box, a toggle button, or the like, instead of a push button. Also, for example, each button has been described as a soft button displayed on the viewer 4100, but it may be provided in a location other than the viewer 4100, or may be a physical button such as a physical switch.

[0170] The date and time display area 4150 is an area that displays the recording date and time of the video displayed in the video display area 4110. The date and time display area 4150 may also be configured to display the current date and time.

[0171] The vehicle status display area 4160 is an area where information such as the vehicle speed at the time of recording the video is displayed. The vehicle status display area 4160 displays the status of the vehicle to which the image capturing device 3100, 3200 that recorded the video displayed in the video display area 4110 is attached.

[0172] The speed graph display area 4170 is an area in which the speed of the vehicle is displayed in a graph form. In addition, the speed graph display area 4170 may also display acceleration in a graph form in addition to the speed.

[0173] The playlist display area 4180 is an area where information about stored video data is displayed. When information about video data in the playlist display area 4180 is selected, the display control unit 3811 accesses the video data in the server 3700 and displays it in the video display area 4110. At this time, the video data may be stored in the information display terminal 3800, in addition to in the server 3700, or may be stored in an external storage device.

[0174] FIG. 13 is a diagram showing an example of a screen after image processing, displayed by the display control unit 3811 of the first embodiment. The display control unit 3811 controls the display of information processed in the server 3700 on the viewer 4200. The image display area 4210 of the viewer 4200 in the first embodiment is an image that has been subjected to fog correction by selecting the fog correction button 4140 for the image display area 4110 of FIG. 12. That is, the image after image processing is included in the image display area 4210. At this time, the fog correction button 4240 is displayed in a different color from the other buttons so that it can be seen that it has been selected. Furthermore, at this time, the storage control unit 3813 stores the image captured by the imaging device 3100 in the storage area. As a result, the information display terminal 3800 can store the image before image processing in the storage unit 3820 while displaying the image after image processing in the image display area 4210.

[0175] In the first embodiment, an example has been described in which the storage control unit 3813 controls to store the image before the image processing, and the display control unit 3811 controls to display the image after the image processing. However, the image before the image processing may be reproduced and displayed using the image after the image processing. More specifically, the storage control unit 3713 controls to store the image after the image processing and the image processing performed on the image in the storage unit 3720. Next, the image processing unit 3711 controls to reproduce the image before the image processing based on the information of the image processing performed on the image after the image processing. Then, the display control unit 3811 controls to display the reproduced image before the image processing. At this time, the display control unit 3811 may display which data is the image displayed in the video display area 4210. More specifically, the display control unit 3811 may display whether the displayed data is the original data, the data after the image processing, or the data reproduced from the data after the image processing to the data before the image processing.

[0176] [Variation 1] A first modification of the first embodiment will be described. FIG. 14 is a diagram showing an example of a screen after image processing displayed by the display control unit 3811 of the first modified example of the first embodiment. The video display area 4310 of the viewer 4300 in the first modified example includes a mist correction area 4311 in which a part of the video display area 4310 has been subjected to mist correction, and a sharpening area 4313 in which a sharpening process has been performed on an area different from the mist correction area 4311. In this way, the image processing related to the mist correction button 4340 of the first modified example and the image processing related to the sharpening button 4342 are configured so that the areas to be processed in the image are different. The mist correction button 4340 is an example of a first object. The sharpening button 4342 is an example of a second object. The information display terminal 3800 of the first modified example can execute any image processing on different areas. For example, the user selects a predetermined area (an area corresponding to defog area 4311) in image display area 4110, and selects defog button 4340. The user also selects a predetermined area (an area corresponding to sharpening area 4313) in image display area 4110, and selects sharpening button 4342. By making such selections, a screen such as that shown in FIG. Moreover, the display control unit 3811 can display what kind of processing has been performed on which area by displaying the processing content in each processing area. That is, the display control unit 3811 can display a processing content display section 4312 in the mist correction area 4311 so as to show that the mist correction area 4311 has performed mist correction. Moreover, the display control unit 3811 can display a processing content display section 4314 in the sharpening area 4313 so as to show that the sharpening processing has been performed. Furthermore, the display control unit 3811 can display that the mist correction button 4340, the sharpening button 4342, and the partial processing button 4344 are being executed by changing the colors of these buttons. In the example of the first modification, defogging and sharpening are described as examples of image processing for different regions, but the present invention is not limited to this and may be other image processing. Also, the same image processing (e.g., defogging) may be performed to different degrees on different regions.

[0177] [Variation 2] A second modification of the first embodiment will be described. 15 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 2 of the first embodiment. Objects displayed in the viewer 4400 in Modification 2 are a sunny correction button 4440, a cloudy correction button 4441, a rain correction button 4442, and a fog correction button 4443. The sunny correction button 4440, the cloudy correction button 4441, the rain correction button 4442, and the fog correction button 4443 are examples of objects related to image processing related to weather. In Modification 2, different image corrections are applied to sunny, rainy, and foggy conditions using weather information in the viewer. The display control unit 3811 accepts an instruction for image quality correction such as gamma correction, color correction (color emphasis, color balance), and edge emphasis corresponding to the weather, by selecting each button. For example, the display control unit 3811 requests the image processing unit 3711 of the server 3700 to perform image processing such as gamma correction, color correction (color emphasis, color balance), and edge emphasis on the image so as to erase the rain by selecting the rain correction button 4442. In addition, the display control unit 3811 requests the image processing unit 3711 of the server 3700 to perform image processing such as gamma correction, color correction (color emphasis, color balance), and edge emphasis on the image so as to make the fog transparent and make the other side of the fog visible, by selecting the fog correction button 4443. In addition, the display control unit 3811 may display, as the names of these buttons, button names for image correction according to the purpose of the image, such as an operation button for erasing a snowstorm, a button for making the fog transparent so that the other side of the fog can be seen, and a button for erasing rain. Furthermore, the display control unit 3811 may provide new objects related to "clear", "cloudy", "rain", and "fog" in the viewer 4400, or provide objects related to "clear", "slightly cloudy", "smoke", "sandstorm", "blizzard", "drizzle", "sleet", "snow", "grail", "lightning", and the like instead of "clear", "cloudy", "rainy", "rainy", "rainy", "rainy", "lightning", and the like. Instead of providing objects for each weather, an almighty automatic correction may be performed in which these weather conditions are automatically detected and correction is performed automatically. When the almighty automatic correction is configured to be possible, the display control unit 3811 displays a button for performing / not performing image correction on the screen. If the display control unit 3811 determines that there is a limit to the almighty automatic correction, it creates some kind of state. For example, the display control unit 3811 displays another correction means. With these, it is possible to perform appropriate image processing in accordance with the weather.

[0178] [Variation 3] A third modification of the first embodiment will be described. 16 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 3 of Embodiment 1. An image captured by the imaging device 3100 of the vehicle 3400 traveling on a road surface in foggy weather is displayed in the video display area 4510 of the viewer 4500 in Modification 3. At this time, the display control unit 3811 controls to highlight and display the fog correction button 4540 as an appropriate correction method. That is, the display control unit 3811 changes the button displayed on the screen based on the state of the image. This makes it possible to perform appropriate image processing based on the information in the image. As one aspect of changing the button, the display control unit 3811 can display a new button. As another aspect of changing the button, the outline of the button can be highlighted. As yet another aspect of changing the button, the display control unit 3811 can display the button by shining it. For example, when the vehicle 3400 runs in a city or a forest, the brightness changes depending on the location and / or direction of the run, so the display control unit 3811 may control to display a new brightness correction button, highlight the outline of the brightness correction button, or display the brightness correction button by shining it accordingly.

[0179] [Variation 4] A fourth modification of the first embodiment will be described. 17 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 4 of Embodiment 1. An image captured by the imaging device 3100 of the vehicle 3400 immediately before entering a tunnel is displayed in the video display area 4610 of the viewer 4600 in Modification 4. At this time, the current location icon 4631 in the map image display area 4630 indicates that the vehicle 3400 is immediately before entering a tunnel. The communication control unit 3812 of the information display terminal 3800 acquires road network information related to the position information of the vehicle 3400 from a map server (not shown) via the network 3900. Then, the display control unit 3811 controls to display a tunnel correction button 4640 suitable for image processing in a tunnel. That is, the display control unit 3811 controls to display a button suggesting image processing for an image based on the road network information on the screen. Here, by selecting the tunnel correction button 4640, the image processing unit 3711 automatically performs an appropriate type of correction for the tunnel position. Specifically, by selecting the tunnel correction button 4640, the image processing unit 3711 performs image processing to adjust the brightness to correct the darkness of the tunnel, reduce the influence of lighting provided in the tunnel, and reduce the influence of the lighting device of the vehicle ahead. Thus, it is possible to perform appropriate image processing based on the road network information. Here, road network information is information relating to road conditions, and more specifically, this information includes, in addition to tunnel location information, speed limit information, road width information, 12-hour weekday traffic volume information, travel speed information (for a certain section, the distance of the section divided by the time required to travel, the time required includes the speed at which stops due to traffic lights, congestion, etc. are made), number of lanes information, etc.

[0180] [Variation 5] A fifth modification of the first embodiment will be described. FIG. 18 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of the fifth modified example of the first embodiment. The sensor unit 3017 of the fifth modified example measures acceleration information of the vehicle 3400. That is, the vehicle information of the fifth modified example includes the acceleration information measured by the sensor unit 3017. The image display area 4710 of the viewer 4700 in the fifth modified example displays an image of the vehicle 3400 that is accelerating rapidly, captured by the image capture device 3100. At this time, the acceleration graph display area 4720 is a graph showing a large upward inclination of the X-axis, and the speed graph display area 4770 is a graph showing a rapid increase in speed. In addition, the viewer 4700 may be provided with an acceleration display unit 4721 such as "1G or more detected" at this time. The display control unit 3811 controls to display an acceleration correction button 4740 suitable for image processing when the vehicle 3400 accelerates. That is, the display control unit 3811 controls to display on the acceleration correction button screen based on information from the sensor unit 3017 that detects the acceleration of the vehicle 3400 having the image capture device 3100. Here, when the acceleration correction button 4740 is selected, image processing is performed to correct screen shaking and / or poor visibility caused by sudden acceleration. This makes it possible to perform appropriate image processing based on information from the acceleration sensor of the moving object. The image capture device 3100 is an example of an image capture unit. The vehicle 3400 is an example of a moving object. The sensor unit 3017 is an example of an acceleration sensor. The acceleration correction button is an example of an object that suggests image processing for an image. Furthermore, the display control unit 3811 accepts one or more acceleration correction settings by the user. Specifically, the display control unit 3811 accepts one or more acceleration correction settings by the user inputting a numerical value for performing the acceleration correction, dragging to the acceleration graph display area 4720 or the speed graph display area 4770, etc. At this time, the display control unit 3811 displays bands of different colors in the range for each type of correction in the acceleration graph display area 4720 or the speed graph display area 4770, displays the dragged range, etc.

[0181] [Variation 6] A sixth modification of the first embodiment will be described. 19 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 6 of Embodiment 1. An image captured by the imaging device 3100 of the vehicle 3400 is displayed in the image display area 4810 of the viewer 4800 in Modification 6. At this time, a current location icon 4831 in a map image display area 4830 where the vehicle 3400 is traveling indicates the location on the map where the vehicle 3400 is traveling. The display control unit 3811 can set a processing area 4832 on the map image display area 4830 based on a user operation or the like, and when the current location icon 4831 enters this area, controls to display a fog compensation button 4840. That is, the display control unit 3811 controls to display on the screen a button or the like that suggests image processing for an image based on the position information of the vehicle 3400 having the image capturing device 3100. This makes it possible to perform appropriate image processing based on the position information of the vehicle 3400. As described above, the processing area 4832 may be set based on a user operation or the like, or may be set automatically. For example, in a case where an image processing button for brightening an image is set to be displayed when the vehicle 3400 enters a tunnel, the display control unit 3811 performs control so that an image processing button for brightening an image is displayed on the screen when the display control unit 3811 determines that the vehicle 3400 has entered a tunnel based on the position information of the vehicle 3400.

[0182] [Variation 7] A seventh modification of the first embodiment will be described. 20 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 7 of the first embodiment. An image captured by the imaging device 3100 of the vehicle 3400 of a road where it is expected to be cloudy with rain, is displayed in an image display area 4910 of the viewer 4900 in Modification 7. At this time, the viewer 4900 can be configured to display weather information 4911 for that road. In this embodiment, the viewer 4900 displays weather information 4911 relating to cloudy with rain. The communication control unit 3812 acquires weather information of the route taken by the vehicle 3400 from a weather information providing server (not shown) based on the position information of the vehicle 3400 and the date and time information included in the image captured by the image capture device 3100. Then, the display control unit 3811 controls to display a button suggesting image processing based on the position information of the vehicle 3400 and the weather information corresponding to the position information. That is, the display control unit 3811 controls to display an object suggesting image processing for the image on the screen based on the position information of the vehicle 3400 having the image capture device 3100 and the weather information of the date and time when the image was captured. In the seventh modification, image correction according to the time period recorded by the viewer can be applied. For example, the image processing unit 3711 can apply different image corrections to morning, night, and evening based on an instruction from the communication control unit 3812 and the weather information 4911. In the example of the seventh modification, the display control unit 3811 controls the viewer 4900 to display a rain correction button 4941 based on the position information of the vehicle 3400 and the weather information. Here, when the rain correction button 4941 is selected, image processing is executed to correct darkness due to rain clouds and poor visibility due to rain. This makes it possible to perform appropriate image processing based on the position information of the vehicle 3400 and the weather information of the date and time at that position. Here, the weather information includes information including sunny, clear, slightly cloudy, haze, dust storm, blizzard, drizzle, sleet, fog, snow, hail, hail, and thunder in addition to cloudy and rain.

[0183] [Variation 8] An eighth modification of the first embodiment will be described. 21 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 8 of the first embodiment. An image captured by the imaging device 3100 of the vehicle 3400 approaching an intersection is displayed in the image display area 5010 of the viewer 5000 in Modification 8. At this time, the current location icon 5031 in the map image display area 5030 indicates that the vehicle 3400 is approaching the intersection. The speed graph display area 5070 shows that the vehicle 3400 is decelerating as it approaches the intersection. The display control unit 3811 controls to display the brake correction button 5040 in response to the deceleration of the vehicle 3400. That is, the display control unit 3811 controls to display an object suggesting image processing for an image on the screen based on the operation information of the vehicle 3400 having the image capturing device 3100. Here, by selecting the brake correction button 5040, blurring of the field of view due to deceleration can be reduced, and the influence of the brake lamp on the image display area 5010 can be reduced. This makes it possible to perform appropriate image processing based on the operation information of the vehicle 3400. Here, the operation information of the vehicle 3400 is information related to the operation among the vehicle information. In addition to the brake, the vehicle information includes information that can be obtained by the operation of the turn signal, back, and other OBD (On-board diagnostics).

[0184] [Variation 9] A ninth modification of the first embodiment will be described. Fig. 22 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of the ninth modification of the first embodiment. An image captured by the imaging device 3100 of the vehicle 3400 is displayed in the image display area 5110 of the viewer 5100 in the ninth modification. At this time, information related to the date and time is displayed in the date and time display area 5150. In the example of Fig. 22, it is shown that the time is night. The display control unit 3811 controls to display the night correction button 5140 based on date and time information, which is attribute information of the image. That is, the display control unit 3811 controls to display a button suggesting image processing for the image on the screen based on the date and time when the image was captured. Here, by selecting the night correction button 5140, the darkness of the night is corrected to be bright so that what is around can be displayed. This makes it possible to perform appropriate image processing based on the date and time when the image was captured. Here, the night correction button is an example of a button that suggests image processing for an image based on the date and time, and may be, for example, a button for correcting brightness due to a sunset to clarify the field of view. The automatic correction may be performed by combining the corrections according to the above-mentioned modification 3 to modification 9. The combination of the corrections according to the modification 3 to modification 9 may be set under an OR condition or an AND condition.

[0185] [Variation 10] A tenth modification of the first embodiment will be described. FIG. 23 is a diagram showing an example of a screen after image processing displayed by the display control unit 3811 of the modification 10 of the first embodiment. In the image display area 5210 of the viewer 5200 in the modification 10, an image captured by the image capture device 3100 of the vehicle 3400 traveling on a road surface in foggy weather is displayed. In addition, in the fog correction area 5211, which is a part of the image display area 5210, the result of fog correction is displayed. That is, in the fog correction area 5221, the result of image processing performed on a part of the image captured by the image capture device 3100 is displayed. When an instruction is given to perform image processing on the fog correction area 5221, the image processing unit 3711 performs image processing on a part of the image captured by the image capture device 3100. In addition, the fog correction area 5211 is an area set according to a user operation or the like before performing image processing. That is, the part of the image is an area set among the entire area of ​​the image captured by the image capture device 3100. More specifically, the display control unit 3811 sets the range of image processing by selecting the partial processing button 5245 and specifying the range on the screen. Next, when the fog correction button 5240 is selected, the display control unit 3811 determines that the set range is the fog correction area 5211 where fog correction is performed. Then, the communication control unit 3812 of the information display terminal 3800 requests the server 3700 to perform fog correction for the fog correction area 5211 via the network 3900. Upon receiving the request, the image processing unit 3711 performs fog correction for the fog correction area 5211. The communication control unit 3812 acquires the result of fog correction for the fog correction area 5211. The display control unit 3811 displays the result of image processing for the fog correction area 5211. This makes it possible to perform image processing on a part of the captured image. This also makes it possible to set the area where image processing is performed.

[0186] [Variation 11] An eleventh modification of the first embodiment will be described. FIG. 24 is a diagram showing an example of a screen after image processing displayed by the display control unit 3811 of the modified example 11 of the embodiment 1. In the image display area 5310 of the viewer 5300 in the modified example 11, an image captured by the image capturing device 3100 of the vehicle 3400 in the middle of the night is displayed as an example. In addition, the night correction area 5311, which is a part of the image display area 5310, displays the result of night correction. Furthermore, the date and time display area 5350 indicates that the time is night. When the date and time display area 5350 is night, the night correction area 5311 displays the result of night correction. That is, the image processing on the image is image processing on a part of the image captured by the image capturing device 3100, and the part of the image is an image captured during a set time period among the images captured by the image capturing device 3100. The display control unit 3811 sets the range of image processing based on the operation of the partial processing button 5345 and the date and time information of the image. Next, the display control unit 3811 determines that the range is the night correction area 5311 where correction is performed at night based on the operation of the night correction button 5340. Furthermore, the communication control unit 3812 instructs the server 3700 to perform image processing on the night correction area 5311 via the network 3900. Next, the image processing unit 3711 executes image processing on the night correction area 5311. Furthermore, the communication control unit 3812 acquires the result of the image processing on the night correction area 5311. Finally, the display control unit 3811 displays the result of the image processing on the night correction area 5311. This makes it possible to perform appropriate image processing on a partial area of ​​the image taken during the set time period. In the eleventh modification example, the partial treatment button 5345 and the night correction button 5340 have been described as examples, but the display control unit 3811 may be provided with a means for inputting a time period into the viewer 5300, and may instruct the viewer 5300 to execute the set image processing on images captured during the time period input via this means.

[0187] [Variation 12] A twelfth modification of the first embodiment will be described. FIG. 25 is a diagram showing an example of a screen after image processing, displayed by the display control unit 3811 of the modified example 12 of the first embodiment. The position information acquisition unit 3020 of the modified example 12 acquires the position information of the vehicle 3400. That is, the other information of the modified example 12 includes the position information acquired by the position information acquisition unit 3020. In the image display area 5410 of the viewer 5400 in the modified example 12, an image captured by the imaging device 3100 of the vehicle 3400 is displayed. In addition, the fog correction area 5411, which is a part of the image display area 5410, displays the result of fog correction. Furthermore, the current location icon 5431 in the map image display area 5430 indicates the location on the map where the vehicle 3400 is traveling. In addition, the map image display area 5430 displays a processing area 5432. When the current location icon 5431 is in this processing area 5432, the fog correction area 5411 displays the result of fog correction. In other words, image processing of an image is image processing of a portion of an image captured by the photographing device 3100, and the portion of the image is an image captured within a range in which the position information of the vehicle 3400 having the photographing device 3100 is set. First, the display control unit 3811 sets the range of image processing based on the operation of the partial processing button 5445 and the position information. Next, based on the operation of the fog correction button 5440, it is determined that the range is the fog correction area 5411 where fog correction is performed at a predetermined position. Furthermore, the communication control unit 3812 instructs the server 3700 via the network 3900 to perform image processing on the fog correction area 5411. Next, the image processing unit 3711 executes image processing on the fog correction area 5411. Furthermore, the communication control unit 3812 acquires the result of the image processing on the fog correction area 5411. Finally, the display control unit 3811 displays the result of the image processing on the fog correction area 5411. This makes it possible to appropriately perform image processing on a partial area of ​​an image captured by the vehicle 3400 moving at the set position. In the twelfth modification example, the partial action button 5445 and the fog correction button 5440 have been described as examples. However, the display control unit 3811 may be provided with a means for inputting latitude information and longitude information to the viewer 5300, and may instruct the viewer 5300 to execute the set image processing on an image captured at the latitude and longitude input via this means. In addition, the input of location information is not limited to latitude and longitude information, and the display control unit 3811 may instruct the display control unit 3811 to execute the set image processing on an image captured at a location within a range selected by the user on the map displayed on the viewer 5300. In addition, the display control unit 3811 may display historical route information of the user's vehicle on the viewer 5300, and instruct the viewer 5300 to perform the set image processing on images captured at positions within a range selected by the user on the displayed historical route.

[0188] [Variation 13] A thirteenth modification of the first embodiment will be described. FIG. 26 is a diagram showing an example of a screen after image processing displayed by the display control unit 3811 of the modified example 13 of the first embodiment. In the image display area 5510 of the viewer 5500 in the modified example 13, an image captured by the image capturing device 3100 of the vehicle 3400 on a road where it is expected to be cloudy and then rain is displayed. In addition, in the fog correction area 5511 which is a part of the image display area 5510, the result of fog correction is displayed. Furthermore, the viewer 5500 can be made to display weather information 5512 on the road. Here, the viewer 5500 displays weather information 5512 related to cloudy and then rain. When the weather at the date and time of the image being played is cloudy, the fog correction area 5511 displays the result of fog correction. That is, the image processing on the image is image processing on a part of the image captured by the image capturing device 3100, and the part of the image is an image showing the weather for which weather information at the date and time when the image was captured at the position of the vehicle 3400 having the image capturing device 3100 is set. First, the communication control unit 3812 of the information display terminal 3800 acquires weather information of the route taken by the vehicle 3400 from a weather information providing server (not shown) based on the position information of the vehicle 3400 and the date and time information included in the image captured by the image capturing device 3100. Next, the display control unit 3811 sets the range of image processing based on the operation of the partial action button 5545, the time information, the weather information, and the like. Furthermore, based on the operation of the fog correction button 5540, it is determined that the range is the fog correction area 5511 in which fog correction is performed at a predetermined position. Furthermore, the communication control unit 3812 instructs the server 3700 via the network 3900 to perform image processing on the fog correction area 5511. Next, the image processing unit 3711 executes image processing on the fog correction area 5511. Furthermore, the communication control unit 3812 acquires the result of the image processing on the fog correction area 5511. Finally, the display control unit 3811 displays the result of the image processing on the fog correction area 5511. This makes it possible to perform appropriate image processing on a partial area of ​​an image captured based on the weather at the position of the vehicle 3400. Here, the weather information includes not only cloudy and rainy weather, but also sunny, fine, slightly cloudy, haze, dust storm, blizzard, drizzle, sleet, fog, snow, hail, and thunder.

[0189] [Variation 14] A fourteenth modification of the first embodiment will be described. FIG. 27 is a diagram showing an example of a screen after image processing, displayed by the display control unit 3811 of the modification 14 of the first embodiment. The sensor unit 3017 of the modification 14 measures acceleration information of the vehicle 3400. That is, the vehicle information of the modification 14 includes the acceleration information measured by the sensor unit 3017. The image display area 5610 of the viewer 5600 in the modification 14 displays an image of the vehicle 3400 that is accelerating rapidly, captured by the image capture device 3100. At this time, the acceleration graph display area 5620 is a graph showing a large upward inclination of the X-axis, and the speed graph display area 5670 is a graph showing a rapid increase in speed. In addition, the viewer 5600 may be provided with an acceleration display unit 5621 such as "1G or more detected". In addition, the acceleration correction area 5611, which is a part of the image display area 5610, displays the result of acceleration correction. Furthermore, when the acceleration sensor of the vehicle 3400 detects 1 G or more, the result of the acceleration correction is displayed in the acceleration correction area 5611. That is, the image processing for the image is image processing for a part of the image captured by the image capturing device 3100, and the part of the image is an image in which the sensor information of the vehicle 3400 having the image capturing device 3100 is within a set threshold value. First, the communication control unit 3812 of the information display terminal 3800 acquires image information and acceleration information of the vehicle 3400 from the server 3700 via the network 3900. Next, the display control unit 3811 sets the range of image processing based on the operation of the partial processing button 5645, the acceleration information of the vehicle 3400, and the like. Furthermore, based on the operation of the acceleration correction button 5640, it is determined that the range is the acceleration correction area 5611 in which acceleration correction is performed at a predetermined position. Next, the communication control unit 3812 instructs the server 3700 via the network 3900 to perform image processing on the acceleration correction area 5611. Furthermore, the image processing unit 3711 executes image processing on the acceleration correction area 5611. Next, the communication control unit 3812 acquires the result of image processing on the acceleration correction area 5611. Finally, the display control unit 3811 displays the result of image processing on the acceleration correction area 5611. This makes it possible to appropriately perform image processing on a partial area of ​​an image captured based on sensor information of the vehicle 3400. Here, the sensor information includes, in addition to the acceleration of the vehicle 3400, information such as the speed of the vehicle 3400 (vehicle speed), engine RPM, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air flow rate (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of remaining fuel in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on / off)), brake opening, steering wheel rotation steering angle, gear position, and door open / closed status information. The automatic correction may be performed by combining the corrections in the above-described modified examples 10 to 14. The combination of the corrections in the modified examples 10 to 14 may be set as an OR condition or as an AND condition.

[0190] [Variation 15] A fifteenth modification of the first embodiment will be described. 28 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit 3811 of Modification 15 of Embodiment 1. In the image display area 5710 of the viewer 5700 in Modification 15, an image captured by the imaging device 3100 of the vehicle 3400 is displayed. The display control unit 3811 controls to set an automatic fog correction area 5711 as a range for correction based on a preset setting, and to set a manual fog correction area 5713 as a range for correction based on a user operation. The display control unit 3811 also controls to display a processing content display area 5712 such as "automatic processing" for the automatic fog correction area 5711, and to display a processing content display area 5714 such as "manual processing" for the manual fog correction area 5713. That is, the screen includes information on whether the range for image processing was selected automatically or manually. This makes it possible to distinguish between a manually set range and an automatically set range for the executed image processing.

[0191] [Variation 16] A sixteenth modification of the first embodiment will be described. 29 is a diagram showing an example of a screen displayed by the display control unit 3811 of the modified example 16 of the first embodiment. In the video display area 5810 of the viewer 5800 in the modified example 16, an original image display unit 5811 before image processing, a processing 1 result display unit 5813 after image processing, a processing 2 result display unit 5815 after image processing, and a processing 3 result display unit 5817 after image processing are displayed as different streams. On each display unit, an image captured by the imaging device 3100 of the vehicle 3400 is displayed. That is, the screen includes an image captured by the imaging device 3100 and one or more images after image processing. The display control unit 3811 controls to display a plurality of images by selecting a stream display button 5845. Here, the display control unit 3811 may display a plurality of images in a chronological order. Furthermore, here, the display control unit 3811 controls to display an original display unit 5812 such as "original" for the original image display unit 5811, to display a processing 1 content display unit 5814 such as "processing 1" for the processing 1 result display unit 5813, to display a processing 2 content display unit 5816 such as "processing 2" for the processing 2 result display unit 5815, and to display a processing 3 content display unit 5818 such as "processing 3" for the processing 3 result display unit 5817. This allows images before and after image processing to be displayed for comparison. Moreover, the display control unit 3811 displays the original image display unit 5811, the processing 1 result display unit 5813, the processing 2 result display unit 5815, and the processing 3 result display unit 5817 as a plurality of streams. Furthermore, the display control unit 3811 displays the original image display unit 5811, the processing 1 result display unit 5813, the processing 2 result display unit 5815, and the processing 3 result display unit 5817 as a selection target for writing or playing. At this time, the display control unit 3811 can control the selected display unit to play or to output as an image file. In the sixteenth modification, the display control unit 3811 opens a plurality of candidate video windows, the processing 1 result display unit 5813, the processing 2 result display unit 5815, and the processing 3 result display unit 5817, and plays them in time synchronization with the original image display unit 5811, which is the source video window. The display control unit 3811 accepts the selection of the best one from among them by the user's operation of a selection button or the like. The storage control unit 3813 stores the selection result and the relationship with the information of the original video in the storage unit 3820. Furthermore, the storage control unit 3813 trains the learning model to learn the user's image processing tendency based on the stored selection result and the relationship with the information of the original video, and stores the result in the storage unit 3820. The display control unit 3811 adjusts the image processing based on the result of the machine learning. That is, the display control unit 3811 adjusts the parameters of the image processing based on the learning information regarding the selection of the image after one or more image processes. This makes it possible to learn the selected image processing result and perform subsequent image processing more appropriately. The display control unit 3811 may also perform image processing on a part of the image.

[0192] [Variation 17] A seventeenth modification of the first embodiment will be described. 30 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit 3811 of Modification 17 of Embodiment 1. In the image display area 5910 of the viewer 5900 in Modification 17, an image captured by the imaging device 3100 of the vehicle 3400 is displayed. When the defog button 5940 is selected, the display control unit 3811 controls the image processing results of defogging to be displayed in the video display area 5910. The display control unit 3811 also controls the image display area 5910 to display a processing content display unit 5911 such as "Defog in progress." That is, the screen includes information on what kind of image processing has been performed on the image captured by the imaging device 3100. This makes it possible to check what kind of image processing has been performed.

[0193] [Variation 18] An eighteenth modification of the first embodiment will now be described. 31 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 18 of Embodiment 1. In the playlist display area 6080 of the viewer 6000 in Modification 18, recording information in the storage unit 3720 or the storage unit 3820 is displayed. The storage control unit 3813 stores information on what kind of image processing was performed on the image captured by the imaging device 3100 in the storage unit 3820 as log information. At this time, the storage control unit 3813 can control the log information to be recorded separately into static information that does not change during shooting and dynamic information that changes depending on the video state during shooting. More specifically, the storage control unit 3813 controls the recording of one static information for a file and one dynamic information at least at the change point, preferably across multiple frames such as all frames. In addition, in the eighteenth modification, the server 3700 may be provided with a function to record according to a user's selection of whether to record with or without fog removal processing, and a function to record information that can determine which was selected in the viewer. The storage control unit 3713 of the server 3700 can perform the same processing on the storage unit 3720. The display control unit 3811 can control the reading and processing of these log information at different timings. The storage control unit 3813 may be configured to record drive recorder parameters related to image quality, such as parameters of the drive recorder's image sensor and ISP (described later), in association with the drive recorder's video data captured with those parameters. In this case, the display control unit 3811 may perform playback using those parameters when playing back on the viewer or the drive recorder. The display control unit 3811 may have a function of switching between playing back without using those parameters when playing back using those parameters. At this time, the display control unit 3811 can display information about what kind of image processing has been performed on the image captured by the image capturing device 3100. That is, the display control unit 3811 controls to display what kind of image processing has been performed and at what timing on the image captured by the image capturing device 3100, as in the playlist 6081. This makes it possible to confirm what kind of image processing has been performed. Furthermore, when playing the playlist 6081, the display control unit 3811 may display a selection as to whether to play the image before image processing or to play the image after image processing based on the log information. Furthermore, at this time, the display control unit 3811 may be configured to convert the image after image processing to the image before image processing based on the log information and output it to the screen. At this time, the image processing unit 3711 performs image processing so as to return the image to the original image based on the log information related to the image processing in the storage unit 3720 and the image after image processing. Next, the communication control unit 3812 acquires the image that has been returned to the original image. Furthermore, the display control unit 3811 displays the image that has been returned to the original image. This allows the image after image processing to be returned to the original image before image processing.

[0194] [Variation 19] A nineteenth modification of the first embodiment will be described. 32 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit 3811 of Modification 19 of Embodiment 1. In the image display area 6110 of the viewer 6100 in Modification 19, an image captured by the imaging device 3100 of the vehicle 3400 traveling in fog is displayed. The image processing unit 3711 of the server 3700 analyzes the image and detects fog. Next, the communication control unit 3712 transmits the fog detection result to the information display terminal 3800. After that, the display control unit 3811 controls to display the detection information display unit 6111 such as "Fog detected" based on the information that fog is detected. Also, at this time, the display control unit 3811 controls to display the fog correction button 6140 based on the information that fog is detected. That is, the display control unit 3811 changes the buttons included in the screen according to the object detected from the image. This makes it possible to change the method of image processing based on the object in the image.

[0195] [Variation 20] A twentieth modification of the first embodiment will now be described. FIG. 33 is a diagram showing an example of a screen displayed by the display control unit 3811 of the modification 20 of the first embodiment. The viewer 6200 in the modification 20 includes an image processing adjustment area 6290. The display control unit 3811 accepts adjustment of image quality parameters by a user's operation of the image processing adjustment area 6290. The image processing adjustment area 6290 includes a hue adjustment object 6291, a brightness adjustment object 6292, a contrast adjustment object 6293, a saturation adjustment object 6294, a gamma value adjustment object 6295, a sigma value adjustment object 6296, a radius adjustment object 6297, and a fluctuation amount adjustment object 6298. When the hue adjustment object 6291 is selected, image processing for adjusting the hue of the image is executed. When the brightness adjustment object 6292 is selected, image processing for adjusting the brightness of the image is executed. When the contrast adjustment object 6293 is selected, image processing for adjusting the contrast of the image is executed. Moreover, by selecting a saturation adjustment object 6294, image processing is performed to adjust the saturation of the image. Furthermore, by selecting a gamma value adjustment object 6295, image processing is performed to adjust the gamma value of the image. Furthermore, by selecting a sigma value adjustment object 6296, image processing is performed to adjust the sigma value of the image. Furthermore, when processing is performed on a part of the image, by selecting a radius adjustment object 6297, image processing is performed to adjust the boundary between the area where the image processing is performed and the area where the image processing is not performed. By selecting a variation adjustment object 6298, image processing is performed to adjust the number of images per unit time. That is, the screen includes an adjustment object that adjusts the degree of image processing. In the modification 20, each adjustment object is displayed in the form of a slider, but in addition to the slider, a radio button, a check box, a pull-down menu, or the like may be used. When the display control unit 3811 receives an operation of an operation unit provided on these adjustment objects, the display control unit 3811 adjusts the degree of image processing in accordance with the received operation. For example, in the modification 20, when the hue adjustment object 6291 is moved to the left end, the display control unit 3811 requests the image processing unit 3711 to perform image processing to set the hue to 0. When the hue adjustment object 6291 is moved to the center, the display control unit 3811 requests the image processing unit 3711 to perform image processing to set the hue to 50. When the hue adjustment object 6291 is moved to the right end, the display control unit 3811 requests the image processing unit 3711 to perform image processing to set the hue to 100. The same applies to the brightness adjustment object 6292, the contrast adjustment object 6293, the saturation adjustment object 6294, the gamma value adjustment object 6295, the sigma value adjustment object 6296, the radius adjustment object 6297, and the fluctuation amount adjustment object 6298. That is, the display control unit 3811 adjusts the degree of image processing in response to the operation of the adjustment object, thereby making it possible to adjust the degree of image processing such as brightness and darkness. The adjustment object can be configured to adjust the image while recording or playing it. At this time, the storage control unit 3813 controls to store the timing of the adjustment object operation in association with the timing of recording the image or the timing of playing the image. This makes it possible to know at what timing the degree of image processing was adjusted. Furthermore, the storage control unit 3813 stores the adjustment result and the relationship with the information of the original video in the storage unit 3820. Furthermore, the storage control unit 3813 trains the learning model to learn the user's image processing tendency based on the stored adjustment result and the relationship with the information of the original video, and stores the result in the storage unit 3820. The display control unit 3811 adjusts the image processing based on the result of this machine learning. That is, the display control unit 3811 adjusts the degree of image processing based on learning information of the degree of adjustment of past image processing. This allows the display control unit 3811 to learn the adjustment result of the degree of image processing and perform subsequent image processing more appropriately. Furthermore, the display control unit 3811 may perform image processing on a part of the image. Furthermore, the display control unit 3811 may be configured to be able to instruct processing in the viewer to refrain from further correction processing for image quality parameters that have been corrected by the drive recorder. On the other hand, the display control unit 3811 may be configured to instruct processing in the viewer to add more correction to image quality parameters that have not been corrected or cannot be corrected by the drive recorder.

[0196] [Variation 21] A twenty-first modification of the first embodiment will now be described. 34 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 21 of Embodiment 1. The viewer 6300 in Modification 21 includes an image processing adjustment area 6390. The image processing adjustment area 6390 includes an image fixing instruction section 6391, a sharpness fixing instruction section 6392, a banding noise removal fixing instruction section 6393, and a film graining fixing instruction section 6394. In other words, the screen includes a fixing instruction section that prohibits adjustment by an adjustment object. The fixing instruction section is an example of a fixing instruction object. When a fixing instruction is given to the fixing instruction section that fixes the adjustment object, the display control section 3811 prohibits the change of the operation of the adjustment object. In other words, the display control section 3811 can fix the adjustment of the degree of image processing by operating these fixing instruction sections. For example, in the modification 21, by checking the check box of the image fixing instruction section 6391, the display control section 3811 fixes the adjustment of the degree of "hue", "brightness", "contrast", "saturation", and "gamma" in the image adjustment area. At this time, the display control section 3811 displays the image quality adjustment result of the fixed adjustment object, that is, the display control section 3811 prohibits the adjustment by the adjustment object in response to the operation of the fixing instruction object. This makes it possible to fix the degree of image processing. At this time, the display control section 3811 displays the adjustment of "hue", "brightness", "contrast", "saturation", and "gamma" as if the adjustment is fixed. For example, when a check box for operating the fixed instruction part is checked, the display control unit 3811 displays the color of the operation part of the adjustment object in a lighter color so that it cannot be operated, or hides the operation part of the adjustment object so that it cannot be operated.

[0197] [Variation 22] A twenty-second modification of the first embodiment will now be described. 35 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 22 of the first embodiment. In the viewer 6400 of Modification 22, a file button 6401 is expanded to display options related to image output, such as "Browse file", "Save over", "Save as", "Write processing results", "Write with location information deleted", and "Close". In other words, the screen includes an image output object. When the file button 6401 is selected, the display control unit 3811 controls to display a menu related to file processing such as "view file", "overwrite save", "save as", "write processing result", "write with location information deleted", and "close". When any file processing is selected, the processing is performed. Here, when "write with location information deleted", the control unit 3810 deletes the location information linked to the image information and outputs the image information. In other words, the control unit 3810 deletes the location information of the image and outputs the image in response to an operation of the image output object. This allows an operation to delete the location information linked to the image. Here, when "write with location information deleted" is selected, the control unit 3810 may write out the video for posting with the location information deleted. Here, the video for posting is a video converted into a format for video posting. At this time, the control unit 3810 may automatically set a title, thumbnail, and the like for the video for posting. Furthermore, when "write processing result" is selected, the control unit 3810 outputs the image information without deleting the location information associated with the image information. Here, when "write processing result" is selected, the control unit 3810 may write out the video for posting without deleting the location information. Furthermore, when outputting the image information, the control unit 3810 may be configured to output only the image before image processing, may output the image after image processing, or may be configured to output the image before image processing and the image after image processing at the same time. Furthermore, when using the write function for posting, the control unit 3810 may be configured to have a function of storing the original video and the processed video in one file and uploading it. At this time, the video to be written out may be a stream in which both videos are arranged, or may be a video stream in which both videos are arranged in a time sequence and connected. Furthermore, at this time, the control unit 3810 may write out the video by embedding a display indicating that each video is the "original video" and the "processed video". Furthermore, the control unit 3810 may be configured to have a function of selecting a spatial range into which the video is written out by a rectangle or the like when writing out the video. At this time, it is preferable that image quality correction is performed only within the selected range, and then the image is re-encoded and written out.

[0198] [Variation 23] A twenty-third modification of the first embodiment will now be described. 36 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 23 of the first embodiment. An image captured by the imaging device 3100 of the vehicle 3400 traveling behind another vehicle 6511 is displayed in the image display area 6510 of the viewer 6500 in Modification 23. The image display area 6510 is subjected to HDR processing based on the operation of the HDR button 6541, and thereby includes license plate information 6512 that can identify the other vehicle 6511. When the image display area 6510 contains information that can identify an individual, such as license plate information 6512, the display control unit 3811 controls to display the personal information processing button 6540. Based on the operation of the personal information processing button 6540, the display control unit 3811 controls to display the license plate information 6512 in a masked state. That is, when the image processed image contains information that can identify an individual, the display control unit 3811 requests execution of a masking process on the information that can identify an individual. This makes it possible to perform appropriate image processing so that information that can identify an individual is not included. Here, the information that can identify an individual includes a name, a date of birth, a face, a license plate, information that can be linked to a specific individual, and the like.

[0199] [Variation 24] In the above-described embodiment, for example, as shown in FIG. 12, an example in which buttons corresponding to a plurality of image processes are displayed on the screen of the information display terminal 3800 has been described. However, the information display terminal 3800 may display fewer buttons than the number of the plurality of image processes. The display control unit 3811 may request the image processing unit 3711, etc. to perform image processing in accordance with the set operation corresponding to the button. More specifically, the display control unit 3811 may display, for example, one button on the screen, and when the button is clicked, request the image processing unit 3711, etc. to perform a first image processing, when the button is double-clicked, request the image processing unit 3711, etc. to perform a second image processing, and when the button is triple-clicked, request the image processing unit 3711, etc. to perform a third image processing. According to the twenty-fourth modification, it is possible to provide a screen that enables the user to instruct the image processing that he or she desires with a small number of buttons.

[0200] [Embodiment 2] In the first embodiment, the server 3700 performs image processing and the image is displayed on the information display terminal 3800. In this modification, a case where an information processing terminal 6800 processes an image and displays the image will be described with reference to Figs.

[0201] [System configuration of information processing system 6600] FIG. 37 is a schematic diagram showing a system configuration of an information processing system 6600 according to a modified example of the second embodiment. As shown in FIG. 37, the information processing system 6600 includes a vehicle 3400, a plurality of information processing terminals 6800D, 6800E, and 6800F, and a network 3900. The vehicle 3400 is configured to be able to communicate with the plurality of information processing terminals 6800D, 6800E, and 6800F via the network 3900. More specifically, the vehicle 3400 transmits information held by the vehicle 3400 to the plurality of information processing terminals 6800D, 6800E, and 6800F via the network 3900. The plurality of information processing terminals 6800D, 6800E, and 6800F are configured to be able to communicate with the vehicle 3400 via the network 3900. More specifically, the plurality of information processing terminals 6800D, 6800E, 6800F acquire information of the plurality of information processing terminals 6800D, 6800E, 6800F via the network 3900.

[0202] [Functional configuration of the information processing terminal 6800] Fig. 38 is a schematic diagram showing a functional configuration of an information processing terminal 6800 according to a modified example of embodiment 2. The hardware configuration of the information processing terminal 6800 is similar to the hardware configuration of the information display terminal 3800 of embodiment 1. That is, the control unit of the information processing terminal 6800 executes processing based on a program stored in the storage unit of the information processing terminal 6800, thereby realizing the functional configuration and the like of the information processing terminal 6800 shown in Fig. 38. As shown in FIG. 38, the information processing terminal 6800 includes, as its functional components, a display control unit 6811, a communication control unit 6812, a storage control unit 6813, and an image processing unit 6814.

[0203] The display control unit 6811 controls information to be output to an output unit (not shown) of the information processing terminal 6800. More specifically, the display control unit 6811 controls to display a screen including information related to an image.

[0204] The communication control unit 6812 controls the exchange of information with the server 3700 via the network 3900 .

[0205] The storage control unit 6813 stores information in the storage unit of the information processing terminal 6800 and retrieves the stored information.

[0206] The image processing unit 6814 executes image processing on the image in the storage unit of the information processing terminal 6800 based on an operation instruction via the screen. This allows the information processing terminal 6800 to execute image processing. In the second embodiment, the image processing unit 6814 will be described as executing various image processing based on a request via the screen of the information display terminal 3800. However, the information processing terminal 6800 may have an image processing unit for each image processing. In addition, instead of the software of the information processing terminal 6800 executing image processing, dedicated hardware (for example, GPU, etc.) of the information processing terminal 6800 may execute image processing. When the dedicated hardware executes image processing, one piece of hardware of the information processing terminal 6800 may be configured to execute multiple image processing, or multiple pieces of hardware of the information processing terminal 6800 corresponding to each of the multiple image processing may be configured to execute image processing.

[0207] According to the second embodiment, image processing can be performed on the information processing terminal side.

[0208] [Embodiment 3] In the third embodiment, an example in which image processing is performed within the image capturing device 3100 of the vehicle 3400 will be described.

[0209] The control unit 3011 records the image captured by the imaging unit 3015 in the storage medium 3500. The control unit 3011 performs image processing on the recorded image using the processor 3111. At this time, the control unit 3011 records in the storage medium 3500 what type of image processing was performed and at what timing. At this time, the control unit 3011 can transmit the information recorded in the storage medium 3500 via the network 3900.

[0210] The control unit 3011 controls the image processing so that a processed image is written out by performing a predetermined operation during playback in the image capturing device 3100. The control unit 3011 performs a predetermined notification during writing out of an image processed in the image capturing device 3100. More specifically, the control unit 3011 controls the display unit 3013 to display a message saying "Please do not turn off the engine" and controls the audio output unit 3014 to output a message saying "Please do not turn off the engine" during writing out of an image processed in the image capturing device 3100. These notifications can be made continuously (repeatedly).

[0211] In the third embodiment, the control unit 3011 may have an image processing function capable of removing fog. In this case, the control unit 3011 removes fog from the target image in response to a user's operation. Next, the control unit 3011 records the image after the fog removal process in response to a user's operation. Furthermore, the control unit 3011 may be configured to reproduce and play the image before the fog removal process. This reproduction may be performed by the control unit 3011 recording the image before the fog removal process in association with the data after the fog removal process, or by recording information indicating the degree of fog removal by the fog removal process in association with the image after the fog removal process and using this information. For example, if there is a large difference between the state seen by the driver and the state after the image correction, the image is recorded in a state different from the state originally seen by the driver, which may be disadvantageous for the driver if he or she causes an accident, but it is possible to grasp the degree of fog removal by the fog removal process.

[0212] The control unit 3011 may also have a function of recording according to the user's selection of whether to record with or without fog removal processing. Furthermore, the control unit 3011 may have a function of displaying this information on the information display terminal. The control unit 3011 may also have a function of recording information that allows the viewer of the information display terminal to determine whether the selection was made between recording with or without fog removal processing. Note that the information may be transferred from the drive recorder to the information display terminal via the network 3900, or may be transferred using an information recording medium such as an SD card.

[0213] According to the third embodiment, image processing can be performed on the vehicle side.

[0214] [Embodiment 4] Next, a fourth embodiment will be described. The fourth embodiment may be implemented by the information processing system 3000 shown in Fig. 1. Fig. 39 is a functional block diagram realized by the storage unit 22 and the control unit 23. The control unit 23 shown in Fig. 39 may be, for example, a control unit 3810 provided in the information display terminals 3800A, 3800B, 3800C, etc. The storage unit 22 shown in Fig. 39 may be, for example, a storage unit 3820 provided in the information display terminals 3800A, 3800B, 3800C, etc.

[0215] The storage unit 22 stores various information defined by the above description. This may be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the information display terminal 3800 executed by the control unit 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the calculation of the program. It may also be a combination of these. In particular, the storage unit 32 may store, for example, a first program P1 and a second program P2 that serve as a viewer for image data IM as various programs related to the information display terminal 3800 executed by the control unit 33.

[0216] The control unit 23 may function as an acquisition unit 231, a reception unit 232, a correction unit 233, a release unit 234, and a display control unit 235 by executing various programs stored in the storage unit 22. That is, information processing by software stored in the storage unit 22 may be specifically realized by the control unit 23, which is an example of hardware, and may be executed as each functional unit included in the control unit 23. Each functional unit related to the control unit 23 will be described below.

[0217] The acquisition unit 231 may be configured to acquire various information. For example, the acquisition unit 231 may acquire specific information IF1 related to a state or a situation of the moving object as an acquisition step, which will be described in further detail later.

[0218] The receiving unit 232 may be configured to receive various information. For example, the receiving unit 232 may receive a first input by the user as a first receiving step. The receiving unit 232 may receive a second input by the user as a second receiving step. This will be described in further detail later.

[0219] The correction unit 233 may be configured to execute a correction process. For example, as a first correction step, the correction unit 233 may execute a first correction process on image data IM generated by the imaging device 3100 of the moving object at a time corresponding to the specific information IF1 based on the specific information IF1. The first correction process may include an automatic correction process by a program such as artificial intelligence. As a second correction step, the correction unit 233 may execute a second correction process on the image data IM based on a second input. The second correction process may include any manual correction process based on, for example, a user input. Furthermore, the correction unit 233 may execute a preparation step and a processing step. This will be described in more detail later.

[0220] The release unit 234 may be configured to release the correction process performed by the correction unit 233. For example, as a release step, the release unit 234 may release the first correction process performed on the image data IM based on a first input. This will be described in further detail later.

[0221] The display control unit 235 is configured to control various visual information to be displayed on a display unit capable of displaying the visual information. Here, the visual information may be information itself generated in a manner that is visible to the user, such as a screen, an image, an icon, text, etc., or may be rendering information for displaying a screen, an image, an icon, text, etc. For example, the display control unit 235 may display the image data IM on which the first correction process has been executed in a reproducible manner as a display control step. This will be described in further detail later.

[0222] [Outline of information processing] Next, an information processing method of the information processing system 3000 according to the fourth embodiment will be described. The information processing method includes steps of an information display terminal 3800, which is an example of an information processing device. Fig. 40 is an activity diagram showing the flow of information processing according to the fourth embodiment. Below, this information processing will be outlined along this activity diagram.

[0223] First, when the moving object is moving, the photographing device 3100 may record the moving state as image data IM (activity A101). The image data IM may be, for example, data in a video format having a predetermined frame rate. The frame rate of the photographing device 3100 is not particularly limited, but may be, for example, 10 fps or more, preferably 30 fps or more, and more preferably 60 fps or more, and specifically, for example, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, or 250 fps, or may be within a range between any two of the numerical values ​​exemplified here. The generated image data IM may be displayed on the display unit 3013 of the photographing device 3100 as a live view. Such recording of image data IM may be performed continuously until the traveling of the moving object is completed.

[0224] Next, the information display terminal 3800 connected to the photographing device 3100 via a network may receive the series of image data IM generated by continuous recording in the activity A101 (activity A102). That is, the reception unit 332 may receive the image data IM from the photographing device 3100. Of course, instead of the network, the reception unit 332 may receive the image data IM via a storage medium such as an SD card (registered trademark). In the following description, for simplicity, the case via a network is exemplified, but the present invention is not necessarily limited to this.

[0225] Next, the acquisition unit 231 may acquire specific information IF1 related to the state or situation of the moving object as an acquisition step (activity A103). Specifically, the control unit 23 may acquire the specific information IF1 by reading out a dedicated program stored in the storage unit 22, and store the specific information IF1 in the storage unit 22. The specific information IF1 may be included in the image data IM received by the reception unit 332, or may be received as separate information from the photographing device 3100 or an external device. Preferably, the specific information IF1 may be at least one of the position, direction, speed, acceleration, time, time zone, weather, running state, and pedal work of the moving object. For example, an appropriate type of correction may be automatically performed for the tunnel position, or different image corrections may be applied for sunny, rainy, and foggy conditions. Also, different image corrections may be applied for morning, night, and evening, or an appropriate correction method may be selected and corrected according to the situation, weather, etc., such as fog, snow, and rain. Optimal parameters for each may be prepared in advance. More specifically, the buttons may be implemented as buttons for correcting images according to the purpose of the image, such as an operation button for erasing a snowstorm, a button for making fog transparent so that the other side of the fog can be seen, a button for erasing rain, etc. Furthermore, as an all-purpose automatic correction, an automatic correction that appropriately combines these may be implemented.

[0226] According to such an embodiment, more appropriate image processing can be performed by taking into account at least one of the specific moving object conditions, namely, position, speed, acceleration, running condition, and pedal work, thereby further improving usability.

[0227] Next, the correction unit 233 may execute a first correction process on the image data IM generated by the image capturing device 3100 of the moving object at the time corresponding to the specific information IF1 based on the specific information IF1 as a first correction step (activity A104). Specifically, the control unit 23 may determine a correction range for the image data IM based on the specific information IF1 by reading out a dedicated program stored in the storage unit 22. That is, preferably, the correction unit 233 may execute a first correction process on the image data IM as a first correction step, by varying the correction range, which is the range to be corrected, based on the specific information IF1. According to such an aspect, since it is not necessary to correct the entire range, the processing load can be suppressed. Also, since the correction is applied to a preferred range, a function with high usability can be realized. Conversely, the correction unit 233 may execute a first correction process on the image data IM by varying the range that should not be corrected based on the specific information IF1. Furthermore, the first correction process may be executed appropriately by varying both the range that should be corrected and the range that should not be corrected.

[0228] Thereafter, the display control unit 235 may display the image data IM on which the first correction process has been performed in a reproducible manner (activity A105) as a display control step. It is preferable that such a display is capable of reproducing the image data IM as video data by displaying the frames of the image data IM in a frame-by-frame manner.

[0229] During the display control by the display control unit 235, the reception unit 232 may receive a first input from the user as a first reception step. The control unit 33 may be configured to be able to switch between the presence and absence of correction processing on the image data IM based on the first input (activity A106). For example, when the correction processing on the image data IM is being performed, the release unit 234 may release the first correction processing on the image data IM based on the first input as a release step. For example, when the correction processing on the image data IM is released, the correction unit 233 may resume the first correction processing on the image data IM based on the first input. According to this aspect, when the correction processing presented by the device is unnecessary, it is possible to increase visibility by releasing it at the user's will, and usability can be further improved. For example, this can be achieved by providing a button that temporarily releases the correction while leaving the correction applied to the range.

[0230] In summary, the information display terminal 3800, which is an example of an information processing device according to this embodiment, includes a control unit 23. The control unit 23 includes the following units. As a first correction step, the correction unit 233 executes a first correction process on image data IM generated by the imaging device 3100 of the moving object at a time corresponding to the specific information IF1, based on the specific information IF1. As a display control step, the display control unit 235 displays the image data IM on which the first correction process has been executed in a reproducible manner. According to this aspect, appropriate image processing can be performed based on the state or situation of the moving object, thereby improving usability.

[0231] [Details of information processing] Next, the details of the information processing outlined above will be described. Fig. 41 is a schematic diagram showing an example of a spatial correction range of the first correction processing. Fig. 42 is a schematic diagram showing an example of a temporal correction range of the first correction processing. That is, the first correction processing may be applied with a limited location range or time range.

[0232] In the embodiment of FIG. 41, image data IM is displayed by the display control unit 235, and is shown as a playback screen 4. The correction range is a region of interest R set in the frame of the image data IM. The playback screen 4 has a correction region 41 and a non-correction region 42. The region of interest R is shown as the correction region 41. Specifically, the correction region 41 may be determined according to various information included in the identification information IF1.

[0233] For example, the size of the correction area 41 may be changed according to the speed and acceleration of the moving object. For example, when the moving object turns left, a correction process may be executed so that the left side of the moving object is visualized in order to prevent a collision accident. The shape and size of the correction area 41 are entirely arbitrary, and may be, for example, rectangular, circular, or elliptical. The shape may be configured to be changeable based on user input. According to such an embodiment, the correction can be applied to a narrow area where visibility is originally desired to be improved, thereby improving usability.

[0234] Also, for example, appropriate correction processing may be performed so that the position of the sun can be grasped from the time or time zone, and the user can easily view it even in a backlit state. In the embodiment of FIG. 42, a correction time 51 is shown in a seek bar 5 that is located below the playback screen 4 and indicates the playback time of the image data IM. The seek bar 5 may correspond to a concept such as a timeline of a video editing software. For example, the correction time 51 is a time zone in which the afternoon sun shines in, and is a time zone in which the shooting device 3100 shoots in backlight due to the traveling direction of the moving object. In other words, the specific information IF1 is the position and orientation of the moving object. As a first correction step, the correction unit 233 may execute a first correction processing on the image data IM based on the position of the sun estimated from the position and orientation. According to such an embodiment, appropriate image processing can be performed taking into account the position of the sun, thereby further improving usability.

[0235] Furthermore, the various correction processes described above may be implemented as an arbitrary manual correction by the user, which is an example of a second correction process. In other words, the reception unit 232 may receive a second input by the user as a second reception step, and the correction unit 233 may execute a second correction process on the image data IM based on the second input as a second correction step. According to this aspect, in addition to the correction process presented by the device, it is possible to improve visibility by adding correction at the user's discretion, and it is possible to further improve usability.

[0236] In relation to the first correction process and the second correction process, the display control unit 235 may display the image data IM in a reproducible manner in which the first correction process and the second correction process can be distinguished. In other words, the manually added range and the automatically added range can be displayed in a distinguishable manner. FIG. 43 is a schematic diagram showing additional information 43 added to the playback screen 4 when the first correction process and the second correction process are executed. As shown in FIG. 43, additional information 43 such as "Automatic correction in progress" or "Manual correction in progress" may be added to the upper right of the playback screen 4. Here, the additional information 43 is a text annotation, but an illustration or an icon may be used instead. According to such an embodiment, the user can understand the type of correction process, which is user-friendly.

[0237] The method of correction processing is not particularly limited, and at least one of brightness, dynamic range, contrast, gamma, saturation, hue, etc. may be appropriately corrected. In addition, it is preferable that the correction is performed based on what is easily visible to the human eye.

[0238] Conversely, image data IM that is difficult for the user to see may be realized. For example, from the viewpoint of protecting personal information, a function may be realized to blur the faces of an unspecified number of people when they are captured. Mosaicking may be performed at least when recording, when a man is recognized, or when a woman is recognized. In addition, a function may be realized to blur the license plates of other cars. The objects to be blurred may be determined by artificial intelligence or other methods. Such a process of blurring is an example of a correction process that makes an image dirty after it has been made clean. Mosaicing is an example of "making it difficult to see" and may be realized by other methods.

[0239] Moreover, this correction process is different from the correction by an ISP (Imaging Signal Processor) that removes aberrations and noises due to optical elements and optical systems, and is a correction process further performed on image data IM obtained through an ISP. Specifically, it may include color corrections such as gamma correction, color emphasis, and color balance, and edge emphasis. Conversely, the original image data IM refers to image data IM obtained through correction by an ISP. Such correction process may be performed after correction by the ISP and before encoding into an appropriate file format such as H264, or may be performed by correcting image data IM that has already been encoded and encoding it again.

[0240] [Variation 1] Next, a first modification of the fourth embodiment will be described. In the first modification, the acquisition unit 231 may acquire various information other than the specific information IF1 from an external device in addition to the specific information IF1. In the first modification, for example, the specific information IF1 may be the position of a moving body. The acquisition unit 231 may acquire road network information IF2 as an acquisition step, and the correction unit 233 may execute a first correction process on the image data IM based on the position and the road network information IF2 as a first correction step. Specifically, the control unit 23 may acquire the road network information IF2 by reading out a dedicated program stored in the storage unit 22, and further execute the first correction process using the road network information IF2.

[0241] The road network information IF2 has various information as attribute data, such as speed limits, road width, weekday 12-hour traffic volume, travel speed (for a certain section, the distance of the section divided by the time required for travel, the travel time including stops due to traffic lights, congestion, etc.), and number of lanes. The correction unit 233 may determine an appropriate type or range of correction based on this information and execute the first correction process. According to such an embodiment, appropriate image processing can be applied depending on the position and the road conditions at that time, thereby further improving usability.

[0242] [Variation 2] Next, a second modification of the fourth embodiment will be described. In the second modification, similar to the first modification, the acquisition unit 231 may acquire various information other than the specific information IF1 from an external device in addition to the specific information IF1. In the second modification, for example, the specific information IF1 may be the position of the moving body. The acquisition unit 231 may acquire topographical information IF3, and the correction unit 233 may execute a first correction process on the image data IM based on the position and the topographical information IF3 as a first correction step. Specifically, the control unit 23 may acquire the topographical information IF3 by reading out a dedicated program stored in the storage unit 22, and further execute the first correction process using the topographical information IF3.

[0243] The topographical information IF3 may be acquired, for example, from an external server related to an organization that issues map data. The topographical information IF3 may be information including information on topography such as urban areas, suburbs, expressways, general roads, gravel roads, mountains, and inside tunnels. The correction unit 233 may determine an appropriate type of correction or correction range based on this information and execute the first correction process. For example, in a downtown area, the influence of neon lights is strong, so a correction process that alleviates this may be executed. For example, in a forest, a correction process that emphasizes the overall brightness may be executed. According to such an embodiment, appropriate image processing can be executed according to the position and the topography and environment at that time, so that usability can be further improved.

[0244] [Variation 3] Next, a third modification of the fourth embodiment will be described. In the third modification, the correction unit 233 may execute a preparation step and a processing step. The control unit 23 may be configured to be able to call a first program P1 stored in advance and a second program P2 different from the first program P1. In particular, the first program P1 may be a player application recommended for playing image data IM, and the second program P2 may be a general-purpose player application having a correction processing function. In other words, since the first program P1, which is a standard viewer, becomes bloated if various functions are incorporated too much, it can be substituted by using free software such as VLC or image processing in a cloud service provided by various companies. FIG. 44 is an activity diagram showing the flow of information processing related to the third modification of the fourth embodiment. Below, this information processing will be outlined along this activity diagram.

[0245] First, when the accepting unit 232 of the information display terminal 3800 accepts an input for playing a video from a user, the control unit 23 may call up the first program P1 stored in the storage unit 22 (activity A201). Next, the control unit 23 may cause the first program P1 to read image data IM (activity A202). Next, as an operation based on the first program P1, the acquiring unit 231 may extract a plurality of parameters PT included in the image data IM and acquire them (activity A203).

[0246] If the second program P2 is not installed in advance in the information display terminal 3800, the installation of the second program P2 may be suggested. In other words, if the second program P2 is not installed, the display control unit 235 may display a screen recommending the installation. The recommendation screen preferably includes the name of the second program P2, and more preferably includes the URL of the installation destination. According to this embodiment, even if the second program is not installed, the first correction process can be performed via the second program.

[0247] Specifically, when the first program P1, which is a PC viewer, is installed, the user may be asked whether to install an external player, and if an instruction to do so is given, the external player may be installed. During installation or immediately after startup, a button to start the external player may be displayed. A warning when using the external player may be displayed. The warning when using the external player may include the following: when a button on the external player is operated, there is a possibility that the display state of the first program P1, which is a PC viewer, may differ from that of the first program P1, which is a PC viewer, and in that case, it is recommended to close the second program P2, which is an external player, once and start the second program P2, which is an external player, again from the first program P1, which is a PC viewer.

[0248] When the second program P2 is installed in advance in the information display terminal 3800, the control unit 23 may call the second program P2 stored in the storage unit 22 (activity A205). That is, an external player start button or the like may be provided on the UI of the first program P1, which is a viewer, and the second program P2, such as VLC, may be started by pressing this button. Then, as an operation based on the second program P2, the acquisition unit 231 may acquire a plurality of parameters PT included in the image data IM (activity A206). Here, the parameters PT may include at least one of hue, brightness, saturation, contrast, gamma, sharpness, playback start time, playback end time, and loop presence / absence. According to such an embodiment, since each parameter is a general parameter that image data has, a more general-purpose image processing program can be used as the second program, and a device with high usability can be realized. Furthermore, as an operation based on the second program P2, the acquisition unit 231 may acquire operation information IF4 related to the operation of the second program P2 (activity A207).

[0249] Specifically, instead of incorporating libvlc.dll etc. into the viewer, the viewer may be launched by passing the video file name and video effect settings, etc. as "command line arguments". vlc.exe 20200101090000.avi --hue=0 --brightness=1.5 --contrast=0.6 --saturation=1.9 --gamma=0.3 --sharpen-sigma=0.9 Also, the playback start time, playback end time, whether or not to loop, etc. may be passed. A parameter for disabling at least a part of the operation section of the second program P2, which is an external player, may be passed. In addition, the second program P2, which is an external player, may be controlled by sending a control command from the first program P1, which is a viewer. For example, if the second program P2 is a VLC player, it may be controlled from the first program P1, which is a viewer, using http, telnet, etc.

[0250] In other words, the first program P1 may execute a preparatory step. Then, the correction unit 233 may extract parameters PT included in the image data IM as the preparatory step, and pass the values ​​of the extracted parameters PT to the second program P2. Also, the correction unit 233 may pass operation information IF4 related to the operation of the second program P2 to the second program P2 as the preparatory step.

[0251] Next, the correction unit 233 may change the value of the parameter PT by executing a first correction process on the parameter PT as a processing step (activity A208). Then, the display control unit 235 may display image data IM based on the value of the parameter PT changed in the processing unit in a reproducible manner. According to this embodiment, a separate program more specialized for the correction process can be used, which contributes to reducing calculation costs and improving image processing performance.

[0252] Preferably, the display control unit 235 may limit at least a part of the functions of the second program P2 based on the operation information IF4 to display the image data IM in a reproducible manner (activity A209). According to this embodiment, the second program can be used only for correction and reproduction, and the user does not need to use unnecessary functions of the second program, thereby realizing a device with high usability.

[0253] [Variation 4] Next, a fourth modified example of the fourth embodiment will be described. In the third modified example described above, when the second program P2 is not installed, a screen recommending the installation of the second program P2 is displayed. In the fourth modified example, the second program P2 can be used via a network without being installed in the information display terminal 3800. In other words, the second program P2 may be a program stored in an external device. The control unit 23 may be configured to call the second program P2 via a network. According to this aspect, even when the second program is not installed, the first correction process can be performed via the second program in the form of a cloud service.

[0254] Specifically, the first program P1 should have a function for storing an ID or the like for automatically logging in to a cloud image sharpening service, an automatic sending function, an uploading function, a function for determining whether the conversion is complete and either automatically downloading the image when it is complete or notifying the user when it is available for downloading, and a function for playing the downloaded image in synchronization with the image before conversion.

[0255] [Variation 5] Next, a fifth modified example of the fourth embodiment will be described. The above-mentioned correction processes are various, and different correction processes may be performed in a superimposed manner. Different correction processes may be performed on one playback screen 4 at the same time, or different correction processes may be performed on different times. It is preferable that different correction processes are determined according to needs and situations.

[0256] In the former case, for example, when an image capturing device 3100 using a fisheye lens is used, the deck board of a moving object is often reflected near the center of the image data IM, and this portion is often unnecessary as information. Therefore, by not performing correction processing on this central portion, the overall processing load can be reduced. Furthermore, since the information in the uncorrected portion has poor visibility, some additional information 43 may be embedded in this position. For example, the original image data IM before correction may be embedded, or a recommended player, etc. may be presented.

[0257] In the latter case, for example, in the case of a traffic accident in which a person rushes out from behind an object, the focus should be placed on the person who gradually emerges from a dimly lit area, and correction processing should be performed so that the user can appropriately recognize the person. Different processing may be performed inside and outside the vehicle.

[0258] [Variation 6] Next, a sixth modified example of the fourth embodiment will be described. A sharpening process may be performed to sharpen a part of an image captured by an imaging device using a fisheye lens (for example, one or both of the imaging devices 3100 and 3200). The sharpening process is an example of a correction process to correct the captured image. In particular, a sharpening process may be performed to sharpen a part of an image captured by an imaging device using a circular fisheye lens (hereinafter referred to as a "circular fisheye image"). In this case, a processing pattern may be adopted in which the circular fisheye image is sharpened as it is and then a shape conversion (geometric conversion) is performed. In particular, the shape conversion may be a process of projecting the image onto a rectangle or the like, in other words, a process of converting the image into a planar image. In this way, the processing can be simplified in a configuration in which multiple locations are extracted from the circular fisheye image and multiple images are displayed on a rectangular screen. Alternatively, when a part is extracted from the circular fisheye image and an image is displayed, and the image is displayed, the following ability to change the extracted area during playback can be improved. In addition, in cases where multiple extraction regions overlap (for example, multiple rectangular regions overlap, such as the shape of a "turn" or the shape of an "eye"), it is preferable to provide a function to reduce the sharpening process of the overlapping parts. In this way, the process can be simplified by the amount of overlapping sharpening process that is reduced.

[0259] In a processing pattern in which a part of a circular fisheye image is extracted and then sharpened after shape conversion, it is preferable not to perform sharpening processing on areas other than the extracted area. In this way, the sharpening processing range is limited, thereby reducing the processing load.

[0260] A part of a circular fisheye image captured by a circular fisheye lens may be cut out, its shape may be transformed (for example, projected into a rectangle, etc.), and a sharpening process may be performed to sharpen a part of the transformed image. This type of processing can be called a double narrowing down of the sharpening range.

[0261] The sharpening process should not be performed on devices such as low-performance PCs. Whether a device is a low-performance PC or the like can be switched automatically or manually by the user. Low performance means that the processing capability of the device is below a certain specification, and in particular, the sharpening process should not be performed on devices such as PCs that are not equipped with a GPU.

[0262] The device may also be provided with a function for exporting video after sharpening processing, and a function for uploading the video for posting to SNS such as YouTube (registered trademark).

[0263] [Variation 7] Next, a seventh modified example of the fourth embodiment will be described. In the above embodiment, the recorded image data IM is described as a moving object, but the image data IM may be recorded as a parking monitoring function of the image capturing device 3100. In this case, the parking monitoring function of the image capturing device 3100 may execute a correction process different from that executed when the object is moving.

[0264] [Embodiment 5] Next, a fifth embodiment will be described. The fifth embodiment may be implemented by the information processing system 3000 shown in Fig. 1. Fig. 45 is a functional block diagram realized by the control unit 33. The control unit 33 shown in Fig. 45 may be, for example, a control unit 3011 provided in the photographing device 3100 mounted on the vehicle 3400. The storage unit 32 shown in Fig. 45 may be, for example, a memory 3112 provided in the photographing device 3100 or the like.

[0265] The storage unit 32 stores various information defined by the above description. This may be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the imaging device 3100 executed by the control unit 33, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the calculation of the program. Also, a combination of these may be used. In particular, the storage unit 32 may store a third program P3 and a fourth program P4 as various programs related to the imaging device 3100 executed by the control unit 33.

[0266] The control unit 33 may function as an acquisition unit 331, a reception unit 332, a recording unit 333, a correction unit 334, and a display control unit 335 by executing various programs stored in the storage unit 32. That is, information processing by software stored in the storage unit 32 may be specifically realized by the control unit 33, which is an example of hardware, and may be executed as each functional unit included in the control unit 33. Each functional unit related to the control unit 33 will be described below.

[0267] The acquisition unit 331 is configured to acquire various types of information.

[0268] The receiving unit 332 is configured to receive various types of information.

[0269] The recording unit 333 is configured to record data on a storage medium. For example, the recording unit 333 records an image captured by a camera as image data IM. This will be described in more detail below.

[0270] The correction unit 334 is configured to perform a correction process.

[0271] The display control unit 335 is configured to control various visual information to be displayed on a display unit capable of displaying the visual information. Here, the visual information may be information itself generated in a manner that is visible to the user, such as a screen, an image, an icon, text, etc., or may be rendering information for displaying a screen, an image, an icon, text, etc. For example, the display control unit 335 may display an image captured by a camera as a live view. This will be described in more detail later.

[0272] Next, an information processing method of the information processing system 3000 according to the fifth embodiment will be described. In the fifth embodiment, the control unit 33 may be configured to be able to call a third program P3 stored in advance and a fourth program P4 different from the third program P3. In particular, the third program P3 may be a player application recommended for playing back image data IM, and the fourth program P4 may be a general-purpose player application. FIG. 46 is an activity diagram showing the flow of information processing according to the fifth embodiment. Below, this information processing will be outlined along this activity diagram.

[0273] While the moving object is traveling, the following activities A301 to A305 are sequentially repeated. First, while the moving object is traveling, the recording unit 333 may record the video captured by the image capturing unit 3015 of the image capturing device 3100 as image data IM (activity A301). Specifically, the control unit 33 may read out a dedicated program stored in the storage unit 32, thereby causing the image data IM to be sequentially stored in the storage unit 32.

[0274] The image data IM may be, for example, data in a video format having a predetermined frame rate. The frame rate of the image capturing device 3100 is not particularly limited, but may be, for example, 10 fps or more, preferably 30 fps or more, more preferably 60 fps or more, specifically, for example, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, 250 fps, or may be within a range between any two of the numerical values ​​exemplified here.

[0275] Next, the processing of activities A302 and A303 for displaying the image data IM on the display unit 3013 of the photographing device 3100, and the processing of activities A304 and A305 for later playing back the image data IM on the information display terminal 3800 are executed in parallel.

[0276] Specifically, the image data IM is recorded, and the first correction process is performed on each frame (activity A302), and the image is displayed as a live view (activity A303). In other words, the display control unit 335 displays the image captured by the camera as a live view. Here, the values ​​of the parameter PT of the recorded image data IM and the displayed image may be set to be different.

[0277] Fig. 47 is a schematic diagram showing the image data IM displayed as a live view and the image data IM recorded so as to be playable later. As shown in Fig. 47, when the recording unit 333 records the image data IM, the recording unit 333 may record the image as image data IM without correcting the image, while the display control unit 335 may correct the image and display it as a live view. According to this embodiment, it is possible to prevent information loss due to processing of the image data to be recorded while improving the visibility of the image viewed as a live view.

[0278] On the other hand, when image data IM displayed as a live view is recorded, the acquisition unit 331 may extract and acquire various parameters PT included in the image data IM (activity A304). Then, the recording unit 333 may record the parameters PT of the displayed image as a parameter log PL in chronological order together with the image data IM (activity A305). According to this embodiment, it is possible to save the visibility of the image viewed as a live view.

[0279] After the moving object has completed its travel, the image data IM may be reproduced via an information display terminal 3800 connected to the imaging device 3100 via a network. When the image data IM is reproduced on the information display terminal 3800, the reception unit 332 of the information display terminal 3800 receives a third input from the user for reproducing the image data IM as a third reception step (activity A306). Subsequently, the third input may trigger the control unit 23 to read out the image data IM (activity A307).

[0280] Next, if the third program P3 is installed in the information display terminal 3800, the control unit 23 may read the parameter log PL as an operation based on the third program P3 (activity A308). Next, as an operation based on the third program P3, the correction unit 233 may refer to the value of the parameter log PL and execute a first correction process to appropriately change the value of each parameter of the original image data IM based on the value of the parameter log PL (activity A309). After that, as an operation based on the third program P3, the display control unit 235 may display the corrected image data IM (activity A311).

[0281] In other words, the display control unit 335 may read out the image data IM and the parameter log PL based on the third input, and display the corrected image data IM in a reproducible manner. According to this embodiment, when the user desires, an image with high visibility can be reproduced on the spot, and a device with high usability can be realized.

[0282] On the other hand, if the third program P3 is not installed in the information display terminal 3800, the control unit 23 may generate additional information 43 recommending playback in the third program P3 as an operation based on the fourth program P4 (activity A310). After that, as an operation based on the fourth program P4, the display control unit 235 may display the additional information 43 superimposed on the image data IM as recorded (activity A311). In other words, when the image data IM is played back in the fourth program P4, the display control unit 335 may display information encouraging playback in the third program P3. According to such an embodiment, even if the image data IM is played back in an unintended program, playback in a preferred program can be induced, and appropriate information can be provided to the user.

[0283] Fig. 48 is a schematic diagram showing an aspect in which additional information 43 encouraging playback of the recommended third program P3 is superimposed on the playback screen 4. As shown in Fig. 48, the information encouraging playback may be displayed together with the playback screen of the image data IM. According to such an aspect, it is possible to appropriately provide the user with information that the current appearance is undesirable, but that there is no problem with the image data itself.

[0284] Alternatively, a QR code (registered trademark) encoding the URL of the third program P3 may be embedded and recorded along with the string of characters, or a video of a screen with this information written on it may be recorded at the beginning of the playback video file.

[0285] In summary, the photographing device 3100, which is an example of an information processing device according to this embodiment, includes a control unit 33. The control unit 33 includes the following units. The recording unit 333 records the image captured by the camera as image data IM. The display control unit 335 displays the image captured by the camera as a live view. Here, the parameters PT of the recorded image data IM and the displayed image are set to be different. According to this embodiment, by making the image viewed as a live view different from the image data to be recorded, both visibility and customizability can be ensured.

[0286] In other words, various parameters PT related to image quality, such as parameters of the image sensor and ISP of the imaging device 3100 such as a drive recorder, can be recorded in association with the video data of the drive recorder, and playback can be performed using these parameters when playing back on the viewer or main unit.

[0287] [Variation 1] Next, a first modification of the fifth embodiment will be described. In the first modification, the image data IM to be recorded is, unlike the fifth embodiment, image data IM that has been subjected to a correction process and has high visibility, substantially similar to that used for live view. In contrast, if it is desired to play back the original image data IM before correction later, the image data IM may be generated by applying a correction process to the image data IM using an information display terminal 3800 or the like. This correction process is preferably an accurate inverse conversion process, but a correction process that is assumed to be an inverse conversion may also be applied.

[0288] [Variation 2] Next, a second modification of the fifth embodiment will be described. In this second modification, two types of parameters PT are recorded. The parameters PT include parameters that are constant throughout the entire time series of image data IM, and parameters that change on a frame-by-frame basis. The former are recorded on an image data IM basis, and the latter are recorded on a frame-by-frame basis of image data IM, thereby making it possible to reduce the processing load and perform correction processing efficiently.

[0289] In other words, the parameters PT are recorded separately as static ones that do not change during shooting, and dynamic ones that change depending on the video condition during shooting, etc. For example, one static one is recorded for each file, and one dynamic one is recorded at least at the change point, preferably across multiple frames such as all frames, and these are then recorded at different times in the viewer. Each may be read and processed separately.

[0290] [others] A program that causes a computer to function as a control unit of an information processing device, for example, the control unit 33 in the image capturing device 3100 or the control unit 23 in the information display terminal 3800, may be provided.

[0291] Since there is a problem that a mechanism is required to switch which of different algorithms such as fog, HDR, and sharpening is to be applied to the currently played video, the following aspect may be adopted for the correction process. 1. Obtain the position and direction of the sun from GPS and change or vary the correction content applied to the image. 2. Using GPS location information and map information, the system determines whether the vehicle is in a city, a forest, etc., and then changes the applicable algorithm. 3. If fog lamp on action is detected, apply fog algorithm. 4. When the contrast difference is small and the image is white, it is determined to be foggy and a fog removal algorithm is applied. 5. When there is a lot of black, it is judged as darkness and made brighter. 6. CLAHE, gamma, and contrast can be varied as a set.

[0292] The camera 3100 may have a function for writing out images that have been subjected to image quality correction by the camera 3100, such as a drive recorder, into a file separate from the recorded images.

[0293] The drive recorder may also have a function for writing out video with image quality correction by performing a specified operation during playback on the drive recorder itself.

[0294] A notification not to turn off the engine while the image with image quality correction is being written out may be continuously given, and in particular it may be repeated by voice.

[0295] The viewer may be provided with an operation unit, such as radio buttons, check boxes, pull-down menus, sliders, etc., that adjusts image quality parameters that can be operated by the user, and while playing back the recorded video, the image quality adjustment results according to the operation of the operation unit are displayed. When a fixation instruction is given to fix the operation unit in the displayed state, such as a check box that gives an instruction to fix the operation unit, changes to the operation of the operation unit are prohibited, and the image quality adjustment results of the fixed operation unit are displayed.

[0296] The viewer may be provided with an operation unit that allows the user to adjust image quality parameters, such as radio buttons, check boxes, pull-down menus, sliders, etc., and may have a function for recording the user's operation of the operation unit in synchronization with the time of the video. The viewer may also have a function for adjusting image quality according to the recorded user operation of the operation unit and playing it back.

[0297] An imaging device 3100 such as a drive recorder or an information display terminal 3800 on which a first program P1 serving as a viewer is installed may be provided with an export function for videos to be posted with the location information deleted, and an export function for videos to be posted without deleting the location information.

[0298] A selection function may be provided that displays the corrected video of the recorded video as multiple streams with different correction states, and allows the user to select the stream to be exported or played back from the multiple streams displayed on the display screen. For example, multiple candidate video windows may be opened and played back in chronological synchronization with the original video window, and the user may select the best one from among them using a selection button or the like.

[0299] The relationship between the user's selection result or selection state and the information of the original video at that time may be used to determine image quality adjustment parameters, etc., for playing another video. Here, the relationship may be recorded, or may be learned by a learning model and then recorded.

[0300] A function may be provided for selecting a spatial range for writing the video using a rectangle or the like when writing the video, and image quality correction may be performed only within the selected range before re-encoding and writing the video.

[0301] For example, it would be good to have a function to export the video for posting to SNS such as YouTube, and a function to upload the original video and the processed video in a single file. Both videos may be streamed side by side, or a video stream in which the two videos are connected one after the other in terms of time. In this case, it would be good to embed a display indicating that each video is the "original video" and the "processed video" before exporting.

[0302] For images captured by a spherical or hemispherical camera, it is advisable to unfold the image, convert a part of it into coordinates, and then process the image. This can reduce the processing load. Alternatively, for example, processing may be performed on a doughnut-shaped portion including the outermost periphery, while no processing or a different processing may be performed on the inner circular region of the doughnut hole.

[0303] Furthermore, the following aspect can also be provided.

[0304] In the information processing device, the screen includes an object, and the control unit receives an operation instruction for the object selected by a user, receives an instruction to display a processing result related to the image corresponding to the operation instruction, and controls the information processing device to display a screen including the processing result related to the image.

[0305] According to this aspect, any image processing corresponding to the manipulation of the object can be executed by the user's selection.

[0306] In the information processing device, the screen includes objects, which differ for each different processing result related to the image, and the control unit accepts an instruction to display the processing result related to the image corresponding to the object selected by a user, and controls the information processing device to display a screen including the processing result related to the image.

[0307] According to this aspect, any image processing corresponding to the object can be executed by the user's selection.

[0308] In the information processing device, when a first object and a second object are selected via the screen, the control unit accepts an instruction to display a first processing result related to the image corresponding to the first object and an instruction to display a second processing result related to the image corresponding to the second object, and controls the information processing device to display a screen including the first processing result related to the image and the second processing result related to the image.

[0309] According to this aspect, a plurality of arbitrary image processes can be executed simultaneously.

[0310] In the information processing device, the image processing related to the first object and the image processing related to the second object have different target areas in the image.

[0311] According to this aspect, a plurality of arbitrary image processes can be simultaneously executed on different regions.

[0312] In the information processing device, the object displayed on the screen is an object related to image processing relating to weather.

[0313] According to this aspect, it is possible to perform appropriate image processing in accordance with the weather.

[0314] In the information processing device, the control unit changes an object to be displayed on the screen based on a state of the image.

[0315] According to this aspect, it is possible to perform appropriate image processing based on information within the image.

[0316] In the information processing device, the control unit controls to display, on the screen, an object suggesting image processing for the image based on road network information.

[0317] According to this aspect, it is possible to perform appropriate image processing based on the road network information.

[0318] In the information processing device, the control unit controls to display on the screen an object suggesting image processing for the image based on information from an acceleration sensor that detects acceleration of a moving body having the shooting unit.

[0319] According to this aspect, it is possible to perform appropriate image processing based on information from the acceleration sensor of the moving body.

[0320] In the information processing device, the control unit controls to display, on the screen, an object suggesting image processing for the image based on position information of a moving body having the shooting unit.

[0321] According to this aspect, it is possible to perform appropriate image processing based on the position information of the moving object.

[0322] In the information processing device, the control unit controls the display of an object on the screen suggesting image processing for the image based on location information of a moving body having the photographing unit and weather information for the date and time the image was photographed.

[0323] According to this aspect, it is possible to perform appropriate image processing based on the position information of the moving object and the weather information for the date and time at that position.

[0324] In the information processing device, the control unit controls to display, on the screen, an object suggesting image processing for the image based on operation information of a moving object having the shooting unit.

[0325] According to this aspect, it is possible to perform appropriate image processing based on operation information of the moving object.

[0326] In the information processing device, the control unit controls to display, on the screen, an object suggesting image processing for the image based on a date and time when the image was captured.

[0327] According to this aspect, it is possible to perform appropriate image processing based on the date and time when the image was captured.

[0328] In the information processing device, the image processing for the image is image processing for a part of the image captured by the imaging unit.

[0329] According to this aspect, it is possible to perform image processing on a part of a captured image.

[0330] In the information processing device, the part of the image is a set area of ​​an entire area of ​​the image captured by the imaging unit.

[0331] According to this aspect, it is possible to set an area in which image processing is performed.

[0332] In the information processing device, the part of the image is an image captured by the imaging unit during a set time period.

[0333] According to this aspect, it is possible to perform appropriate image processing on a partial area of ​​an image captured during a set time period.

[0334] In the information processing device, the part of the image is an image captured within a range in which position information of a moving body having the imaging unit is set.

[0335] According to this aspect, it is possible to perform appropriate image processing on a partial area of ​​an image captured by a moving object moving within a set position.

[0336] In the information processing device, the part of the image is an image showing weather at a location of a moving body having the image capturing unit, the weather information being set for a date and time when the image was captured.

[0337] According to this aspect, it is possible to perform appropriate image processing on a partial area of ​​a captured image based on the weather at the location of the moving body.

[0338] In the information processing device, the portion of the image is an image in which sensor information of a moving body having the imaging unit is within a set threshold.

[0339] According to this aspect, it is possible to perform appropriate image processing on a partial area of ​​an image captured based on sensor information of a moving object.

[0340] In the information processing device, the screen includes information as to whether the range in which the image processing is performed has been automatically selected or manually selected.

[0341] According to this aspect, it is possible to identify whether the scope of the executed image processing has been set automatically or manually.

[0342] In the information processing device, the screen includes an image captured by the imaging unit and an image after one or more image processes.

[0343] According to this aspect, it is possible to display a comparison of before and after image processing.

[0344] In the information processing device, the control unit adjusts parameters of image processing based on learning information regarding selection of the one or more images after image processing.

[0345] According to this aspect, the selected image processing result can be learned, and subsequent image processing can be performed more appropriately.

[0346] In the information processing device, the screen includes an image after image processing, and the control unit stores the image captured by the imaging unit in a storage area.

[0347] According to this aspect, the original image can be stored while the processed image is displayed on the screen.

[0348] In the information processing device, the screen includes information about what kind of image processing has been performed on the image captured by the imaging unit.

[0349] According to this aspect, it is possible to check what kind of image processing has been performed.

[0350] In the information processing device, the control unit stores information about what kind of image processing has been performed on the image captured by the imaging unit in a storage area as log information.

[0351] According to this aspect, it is possible to check what kind of image processing has been performed.

[0352] In the information processing device, the control unit converts the image after image processing into an image before image processing based on the log information and outputs the image to the screen.

[0353] According to this aspect, it is possible to restore an image after image processing to the original image before image processing.

[0354] In the information processing device, the control unit changes an object included on the screen in accordance with an object detected from the image.

[0355] According to this aspect, the method of image processing can be changed based on the object in the image.

[0356] In the information processing device, the screen includes an adjustment object that adjusts a degree of the image processing, and the control unit adjusts the degree of the image processing in response to an operation of the adjustment object.

[0357] According to this embodiment, the degree of image processing such as brightness and darkness can be adjusted.

[0358] In the information processing device, the control unit stores a timing of the operation of the adjustment object and a timing of recording the image or a timing of playing back the image in association with each other.

[0359] According to this embodiment, it becomes possible to know at what timing the degree of image processing has been adjusted.

[0360] In the information processing device, the screen includes a fixed instruction object that prohibits adjustment by the adjustment object, and the control unit prohibits adjustment by the adjustment object in response to an operation of the fixed instruction object.

[0361] According to this aspect, the degree of image processing can be fixed.

[0362] In the information processing device, the control unit adjusts a degree of image processing based on learning information of a past degree of adjustment of the image processing.

[0363] According to this aspect, the adjustment result of the degree of image processing can be learned, and subsequent image processing can be performed more appropriately.

[0364] In the information processing device, the screen includes an image output object, and the control unit deletes position information of the image and outputs the image in response to an operation of the image output object.

[0365] According to this aspect, it is possible to perform an operation to delete the location information linked to the image.

[0366] In the information processing device, when the image processed image contains information that can identify an individual, the control unit requests execution of a mask process on the information that can identify an individual.

[0367] According to this aspect, it is possible to perform appropriate image processing so that information that can identify an individual is not included.

[0368] In the information processing device, the control unit executes image processing based on an operation instruction via the screen.

[0369] According to this aspect, the information processing device can execute image processing.

[0370] An image processing device capable of communicating with the information processing device, the image processing device including a control unit, the control unit executing image processing based on a request from the information processing device.

[0371] A program for causing a computer to function as a control unit of the information processing device.

[0372] Of course, this is not limited to this.

[0373] The scope of the present invention is not limited to the configurations explicitly described in the specification, but includes combinations of various aspects of the present invention disclosed in this specification. The configurations of the present invention that are to be patented are specified in the attached claims, but it is our intention to include configurations disclosed in this specification in the claims in the future, even if they are not currently specified in the claims.

[0374] The present invention is not limited to the configuration described in the above-mentioned embodiment. The components of each of the above-mentioned embodiments and modifications may be arbitrarily selected and combined. Any components of each of the embodiments and modifications may be arbitrarily combined with any components described in the means for solving the invention or any components that embody any components described in the means for solving the invention. We intend to acquire rights to these as well through amendments to this application or divisional applications. Even if there is a description such as "in the case of" or "when", this is not intended to describe a configuration that is limited to that case or time. We also disclose configurations that are not in these cases or times, and we intend to acquire rights. Furthermore, the parts described in order are not limited to this order. We also disclose configurations in which some parts are deleted or the order is changed, and we intend to acquire rights.

[0375] In addition, we intend to obtain rights to the overall design or partial design by converting to a design registration application. The drawings depict the entire device in solid lines, but they include not only the overall design but also partial designs claimed for a part of the device. For example, some parts of the device may be partial designs, and the drawings include some parts of the device as partial designs regardless of the parts. A part of the device may be a part of the device, or a part of that part. We intend to obtain rights to the overall design, as well as partial designs in which any part of the solid line part of the drawing is shown as a broken line part. In addition, the modules, parts, and components inside the device's casing, all of which are shown in the drawings, are subject to trade independently, and we intend to obtain rights to them by converting to a design registration application. [Explanation of symbols]

[0376] 11: Control section 22: Storage section 23: Control section 231: Acquisition Department 232: Reception 233: Correction section 234: Release section 235: Display control unit 32: Storage section 33: Control section 331: Acquisition Department 332: Reception 333: Recording Department 334: Correction section 335: Display control unit 4: Playback screen 41: Correction area 42: Non-correction area 43: Additional information 5: Seek bar 51: Correction time IF1: Specific information IF2: Road network information IF3: Terrain information IF4: Operation information IM: Image data P1: First program P2: Second Program P3: The third program P4: The fourth program PL: Parameter log PT : Parameter R: Region of interest 3000: Information Processing Systems 3011: Control unit 3013: Display section 3014: Audio output unit 3015: Photography Department 3111: Processor 3100: Imaging device 3200: Imaging equipment 3400:Vehicle 3500: Recording media 3700: Server 3710: Control unit 3711: Image processing unit 3712: Communication control section 3713: Memory control unit 3720: Storage section 3730: Communications Department 3800: Information display terminal 3810: Control unit 3811: Display control unit 3812: Communication control unit 3813: Memory control unit 3820: Storage section 3830: Communications Department 3840: Input section 3850: Output section 3900: Network 4100: Playback window 4101: File button 4102: Display button 4103: Play button 4104: Tool button 4105: Information button 4110: Video display area 4120: Acceleration graph display area 4130: Map image display area 4131: Current location icon 4140: Fog compensation button 4141:HDR button 4142: Sharpening button 4143: Other processing buttons 4144: Partial processing button 4145: Stream display button 4146: Operation button 4150: Date and time display area 4160: Vehicle status display area 4170: Speed ​​graph display area 4180: Playlist display area 4200: Playback window 4210: Video display area 4240: Fog compensation button 4300: Playback window 4310: Video display area 4311: Fog correction area 4312: Processing content display section 4313: Sharpening area 4314: Processing content display section 4340: Fog compensation button 4342: Sharpening button 4344: Partial processing button 4400: Playback window 4440: Sunny weather correction button 4441: Cloud correction button 4442: Rain correction button 4443: Fog compensation button 4500: Playback window 4510: Video display area 4540: Fog compensation button 4600: Playback window 4610: Video display area 4630: Map image display area 4631: Current location icon 4640: Tunnel correction button 4700: Playback window 4720: Acceleration graph display area 4721: Acceleration display section 4740: Acceleration correction button 4770: Speed ​​graph display area 4800: Playback window 4810: Video display area 4830: Map image display area 4831: Current location icon 4832: Processing area 4840: Fog compensation button 4900: Playback window 4910: Video display area 4911:Weather information 4941: Rain correction button 5000: Playback window 5010: Video display area 5030: Map image display area 5031: Current location icon 5040: Brake correction button 5070: Speed ​​graph display area 5100: Playback window 5110: Video display area 5140: Night correction button 5150: Date and time display area 5200: Playback window 5210: Video display area 5211: Fog correction area 5240: Fog compensation button 5300: Playback window 5310: Video display area 5311: Night correction area 5340: Night correction button 5350: Date and time display area 5400: Playback window 5410: Video display area 5411: Fog correction area 5430: Map image display area 5431: Current location icon 5432: Processing area 5440: Fog compensation button 5500: Playback window 5510: Video display area 5511: Fog correction area 5512:Weather information 5540: Fog compensation button 5541: Rain correction button 5600: Playback window 5610: Video display area 5611: Acceleration correction area 5620: Acceleration graph display area 5621: Acceleration display section 5640: Acceleration correction button 5700: Playback window 5710: Video display area 5711: Auto fog correction area 5712: Processing content display section 5713: Manual fog correction area 5714: Processing content display section 5740: Fog compensation button 5745: Stream display button 5800: Playback window 5810: Video display area 5811: Original image display section 5812: Original display section 5813: Processing 1 result display section 5814: Processing 1 content display section 5815: Processing 2 result display section 5816: Processing 2 content display section 5817: Processing 3 result display section 5818: Processing 3 content display section 5846: Operation button 5900: Playback window 5910: Video display area 5911: Processing content display section 5940: Fog compensation button 6000: Playback window 6080: Playlist display area 6081: Playlist 6100: Playback window 6110: Video display area 6111: Detection information display section 6140: Fog compensation button 6200: Playback window 6290: Image processing adjustment area 6291: Hue adjustment object 6292: Brightness adjustment object 6293: Contrast adjustment object 6294: Saturation adjustment object 6295: Gamma adjustment object 6296: Sigma value adjustment object 6297: Radius adjustment object 6298: Fluctuation adjustment object 6300: Playback window 6390: Image processing adjustment area 6391: Image adjustment fixed object 6392: Sharpness adjustment fixed object 6393: Banding noise removal adjustment fixed object 6394:Film grain adjustment fixed object 6400: Playback window 6401: File button 6410: Video display area 6500: Playback window 6510: Video display area 6511: Other vehicles 6512: License plate information 6540: Personal information processing button 6541: HDR button 6600: Information Processing Systems 6800: Information processing terminal 6811: Display control unit 6812: Communication control unit 6813: Memory control unit 6814: Image processing unit

Claims

1. A system A control unit is provided, The control unit is configured to execute the following steps: In the recording step, the image captured by the camera is recorded as image data. In the display control step, the image captured by the camera is displayed as a live view. The parameters of the recorded image data and the displayed image are set to be different from each other; In the recording step, the image is recorded as the image data without correction; In the display control step, the image is corrected and displayed as the live view, In the recording step, parameters of the displayed image are recorded in chronological order together with the image data as a parameter log.

2. 2. The system of claim 1, The control unit is configured to further execute a receiving step, In the receiving step, an input by a user is received, The display control step includes reading out the image data and the parameter log based on the input, and displaying the corrected image data in a reproducible manner.

3. 1. An information processing method, comprising: A method comprising the steps of the system according to claim 1 or 2.

4. A program, A computer that functions as a control unit of the system of claim 1 or claim 2.

Images