Information processing device, information processing method, and program

Abstract

To provide a technique for acquiring an image with reduced influence from an environment.SOLUTION: An information processing apparatus according to an embodiment of the present invention is configured to: acquire an image; acquire a pixel value of a luminance reference image and a pixel value of a color reference image that are included in the acquired image; subject luminance adjustment processing to the acquired image in response to the pixel value of the acquired luminance reference image; and subject color adjustment processing to the acquired image in response to the pixel value of the acquired color reference image.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to techniques for adjusting acquired images. [Background technology]

[0002] Patent Document 1 proposes a technology for improving the color reproducibility of an image captured by an on-board camera on a monitor. Specifically, in the technology proposed in Patent Document 1, the on-board camera is positioned so that the vehicle body, which is a reference color carrier, is within the capture area. The on-board device extracts the image area of ​​the reference color carrier from the captured image as a reference image, and performs color correction of the captured image based on the extracted reference image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-8459 A Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present disclosure is to provide a technique for acquiring images with reduced environmental influences, thereby improving inference accuracy, in one example, by using the acquired images in an inference process using a trained machine learning model. [Means for solving the problem]

[0005] An information processing device according to a first aspect of the present disclosure includes a control unit configured to acquire an image, acquire pixel values ​​of a luminance standard image and a color standard image appearing in the acquired image, apply a luminance adjustment process to the acquired image according to the pixel values ​​of the acquired luminance standard image, and apply a color adjustment process to the acquired image according to the pixel values ​​of the acquired color standard image.

[0006] An information processing method according to a second aspect of the present disclosure is an information processing method executed by a computer, and includes acquiring an image, acquiring pixel values ​​of a luminance standard image and a color standard image appearing in the acquired image, applying a luminance adjustment process to the acquired image according to the pixel values ​​of the acquired luminance standard image, and applying a color adjustment process to the acquired image according to the pixel values ​​of the acquired color standard image.

[0007] A program according to a third aspect of the present disclosure is a program for causing a computer to execute an information processing method, the information processing method including acquiring an image, acquiring pixel values ​​of a luminance standard image and a color standard image appearing in the acquired image, applying a luminance adjustment process to the acquired image according to the pixel values ​​of the acquired luminance standard image, and applying a color adjustment process to the acquired image according to the pixel values ​​of the acquired color standard image.

[0008] In each of the above aspects, the image obtained after applying each adjustment process may be used for inference processing of a trained machine learning model generated by machine learning. This is expected to suppress deterioration of the inference accuracy of the trained machine learning model due to influences from the environment (i.e., improve the inference accuracy). The machine learning model has one or more calculation parameters that are adjusted by machine learning. The type of machine learning model is not particularly limited, and may be, for example, a neural network, a support vector machine, a regression model, etc. may be selected from. Effect of the Invention

[0009] According to the present disclosure, it is possible to acquire an image in which the influence of the environment is reduced. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 illustrates an example of a situation to which the present disclosure is applied. [Figure 2A] FIG. 2A is a schematic diagram showing an example of the positional relationship between the imaging device and each reference according to the embodiment, as viewed from the side. [Figure 2B] FIG. 2B is a schematic diagram showing an example of the positional relationship between the imaging device and each reference according to the embodiment, as viewed from above. [Diagram 3] FIG. 3 is a schematic diagram showing an example of an image obtained by the imaging device according to the embodiment. [Figure 4] FIG. 4 is a schematic diagram showing an example of a method for obtaining a reference value corresponding to a color sample according to an embodiment according to the invention in accordance with an illuminance. [Diagram 5] FIG. 5 is a diagram illustrating an example of a hardware configuration of the in-vehicle device according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a software configuration of the in-vehicle device according to the embodiment. [Figure 7] FIG. 7 is a flowchart illustrating an example of a processing procedure of the in-vehicle device according to the embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an example of a processing procedure of a subroutine for brightness adjustment according to the embodiment. [Figure 9] FIG. 9 is a flowchart illustrating an example of a processing procedure of a subroutine for color adjustment according to the embodiment. [Figure 10] FIG. 10 shows a schematic example of a color reference according to a modified example. [Figure 11] FIG. 11 is a diagram illustrating an example of a software configuration of an in-vehicle device according to a modified example. [Figure 12] FIG. 12 is a flowchart showing an example of a processing procedure of an in-vehicle device according to the modified example. [Figure 13] FIG. 13 is a flowchart showing an example of a processing procedure of a distortion correction subroutine according to the modified example. [Figure 14] FIG. 14 is a schematic diagram showing an example of an image obtained by an imaging device according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] According to the conventional method, the color change occurring in the acquired image can be grasped by the color of the reference color carrier reflected in the image. Therefore, by correcting the color of the image based on the image area (reference image) of the reference color carrier, the reproducibility of the color in the image can be improved.

[0012] However, the object of the influence of the environment is not limited to the color of the image. As another example of the influence of the environment, for example, when strong light such as sunlight or high beams is shining, the brightness of the image increases, and the visibility of the image may be impaired. Since it is difficult to deal with the influence of such strong light by color correction alone, it has been difficult to obtain an image in which the influence of the environment is reduced with conventional methods.

[0013] In response to this, an information processing device according to a first aspect of the present disclosure acquires an image, acquires pixel values ​​of a luminance standard image and a color standard image that appear in the acquired image, and performs luminance adjustment processing on the acquired image in accordance with the pixel values ​​of the luminance standard image. and applying a color adjustment process to the captured image in response to pixel values ​​of the captured color reference image.

[0014] In the first aspect of the present disclosure, in addition to color adjustment of an image based on pixel values ​​of a color-reference image, brightness adjustment of an image based on pixel values ​​of a brightness-reference image is performed. The brightness reference may be appropriately configured to provide a reference for brightness in an image. In one example, the brightness reference may be configured with a light source that emits light at a specific brightness and can be observed at a specific brightness if there is no environmental influence. According to this image brightness adjustment process, the pixel values ​​of the brightness-reference image are used to grasp the change in brightness due to the influence of the environment, and the influence acting on the brightness of the image can be reduced. For example, when strong light is shining in, the degree to which the brightness of the image is increased due to the strong light is grasped by the pixel values ​​of the brightness-reference image, and the brightness of the image is reduced accordingly, thereby obtaining an image in which the influence of the strong light is suppressed. Therefore, according to the first aspect of the present disclosure, an image in which the influence of the environment is reduced can be obtained.

[0015] Hereinafter, an embodiment according to one aspect of the present disclosure (hereinafter also referred to as "the present embodiment") will be described with reference to the drawings. However, the present embodiment described below is merely an example of the present disclosure in all respects. Various improvements or modifications may be made without departing from the scope of the present disclosure. In implementing the present disclosure, a specific configuration according to the embodiment may be appropriately adopted. Note that while the data appearing in this embodiment is described in natural language, more specifically, it is specified in pseudo-language, commands, parameters, machine language, etc. that can be recognized by a computer.

[0016] [1 Application example] Fig. 1 is a schematic diagram showing an example of a scene to which the present disclosure is applied. In the present embodiment shown in Fig. 1, a scene is assumed in which the present disclosure is applied to an image 20 obtained by an imaging device S mounted on a vehicle V.

[0017] The in-vehicle device 1 according to the present embodiment is one or more computers that executes luminance adjustment and color adjustment processing on an image 20 acquired by an imaging device S. Specifically, the in-vehicle device 1 acquires an image 20. The in-vehicle device 1 acquires a pixel value 25 of an image of luminance standard 5 and a pixel value 26 of an image of color standard 6 from the acquired image 20. Each pixel value (25, 26) may be referred to as an actual measurement value. The in-vehicle device 1 applies luminance adjustment processing to the acquired image 20 according to the pixel value 25 of the acquired image of luminance standard 5. In addition, the in-vehicle device 1 applies color adjustment processing to the acquired image 20 according to the pixel value 26 of the acquired image of color standard 6. Then, the in-vehicle device 1 outputs the image 20 obtained after applying the luminance adjustment and color adjustment processing. The in-vehicle device 1 according to the present embodiment is an example of an information processing device according to one aspect of the present disclosure.

[0018] As described above, according to this embodiment, in addition to the processing of color adjustment of image 20 based on pixel values ​​26 of the image with color standard 6, processing of brightness adjustment of image 20 based on pixel values ​​25 of the image with brightness standard 5 is also performed. According to this processing of brightness adjustment of image 20, the change in brightness due to environmental influences is grasped from pixel values ​​25 of the image with brightness standard 5, and thus the influence acting on the brightness of image 20 can be reduced. Therefore, according to this embodiment, it is possible to obtain image 20 in which environmental influences are reduced.

[0019] The image 20 may be generated by the imaging device S as appropriate. The installation location of the imaging device S may be determined as appropriate depending on the embodiment. In this embodiment, the imaging device S is mounted on a vehicle V. In the example of FIG. 1, the imaging device S is disposed at the rear of the vehicle V, and the imaging direction of the imaging device S is directed toward the rear of the vehicle V. As a result, the imaging device S can capture the situation behind the vehicle V. The imaging device S can capture an image of the vehicle V. However, when the imaging device S is mounted on the vehicle V, the installation location and imaging direction of the imaging device S need not be limited to this example. In another example, the imaging device S may be disposed in another location, such as the front, side, or ceiling of the vehicle V. The imaging direction of the imaging device S may be oriented in another direction, such as the front or side of the vehicle V. The imaging device S may also be installed inside or outside the vehicle.

[0020] The luminance standard 5 may be appropriately configured so that it is observed in a predetermined luminance state in the image 20 unless it is influenced by the environment. The luminance adjustment process may be configured by any process that adjusts the luminance of the image 20 in a direction in which the image of the luminance standard 5 approaches the predetermined state when the luminance standard 5 is not observed in the predetermined state in the image 20. In other words, the luminance adjustment process may be a process that reduces the deviation when the pixel value 25 of the image of the luminance standard 5 deviates from a predetermined value. Reducing the deviation may include eliminating the deviation.

[0021] In one example, the brightness standard 5 may be composed of one or more brightness samples that provide a brightness standard. The number of brightness samples may be appropriately determined depending on the embodiment. Applying the brightness adjustment process according to the pixel value 25 of the image of the brightness standard 5 may include comparing the pixel value of the image of the brightness sample with a reference value corresponding to the brightness sample, and reducing the brightness of the image 20 if the pixel value of the image of the brightness sample exceeds the reference value corresponding to the brightness sample as a result of the comparison. In this case, applying the brightness adjustment process may further include omitting the process of reducing the brightness of the image 20 and subjecting the image 20 to the next process as it is if the pixel value of the image of the brightness sample is equal to or less than the reference value corresponding to the brightness sample as a result of the comparison. This simplifies the brightness adjustment process and reduces the processing cost of image processing.

[0022] The luminance sample may be configured with a light source that emits light at a specific brightness, such as an LED (light emitting diode), and can be observed at a specific brightness if there is no influence from the environment. However, the material of the luminance sample may not be limited to such an example, and may be appropriately selected according to the embodiment. If the luminance sample can be observed at a specific brightness if there is no influence from the environment, the luminance sample may be configured with a material other than the light source. A reference value corresponding to the luminance sample may be appropriately given according to the brightness of the luminance sample. Also, if the brightness of the luminance sample when the image 20 is acquired can be grasped, the brightness of the luminance sample does not need to be fixed to a constant value, and may be changed at any timing. In one example, the brightness of the luminance sample may be changed according to the brightness of the surroundings of the luminance sample. For example, the luminance sample may be brightened when the surroundings of the luminance sample are bright, and the luminance sample may be darkened when the surroundings of the luminance sample are dark.

[0023] The color standard 6 may be appropriately configured so that it is observed in a predetermined color state in the image 20 unless it is influenced by the environment. The color adjustment process may be configured by any process that adjusts the color of the image 20 in a direction in which the image of the color standard 6 approaches the predetermined state when the color standard 6 is not observed in the predetermined state in the image 20. In other words, the color adjustment process may be a process that reduces the deviation when the pixel value 26 of the image of the color standard 6 deviates from a predetermined value. As with the brightness adjustment process, reducing the deviation in the color adjustment process may include eliminating the deviation.

[0024] In one example, the color reference 6 may be composed of one or more color samples that provide a color reference. The number of color samples may be appropriately determined depending on the embodiment. Applying the color adjustment process depending on the pixel values ​​26 of the image of the color reference 6 may include comparing the pixel values ​​of the image of the color sample with a reference value corresponding to the color sample, and adjusting the color of the image 20 based on the difference between the pixel values ​​of the image of the color sample and the reference value if the comparison shows that the pixel values ​​of the image of the color sample are not far from the reference value corresponding to the color sample. In this case, applying the color adjustment process may further include omitting the process of adjusting the color of the image 20 and subjecting the image 20 to a next process as it is if the comparison shows that the pixel values ​​of the image of the color sample are not far from the reference value corresponding to the color sample. This simplifies the color adjustment process and reduces the processing costs of image processing.

[0025] The color sample may be composed of an object (e.g., a sticker, a film, etc.) that is given a specific color, such as red, green, or blue. The color given to the color sample may be selected appropriately depending on the embodiment. As long as the color of the color sample when the image 20 is acquired can be grasped, the color of the color sample does not need to be fixed to a constant value and may be changed at any timing. The material of the color sample may be selected appropriately depending on the embodiment.

[0026] The reference value for the color sample may be appropriately given according to the color applied to the color sample. In one example, when the color applied to the color sample is constant, the reference value for the color sample may be set to a constant value. However, the actual color of the color sample may change depending on how the light hits the color sample. In this case, it may be inconvenient to handle the color sample in its original color. Therefore, in another example, applying the color adjustment process may further include acquiring an illuminance value of the color sample measured by a illuminometer, and acquiring a reference value corresponding to the color sample according to the acquired illuminance value. This can further improve the color reproducibility in the image 20. The illuminometer may be placed at any location where the illuminance of the color sample can be measured. In one example, the illuminometer may be placed near at least one of the imaging device S and the color reference 6.

[0027] In this embodiment, the color reference 6 may be composed of a plurality of color samples. In this case, in one example, each color sample may be assigned a different color. Assigning different colors to each color sample may include that the color of some color samples is different from the color of other color samples, and the same color is assigned to two or more color samples. This allows the influence of the environment to be grasped based on the plurality of colors, and the color of the image 20 can be adjusted with high precision. As an example of this embodiment, the color reference 6 may be composed of three or more color samples. In this case, the three or more color samples may include three color samples each assigned with the colors of red (R), green (G) and blue (B) or cyan (C), magenta (M) and yellow (Y). That is, the three or more color samples may include three color samples each corresponding to each primary color. However, when the color reference 6 includes a plurality of color samples, the color scheme for each color sample may not be limited to such an example. In another example, the same color may be assigned to each color sample.

[0028] The brightness reference 5 and the color reference 6 may be appropriately positioned so as to appear in the image 20 (i.e., within the imaging range of the imaging device S). In one example, the brightness reference 5 and the color reference 6 may be fixed at a predetermined position relative to the imaging device S so that the positions of the images of the brightness reference 5 and the color reference 6 in the image 20 are constant. This simplifies the image processing for identifying the images of the brightness reference 5 and the color reference 6, and reduces the cost of the processing for acquiring each pixel value (25, 26). However, the positioning of the brightness reference 5 and the color reference 6 is not limited to this example. In another example, either one of the brightness reference 5 and the color reference 6 may be provided so as to be displaced relative to the imaging device S.

[0029] A method of installing the brightness standard 5 and the color standard 6 on the imaging device S may be appropriately selected depending on the embodiment. In one example, an attachment may be attached to the imaging device S. The attachment may be configured to extend forward of the imaging device S, and may be appropriately attached to, for example, a lens, a housing, or the like of the imaging device S. The brightness standard 5 and the color standard 6 may be provided on this attachment so as to be within the imaging range of the imaging device S. In another example, when the imaging device S is mounted on a vehicle V, the brightness standard 5 and the color standard 6 may be provided on a component of the vehicle V (e.g., a vehicle body, etc.) so as to be within the imaging range of the imaging device S.

[0030] In addition, a part of the image 20 may be cut out and subjected to the output process. That is, an output area to be subjected to the output process may be set in the image 20, and the area other than the output area may be output. The output area may not be subjected to the output processing. The range and position of the output area may not be particularly limited and may be appropriately determined according to the embodiment. In this case, in one example, the brightness standard 5 and the color standard 6 may be arranged so that the image of the brightness standard 5 and the image of the color standard 6 are formed in an area other than the output area within the imaging range of the imaging device S. This makes it possible to prevent the image of the brightness standard 5 and the image of the color standard 6 from being output. However, the output form of the image 20 may not be limited to such an example. In another example, at least one of the image of the brightness standard 5 and the image of the color standard 6 may be included in the output area and output together with the image 20. Alternatively, the entire area of ​​the image 20 may be subjected to the output processing.

[0031] The dimensions of each of the luminance standard 5 (luminance sample) and the color standard 6 (color sample) may be suitably determined such that in the image 20, an image of the luminance standard 5 and an image of the color standard 6 each include one or more pixels. If the image of the luminance standard 5 (luminance sample) includes a plurality of pixels, the pixel value 25 may be obtained in any manner, such as, for example, the average, weighted average, mode, median, percentile, value of a predetermined pixel, etc. Similarly, if the image of the color standard 6 (color sample) includes a plurality of pixels, the pixel value 26 may be obtained in any manner, such as, for example, the average, weighted average, mode, median, percentile, value of a predetermined pixel, etc.

[0032] The content of the output process of the image 20 may be appropriately determined depending on the embodiment. In one example, the image 20 may be output to an output device (e.g., displayed on a display) after brightness adjustment and color adjustment processes are applied.

[0033] In another example, as an output process, the image 20 may be used for inference processing of a trained machine learning model generated by machine learning after brightness adjustment and color adjustment processing are applied. For example, the image 20 is input as target data to the trained machine learning model, and an output corresponding to the result of inferring a solution to a task for the image 20 can be obtained from the trained machine learning model by executing the calculation processing of the trained machine learning model. The inference accuracy of the trained machine learning model may be deteriorated by being influenced by the environment (for example, obtaining the target data for inference in an environment different from the environment in which the training data was obtained). In contrast, in one example of the present embodiment, the inference accuracy of the machine learning model can be expected to be improved by using the image 20 in which the influence from the environment is reduced for the inference processing. Note that the ability acquired by the trained machine learning model may be to perform any inference processing (for example, object detection, etc.) on the image.

[0034] In yet another example, the image 20 may be used as training data for machine learning of a machine learning model after brightness adjustment and color adjustment processing are applied. The ability to be acquired by the machine learning model may be to perform any inference processing on the image. Correct answer labels may be appropriately prepared according to the ability to be acquired by the machine learning model, and a dataset may be generated by associating the prepared correct answer labels with the image 20. In this way, the image 20 with reduced influence of the environment can be used to enable the machine learning model to acquire the ability to perform any inference processing.

[0035] The machine learning model includes one or more calculation parameters for executing the calculation of the inference process, and is adjusted by machine learning. The type of the machine learning model is not particularly limited and may be selected from, for example, a neural network, a support vector machine, a regression model, etc. When a neural network is adopted, the weight of the connection between each node, the threshold value of each node, etc. are examples of the calculation parameters. In the machine learning stage, the values ​​of the calculation parameters are appropriately adjusted (optimized) so that the correct answer to the inference task can be derived from the training data. In the inference stage, the adjusted values ​​of the calculation parameters are used to execute the calculation process on the target data.

[0036] 2A, 2B, and 3 show an example of the brightness standard 5 and the color standard 6. FIG. 2A shows an example of the positional relationship between the imaging device S and each standard (5, 6) according to this embodiment as viewed from the side. FIG. 2B shows an example of the positional relationship between the imaging device S and each standard (5, 6) according to this embodiment as viewed from above. FIG. 3 shows an example of an image 20 obtained by the imaging device S according to this embodiment.

[0037] In the example of Fig. 2A, Fig. 2B and Fig. 3, the luminance standard 5 includes one luminance sample 51. The luminance sample 51 may be constituted by a light source such as an LED. On the other hand, the color standard 6 includes three color samples (61, 62, 63). Each of the color samples (61, 62, 63) may be constituted by, for example, a sticker, a film, or the like. Each of the color samples (61, 62, 63) may be marked with a different color. In one example, each of the color samples (61, 62, 63) may be marked with red, green, and blue, respectively.

[0038] An attachment T is attached to the imaging device S. The attachment T has a portion disposed in front of the imaging device S. The luminance sample 51 and each color sample (61, 62, 63) are disposed in the corresponding portion of the attachment T. In addition, in one example, a certain range on the lower side of the imaging range of the imaging device S is set as the output area 200. The luminance sample 51 and each color sample (61, 62, 63) are disposed above the attachment T with respect to the imaging device S so that the image of the luminance sample 51 and the image of each color sample (61, 62, 63) are formed in an area other than the output area 200. Furthermore, the luminance sample 51 is disposed on the left side, and the three color samples (61, 62, 63) are disposed side by side horizontally on the right side. Thus, in the example of FIG. 3, the brightness sample 51 appears on the upper left side of the output area 200 of the image 20, and the color samples (61, 62, 63) appear side by side on the upper right side of the output area 200 of the image 20.

[0039] In addition, in one example, a luminance meter M is installed below the imaging device S, and the illuminance value of each color sample (61, 62, 63) can be measured by the luminance meter M. Accordingly, in one example, a reference value corresponding to each color sample (61, 62, 63) may be appropriately defined so as to be obtainable according to the illuminance value measured by the luminance meter M.

[0040] Fig. 4 shows a schematic example of a method for obtaining a reference value corresponding to each color sample (61, 62, 63) according to illuminance. In the example of Fig. 4, the reference value according to illuminance is defined as a continuous value and a color solid representation by hue, saturation, and brightness. In this example, during color adjustment processing, the reference value corresponding to each color sample (61, 62, 63) according to illuminance may be obtained by searching the color solid of Fig. 4 based on the original color information and illuminance value attached to each color sample (61, 62, 63).

[0041] The method of defining the reference value according to the illuminance is not limited to the example in FIG. 4, and may be appropriately selected according to the embodiment. In another example, the reference value according to the illuminance may be defined as a discrete value. The reference value according to the illuminance does not have to be defined by a color solid representation. Furthermore, the reference value corresponding to each color sample (61, 62, 63) may be defined by an RGB value.

[0042] [2 Configuration example] [Hardware configuration example] Fig. 5 is a schematic diagram showing an example of a hardware configuration of the in-vehicle device 1 according to the present embodiment. As shown in Fig. 5, the in-vehicle device 1 according to the present embodiment is a computer to which a control unit 11, a storage unit 12, an external interface 13, an input device 14, an output device 15, and a drive 16 are electrically connected.

[0043] The control unit 11 includes a hardware processor, such as a CPU (Central Processing Unit), It includes RAM (Random Access Memory), ROM (Read Only Memory), etc., and stores programs and The control unit 11 (CPU) is an example of a processor resource.

[0044] The storage unit 12 is configured with, for example, a hard disk drive, a solid state drive, etc. The storage unit 12 is an example of a memory resource. In this embodiment, the storage unit 12 stores various information such as a program 81. The program 81 is a program for causing the in-vehicle device 1 to execute information processing (FIGS. 7 to 9 described below) that applies brightness adjustment and color adjustment processing to the image 20. The program 81 includes a series of commands for the information processing.

[0045] The external interface 13 is, for example, a Universal Serial Bus (USB) port, a dedicated port, etc., and is an interface for connecting to an external device. The type and number of the external interfaces 13 may be appropriately determined according to the type and number of the external devices to be connected. In this embodiment, the in-vehicle device 1 may be connected to an imaging device S and a light meter M via the external interface 13.

[0046] The input device 14 is, for example, a device for inputting an operation button or the like. The output device 15 is, for example, a device for outputting a display, a speaker, or the like. A user can operate the in-vehicle device 1 by using the input device 14 and the output device 15. The input device 14 and the output device 15 may be integrally configured by, for example, a touch panel display or the like.

[0047] The drive 16 is a device for reading various information such as programs stored in a storage medium 91. The above-mentioned program 81 may be stored in the storage medium 91. In response to this, the in-vehicle device 1 may acquire the program 81 from the storage medium 91. The storage medium 91 is a medium that accumulates information such as programs by electrical, magnetic, optical, mechanical, or chemical action so that a computer or other device, machine, or the like can read the various information such as the stored programs.

[0048] 5 illustrates a disk-type storage medium such as a CD or a DVD as an example of the storage medium 91. However, the type of the storage medium 91 is not limited to the disk type, and may be other than the disk type. An example of a storage medium other than the disk type is a semiconductor memory such as a flash memory. The type of the drive 16 may be appropriately selected depending on the type of the storage medium 91.

[0049] In addition, regarding the specific hardware configuration of the in-vehicle device 1, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, an FPGA (field-programmable gate array), a GPU (Graphics Processing Unit), etc. At least one of the external interface 13, the input device 14, the output device 15, and the drive 16 may be omitted. The in-vehicle device 1 may be configured with a plurality of computers. In this case, the hardware configurations of the computers may or may not be the same. The in-vehicle device 1 may be any computer that is at least temporarily mounted in the vehicle V and executes information processing. The in-vehicle device 1 may be a computer designed exclusively for the service provided, a general-purpose computer, a mobile phone including a smartphone, a tablet PC (Personal Computer), etc.

[0050] [Software configuration example] 6 is a schematic diagram showing an example of the software configuration of the in-vehicle device 1 according to this embodiment. The control unit 11 of the in-vehicle device 1 loads a program 81 stored in the storage unit 12 onto the RAM, and executes instructions included in the program 81 loaded onto the RAM by the CPU. The in-car device 1 according to the embodiment operates as a computer including an image acquisition unit 111, a value acquisition unit 112, a brightness adjustment unit 113, a color adjustment unit 114, and an output unit 115 as software modules. That is, in this embodiment, each software module of the in-car device 1 is realized by the control unit 11 (CPU).

[0051] The image acquisition unit 111 is configured to acquire an image 20. The value acquisition unit 112 is configured to acquire pixel values ​​25 of an image of luminance standard 5 and pixel values ​​26 of an image of color standard 6 appearing in the acquired image 20 from the image 20. The luminance adjustment unit 113 is configured to apply a luminance adjustment process to the acquired image 20 according to the pixel values ​​25 of the acquired image of luminance standard 5. The color adjustment unit 114 is configured to apply a color adjustment process to the acquired image 20 according to the pixel values ​​26 of the acquired image of color standard 6. The output unit 115 is configured to output the image 20 obtained after applying the luminance adjustment and color adjustment processes.

[0052] In the present embodiment, an example is described in which each software module of the in-vehicle device 1 is realized by a general-purpose CPU. However, some or all of the above software modules may be realized by one or more dedicated processors. Each of the above modules may be realized as a hardware module. Furthermore, with regard to the software configuration of the in-vehicle device 1, modules may be omitted, replaced, or added as appropriate depending on the embodiment.

[0053] [3 Example of operation] 7 is a flowchart showing an example of a processing procedure of the in-vehicle device 1 according to this embodiment. The processing procedure described below is an example of an information processing method according to one aspect of the present disclosure. However, the processing procedure described below is merely an example, and each step may be changed as much as possible. In addition, steps may be omitted, replaced, or added to the following processing procedure as appropriate depending on the embodiment.

[0054] <Step S101> In step S101, the control unit 11 operates as the image acquisition unit 111 and acquires the image 20.

[0055] In this embodiment, the control unit 11 acquires the image 20 generated by the imaging device S mounted on the vehicle V. The control unit 11 may acquire the image 20 directly from the imaging device S. However, the acquisition path of the image 20 is not limited to this example. In another example, the control unit 11 may acquire the image 20 via, for example, a network, a storage medium 91, an external storage device, another computer, etc. After acquiring the image 20, the control unit 11 proceeds to the next step S102.

[0056] <Step S102> In step S102, the control unit 11 operates as the value acquisition unit 112 and acquires from the acquired image 20 the pixel value 25 of the image of the brightness standard 5 and the pixel value 26 of the image of the color standard 6 that appear in the image 20.

[0057] Each pixel value (25, 26) may be appropriately acquired from the image 20. In one example, the brightness reference 5 and the color reference 6 may be fixed at a predetermined position relative to the imaging device S. In this case, the control unit 11 may acquire each pixel value (25, 26) by referring to pixels at a predetermined position corresponding to each reference (5, 6). When the brightness reference 5 is composed of one or more brightness samples, the control unit 11 acquires pixel values ​​of an image of the brightness sample. Similarly, when the color reference 6 is composed of one or more color samples, the control unit 11 acquires pixel values ​​of an image of the color sample. In the example of FIG. 3, the control unit 11 acquires pixel values ​​of the brightness reference 51 and the color reference 6 (61, 62, 63) by referring to pixels at the positions of the brightness sample 51 and each color sample (61, 62, 63). Thus, the pixel values ​​of the image of the luminance sample 51 and the pixel values ​​of the images of the color samples (61, 62, 63) may be obtained. After obtaining each pixel value (25, 26), the control unit 11 advances the process to the next step S103.

[0058] <Step S103> In step S103, the control unit 11 operates as the brightness adjustment unit 113 and applies brightness adjustment processing to the acquired image 20 according to the pixel value 25 of the acquired image of the brightness standard 5.

[0059] The brightness adjustment process may be any process that adjusts the brightness of the image 20 to reduce the deviation of pixel values ​​25 of the image of the brightness standard 5 from a predetermined value. In this embodiment, the brightness standard 5 may be composed of one or more brightness samples (brightness sample 51 in the above example), and the control unit 11 may adjust the brightness of the image 20 based on the result of comparison between the pixel values ​​of the image of the brightness sample and the reference value corresponding to the brightness sample.

[0060] Fig. 8 is a flowchart showing an example of a processing procedure of a brightness adjustment subroutine according to this embodiment. The processing of step S103 according to this embodiment may include the following processing of steps S301 to S304. However, the processing procedure of the subroutine shown in Fig. 8 is merely an example, and each process may be changed as much as possible. Also, for the processing procedure shown in Fig. 8, steps may be omitted, replaced, or added as appropriate depending on the embodiment.

[0061] (Step S301) In step S301, the control unit 11 acquires a reference value corresponding to the luminance sample. The reference value corresponding to the luminance sample may be set appropriately according to the brightness of the luminance sample. In one example, the reference value corresponding to the luminance sample may be given as a setting value in the program 81. In another example, the reference value corresponding to the luminance sample may be acquired from any storage area (e.g., RAM, the storage unit 12, the storage medium 91, an external storage device, another computer, etc.). In yet another example, the reference value corresponding to the luminance sample may be specified by an operator such as a user. After acquiring the reference value, the control unit 11 proceeds to the next step S302.

[0062] (Steps S302 and S303) In step S302, the control unit 11 compares the pixel value (actual measured brightness value) of the brightness sample image with a reference value corresponding to the brightness sample. In step S303, the control unit 11 determines the branch destination of the process according to the result of the comparison in step S302.

[0063] In one example, the control unit 11 may determine that the pixel value exceeds the reference value when the actual measurement value of the luminance indicated by the pixel value is even slightly larger than the reference value, and may determine that the pixel value does not exceed the reference value when the actual measurement value of the luminance indicated by the pixel value is not even slightly larger than the reference value. In another example, the control unit 11 may use a threshold value to determine whether the pixel value exceeds the reference value. That is, the control unit 11 may determine that the pixel value exceeds the reference value when the actual measurement value of the luminance indicated by the pixel value exceeds the sum of the reference value and the threshold value, and may determine that the pixel value does not exceed the reference value when the actual measurement value of the luminance indicated by the pixel value is less than the sum of the reference value and the threshold value. When the actual measurement value of the luminance indicated by the pixel value is equal to the sum of the reference value and the threshold value, the branch destination of the process may be any one. The threshold value may be given appropriately.

[0064] If the comparison result indicates that the actual luminance value indicated by the pixel values ​​of the image of the luminance sample exceeds the reference value corresponding to the luminance sample, the control unit 11 proceeds to the next step S304. On the other hand, if the comparison result indicates that the actual luminance value indicated by the pixel values ​​of the image of the luminance sample does not exceed the reference value corresponding to the luminance sample, the control unit 11 omits the process of step S304 and ends the process of the luminance adjustment subroutine.

[0065] (Step S304) In step S304, the control unit 11 reduces the luminance of the image 20. The amount by which the luminance is reduced may be appropriately determined depending on the embodiment. In one example, the control unit 11 may calculate a difference between a pixel value of the image of the luminance sample and a reference value, and determine the amount by which the luminance is reduced according to the calculated difference. As a specific example, the control unit 11 may determine an amount by which the luminance is reduced so that the actual value of the luminance indicated by the pixel value of the image of the luminance sample becomes equal to or less than the reference value, and reduce the luminance of the image 20 according to the determined amount. The amount by which the luminance is reduced may be determined as a continuous value or as a discrete value. In another example, the amount by which the luminance is reduced may be a constant value.

[0066] When an output area is set within the image 20, the range in which the luminance is reduced may be appropriately determined so as to include at least the output area. Simply put, the control unit 11 may reduce the luminance of the entire image 20. However, the range in which the luminance is reduced is not limited to this example, and may be appropriately determined depending on the embodiment.

[0067] After reducing the luminance of the image 20, the control unit 11 ends the process of the luminance adjustment subroutine. When the process of the luminance adjustment subroutine ends, the control unit 11 advances the process to the next step S104.

[0068] In the example of FIG. 2A, FIG. 2B, and FIG. 3, in step S301, the control unit 11 obtains a reference value corresponding to the luminance sample 51. In step S302, the control unit 11 compares the pixel value of the image of the luminance sample 51 in the image 20 with the reference value corresponding to the luminance sample 51. If the comparison result shows that the pixel value of the image of the luminance sample 51 exceeds the reference value, the control unit 11 reduces the luminance of the image 20 by processing in step S304. In one example, the control unit 11 reduces the luminance of at least the output region 200 of the image 20. On the other hand, if the comparison result shows that the pixel value of the image of the luminance sample 51 does not exceed the reference value, the control unit 11 omits the luminance reduction processing in step S304. Then, the control unit 11 ends the processing of the luminance adjustment subroutine and proceeds to the next step S104.

[0069] <Step S104> Returning to FIG. 7, in step S104, the control unit 11 operates as the color adjustment unit 114 and applies color adjustment processing to the acquired image 20 according to the pixel values ​​26 of the acquired image of the color reference 6.

[0070] The color adjustment process may be any process that adjusts the color of the image 20 to reduce deviations between pixel values ​​26 of the image of the color reference 6 and a predetermined value. In this embodiment, the color reference 6 may be composed of one or more color samples (three color samples 61-63 in the above example), and the control unit 11 may adjust the color of the image 20 based on the results of a comparison between the pixel values ​​of the image of the color sample and the reference values ​​corresponding to the color sample.

[0071] Fig. 9 is a flowchart showing an example of a processing procedure of a subroutine for color adjustment according to this embodiment. The processing of step S104 according to this embodiment may include the following processing of steps S401 to S405. However, the processing procedure of the subroutine shown in Fig. 9 is merely an example, and each step may be changed as much as possible. Furthermore, steps may be omitted, replaced, or added to the processing procedure shown in Fig. 9 as appropriate depending on the embodiment.

[0072] (Step S401) In step S401, the control unit 11 acquires the illuminance value of the color sample measured by an illuminometer. In one example, the control unit 11 may acquire the illuminance value of the color sample directly from the illuminometer. In another example, the control unit 11 may acquire the illuminance value of the color sample directly from the illuminometer. The value of illuminance in the color sample may be acquired via an external storage device, another computer, etc. When the value of illuminance in the color sample is acquired, the control unit 11 advances the process to the next step S402.

[0073] (Step S402) In step S402, the control unit 11 acquires a reference value corresponding to the color sample according to the acquired illuminance value. The reference value corresponding to the color sample may be set appropriately according to the color and illuminance to be applied to the color sample. In one example, the reference value corresponding to the color sample may be given as a setting value in the program 81. In another example, the reference value corresponding to the color sample may be acquired from any memory area (e.g., RAM, the memory unit 12, the memory medium 91, an external memory device, another computer, etc.). In yet another example, the reference value corresponding to the color sample may be specified by an operator such as a user. After acquiring the reference value corresponding to the color sample, the control unit 11 proceeds to the next step S403.

[0074] (Steps S403 and S404) In step S403, the control unit 11 compares the pixel values ​​(actual measured color values) of the image of the color sample with the reference values ​​corresponding to the color sample. In step S404, the control unit 11 determines the branching destination of the process according to the result of the comparison in step S403.

[0075] In one example, the control unit 11 may determine that the pixel value is far from the reference value if the actual measured value of the color indicated by the pixel value is even slightly different from the reference value, and may determine that the pixel value is not far from the reference value if this is not the case (i.e., the actual measured value of the color is equal to the reference value). In another example, the control unit 11 may use a threshold value to determine whether the pixel value is far from the reference value. That is, the control unit 11 may determine that the pixel value is far from the reference value if the difference between the actual measured value of the color indicated by the pixel value and the reference value exceeds the threshold value, and may determine that the pixel value is not far from the reference value if the difference between the actual measured value of the color indicated by the pixel value and the reference value is less than the threshold value. If the difference between the actual measured value of the color indicated by the pixel value and the reference value is equal to the threshold value, the process may branch to any one of the following. The threshold value may be given as appropriate.

[0076] If the comparison shows that the actual measured value of the color indicated by the pixel values ​​of the color sample image is not far from the reference value corresponding to the color sample, the control unit 11 proceeds to the next step S405. On the other hand, if the comparison shows that the actual measured value of the color indicated by the pixel values ​​of the color sample image is not far from the reference value corresponding to the color sample, the control unit 11 omits the process of step S405 and ends the process of the color adjustment subroutine.

[0077] (Step S405) In step S405, the control unit 11 adjusts the color of the image 20 based on the difference between the actual measured value of the color indicated by the pixel value of the image of the color sample and the reference value corresponding to the color sample. A method for adjusting the color of the image 20 based on the difference may be appropriately determined depending on the embodiment. In one example, the control unit 11 may adjust the color of the image 20 so that the pixel value of the image of the color sample approaches the reference value (i.e., reduces the difference). The control unit 11 may adjust the color of the image 20 so that the pixel value of the image of the color sample becomes the reference value.

[0078] When an output region is set within the image 20, the range in which the color is adjusted may be appropriately determined so as to include at least the output region, similar to the range in which the luminance is reduced. Simply, the control unit 11 may adjust the color of the entire image 20. However, the range in which the color is adjusted is not limited to this example, and may be appropriately determined depending on the embodiment.

[0079] After adjusting the color of the image 20, the control unit 11 ends the processing of the subroutine for color adjustment. When the processing of the subroutine for color adjustment ends, the control unit 11 proceeds to the next step S105. Proceed.

[0080] In the example of FIG. 2A, FIG. 2B, and FIG. 3, in step S401, the control unit 11 acquires the illuminance value of each color sample (61, 62, 63) measured by the illuminometer M. In step S402, the control unit 11 acquires a reference value (a total of three reference values) corresponding to each color sample (61, 62, 63) according to the illuminance value. In one example, the control unit 11 may acquire the reference value corresponding to each color sample (61, 62, 63) from the color solid shown in FIG. 4. In step S403, the control unit 11 compares the pixel value of the image of each color sample (61, 62, 63) in the image 20 with the reference value corresponding to each color sample (61, 62, 63). If the comparison result indicates that the pixel value of at least one of the images of the three color samples (61, 62, 63) is far from the reference value, the control unit 11 adjusts the color of the image 20 by processing in step S405. In one example, the control unit 11 adjusts the color of at least the output region 200 of the image 20. On the other hand, if the comparison result shows that all pixel values ​​of the images of the three color samples (61, 62, 63) are not far from the reference value, the control unit 11 omits the color adjustment process in step S405. Then, the control unit 11 ends the processing of the color adjustment subroutine and proceeds to the next step S105.

[0081] <Step S105> 7, the control unit 11 operates as the output unit 115 and outputs the image 20 obtained after applying the luminance adjustment and color adjustment processes. In the example of FIG. 3, the control unit 11 may output the output area 200 of the image 20.

[0082] The output destination and the contents of the output process may be appropriately determined according to the embodiment. In one example, the control unit 11 may output the obtained image 20 to the output device 15 or an output device of another computer. In another example, the control unit 11 may output the image 20 to be subjected to any processing. As a specific example, the control unit 11 may use the image 20 for inference processing of a trained machine learning model. The inference processing of the trained machine learning model may be executed by the in-vehicle device 1, or may be executed by another computer other than the in-vehicle device 1. In addition, the control unit 11 may use the obtained image 20 as training data for machine learning of the machine learning model. The machine learning processing may be executed by the in-vehicle device 1, or may be executed by another computer other than the in-vehicle device 1.

[0083] When the output of the image 20 is completed, the control unit 11 ends the processing procedure of the in-car device 1 according to this operation example. The control unit 11 may repeatedly execute the processes of steps S101 to S105. The timing for repeating the processes may be appropriately determined according to the embodiment. In this way, the in-car device 1 may be configured to repeatedly acquire an image and adjust the brightness and color of the acquired image.

[0084] <Features> In this embodiment, the in-vehicle device 1 performs a process of adjusting the color of the image 20 based on the pixel value 26 of the image of the color standard 6 in step S104, and also performs a process of adjusting the brightness of the image 20 based on the pixel value 25 of the image of the brightness standard 5 in step S103. According to this brightness adjustment process, the pixel value 25 of the image of the brightness standard 5 is used to grasp the change in brightness due to the influence of the environment, and thus the influence acting on the brightness of the image 20 can be reduced. For example, when strong light such as sunlight or high beams is shining, it may be determined in the process of step S303 that the pixel value 25 of the image of the brightness sample 5 exceeds the reference value. In response to this, the process of step S304 can obtain the image 20 in which the influence of this strong light is suppressed. Therefore, according to this embodiment, the image 20 in which the influence of the environment is reduced can be obtained.

[0085] [4 Variations] Although the embodiment of the present disclosure has been described in detail above, the above description is merely an example of the present disclosure in every respect. It goes without saying that various improvements or modifications can be made without departing from the scope of the present disclosure. For example, the following modifications are possible. The following modifications can be combined as appropriate.

[0086] <4.1> In the above embodiment, when the color reference 6 is composed of a plurality of color samples, a frame line may be provided between the plurality of color samples. In this case, the control unit 11 may be further configured to acquire an image of the frame line shown in the image 20, and apply a distortion correction process to the image 20 according to the acquired image of the frame line. The program 81 may further include these instructions. The frame line may be appropriately formed in a predetermined shape so that the presence or absence of distortion in the image 20 can be determined. The shape of the frame line and the arrangement of each color sample relative to the frame line are not particularly limited, and may be appropriately determined according to the embodiment.

[0087] FIG. 10 is a schematic diagram showing an example of a color standard 6A according to this modification. In the above embodiment, the color standard 6 may be replaced with the color standard 6A according to this modification. In the example of FIG. 10, the color standard 6A has a lattice-like frame 69 of straight lines with 3 rows and 5 columns, and a total of 15 windows are configured. Each color sample 65 may be arranged in each window. As a result, the color standard 6A may have 15 color samples 65. In this case, the color to be applied to each color sample 65 may be selected so as to create a gradation in at least one of the row direction and the column direction. Note that the number of rows and columns is not limited to the example of FIG. 10, and may be appropriately changed according to the embodiment. In the following, for convenience of explanation, in this modification, a scene in which the color standard 6A exemplified in FIG. 10 is used is assumed.

[0088] FIG. 11 shows an example of a software configuration of the in-vehicle device 1 according to this modified example. The control unit 11 of the in-vehicle device 1 executes a program 81 further including the above-mentioned instructions, so that the in-vehicle device 1 may be configured to further include a line acquisition unit 116 and a distortion correction unit 117 as software modules. The line acquisition unit 116 may be configured to acquire an image of a frame line 69 appearing in the image 20. The distortion correction unit 117 may be configured to apply a distortion correction process to the image 20 according to the acquired image of the frame line 69. Note that either the line acquisition unit 116 or the distortion correction unit 117 may be realized by one or more dedicated processors. Either the line acquisition unit 116 or the distortion correction unit 117 may be realized as a hardware module.

[0089] Fig. 12 is a flowchart showing an example of a processing procedure of the in-car device 1 according to this modified example. The processing procedure shown in Fig. 12 is an example of an information processing method according to one aspect of the present disclosure. In the processing procedure shown in Fig. 12, the processes of steps S501 and S502 are added between steps S104 and S105 of the processing procedure according to the above embodiment. Except for this point, the processing procedure of this modified example may be the same as that of the above embodiment.

[0090] In step S501, the control unit 11 operates as the line acquisition unit 116 and acquires an image of the frame line 69 appearing in the image 20. In one example, the color reference 6A including the frame line 69 may be fixed at a predetermined position with respect to the imaging device S. In this case, the control unit 11 may acquire the image of the frame line 69 appearing in the image 20 by referring to a pixel at a predetermined position corresponding to the frame line 69.

[0091] In step S502, the control unit 11 applies distortion correction processing to the image 20 according to the acquired image of the frame line 69. The frame line 69 is formed in a predetermined shape (straight line in the example of FIG. 10). Therefore, if no distortion occurs, the frame line 69 appears in the predetermined shape in the image 20. Therefore, the distortion correction processing may be configured by any image processing that corrects the distortion of the image 20 so as to reduce the deviation when the image of the frame line 69 deviates from the predetermined shape.

[0092] Fig. 13 is a flowchart showing an example of a processing procedure of a distortion correction subroutine according to this modified example. The processing of step S502 according to this modified example may include the following processing of steps S701 to S703. However, the processing procedure of the subroutine shown in Fig. 13 is merely an example, and each step may be changed as much as possible. Furthermore, steps may be omitted, replaced, or added to the processing procedure shown in Fig. 13 as appropriate depending on the embodiment.

[0093] In step S701, the control unit 11 determines whether or not the image of the frame line 69 is distorted. Since the frame line 69 is formed in a predetermined shape, if distortion occurs in the image 20, the shape of the frame line 69 reflected in the image 20 will be different from the predetermined shape. Therefore, in one example, the control unit 11 may determine whether or not the image of the frame line 69 is distorted depending on whether or not the shape of the image of the frame line 69 acquired in step S501 is a predetermined shape. That is, the control unit 11 may determine that the image of the frame line 69 is not distorted when the shape of the image of the frame line 69 is a predetermined shape, and may determine that the image of the frame line 69 is distorted when the shape of the image of the frame line 69 is not a predetermined shape.

[0094] In step S702, the control unit 11 determines where the process should branch based on the result of the determination in step S702. If the result of the determination indicates that distortion has occurred in the image of the frame line 69, the control unit 11 advances the process to the next step S703. On the other hand, if the result of the determination indicates that distortion has not occurred in the image of the frame line 69, the control unit 11 omits the process of step S703 and subjects the image 20 to the next process as is. In this modified example, the control unit 11 ends the process of the distortion correction subroutine.

[0095] In step S703, control unit 11 corrects the distortion of image 20 so that the image of frame line 69 has a predetermined shape. When an output area is set within image 20, the range in which distortion is corrected may be appropriately determined so as to include at least the output area. Simply, control unit 11 may correct the distortion of the entire image 20. However, the range in which distortion is corrected is not limited to this example, and may be appropriately determined depending on the embodiment.

[0096] When the distortion of the image 20 is corrected, the control unit 11 ends the processing of the distortion correction subroutine. When the processing of the distortion correction subroutine ends, the control unit 11 proceeds to the next step S105. According to this modification, the distortion of the image 20 can be corrected using the frame line 69. As a result, it is possible to obtain the image 20 in which the influence of the environment is further reduced.

[0097] <4.2> In the above embodiment and modified examples, the processing order of each step included in the information processing of FIG. 7, FIG. 8, FIG. 9, FIG. 12, and FIG. 13 may be changed as appropriate within a range where no contradiction occurs. As an example, the process of acquiring pixel values ​​25 of an image of luminance standard 5 in step S102 may be executed at any timing before executing the process of step S302. The process of acquiring pixel values ​​26 of an image of color standard 6 in step S102 may be executed at any timing before executing the process of step S403. The process of acquiring pixel values ​​25 of an image of luminance standard 5 and the process of acquiring pixel values ​​26 of an image of color standard 6 may be executed at different timings. The process of step S103 and the process of step S104 may be at least partially processed in parallel. The process of step S104 may be executed before the process of step S103. The process of step S301 may be executed at any timing before executing the process of step S302. The process of step S401 may be executed at any timing before executing the process of step S402. The process of step S402 may be executed at any timing before the process of step S403 is executed. The process of step S501 may be executed at any timing before the process of step S701 is executed. The process of step S502 may be executed after the process of step S103 and step S The process of step S502 may be performed at least partially in parallel with at least one of the processes of step S103 and step S104. The process of step S502 may be performed before at least one of the processes of step S103 and step S104.

[0098] <4.3> In the example of Fig. 2B and Fig. 3, the number of luminance samples in the luminance standard 5 is one. However, the number of luminance samples is not limited to one, and may be two or more. That is, the luminance standard 5 may be composed of a plurality of luminance samples.

[0099] Fig. 14 shows an example of an image 20 obtained by the imaging device S when the luminance standard 5 is replaced with a luminance standard 5B having three luminance samples (53, 54, 55). In the example of Fig. 14, the luminance samples (53, 54, 55) are arranged side by side in the horizontal direction. However, the arrangement of the luminance samples (53, 54, 55) is not limited to this example and may be appropriately determined depending on the embodiment.

[0100] As shown in FIG. 14, when the luminance standard 5 includes a plurality of luminance samples, the brightness of each luminance sample may be appropriately determined according to the embodiment. In one example, each luminance sample may be configured to emit light at a different brightness. The luminance samples emitting light at different brightnesses may include some luminance samples emitting light at a different brightness from other luminance samples, and two or more luminance samples emitting light at the same brightness. In one example of FIG. 14, one of the three luminance samples (53, 54, 55) may emit light the brightest, another may emit light in the middle, and the last may emit light the darkest. However, the brightness setting of each luminance sample may not be limited to such an example. In another example, the brightness of each luminance sample may be the same.

[0101] When multiple luminance samples that emit light at different brightnesses are provided, the degree to which the pixel values ​​of the image of each luminance sample are affected by the incoming light varies depending on the brightness of each luminance sample. The weaker the brightness at which the sample emits light, the more susceptible the pixel values ​​of the image of that luminance sample are to the incoming light, and the stronger the brightness at which the sample emits light, the less susceptible the pixel values ​​of the image of that luminance sample are to the incoming light. Therefore, by providing multiple luminance samples that emit light at different brightnesses, it is possible to evaluate the intensity of the light that has entered the imaging range of the imaging device S.

[0102] In this case, the amount by which the luminance of the image 20 is reduced in the process of step S304 may be determined according to the brightness of a luminance sample of which the pixel value of the image exceeds a reference value, among the multiple luminance samples. As an example, the stronger the intensity of the light entering the imaging range of the imaging device S, the greater the effect on the pixel value of the image of the brighter luminance sample (i.e., the pixel value exceeds the reference value). Therefore, the control unit 11 may increase the amount by which the luminance of the image 20 is reduced in the process of step S304, the more the pixel value of the image of the brighter luminance sample exceeds the reference value.

[0103] In the example of FIG. 14, when the pixel value of the image of the brightest luminance sample among the three luminance samples (53, 54, 55) exceeds the reference value, the control unit 11 may greatly reduce the luminance of the image 20 in the process of step S304. When the pixel value of the image of the brightest luminance sample among the three luminance samples (53, 54, 55) does not exceed the reference value and the pixel values ​​of the images of the remaining two luminance samples exceed the reference value, the control unit 11 may moderately reduce the luminance of the image 20 in the process of step S304. When only the pixel value of the image of the dimmest luminance sample among the three luminance samples (53, 54, 55) exceeds the reference value, the control unit 11 may slightly reduce the luminance of the image 20 in the process of step S304. According to this modification, the influence of the environment can be accurately grasped based on a plurality of luminance samples, so that the luminance of the image 20 can be accurately adjusted.

[0104] <4.4> In the example of FIG. 2A, FIG. 2B, and FIG. 3, the luminance sample 51 constituting the luminance standard 5 is disposed near the imaging device S. In this case, the pixel value 25 of the image of the luminance standard 5 is not substantially lower than the reference value. That is, it is sufficient to assume only a scene in which the pixel value of the image of the luminance sample becomes larger than the reference value due to strong light entering. Therefore, the in-vehicle device 1 according to the above embodiment is configured to determine whether or not the pixel value of the image of the luminance sample exceeds the reference value in step S303. This can simplify the luminance adjustment process. However, the luminance adjustment process is not limited to this example. In another example, in the luminance adjustment process, similar to the color adjustment process, the control unit 11 may determine whether or not the pixel value of the image of the luminance sample is far from the reference value. Then, when the pixel value of the image of the luminance sample is far from the reference value, the control unit 11 may adjust the luminance of the image 20 based on the difference between the pixel value of the image of the luminance sample and the reference value.

[0105] <4.5> In the above embodiment, the illuminometer may be omitted. In this case, the process of step S401 may be omitted. In the process of step S402, the control unit 11 may appropriately acquire a reference value corresponding to the color sample, regardless of the illuminance value. The reference value corresponding to the color sample may be a constant value according to the color attached to the color sample.

[0106] 2A, 2B, and 3, the attachment T may be omitted. In this case, the brightness reference 5 and the color reference 6 may be installed at a location other than the attachment T. In another example, the brightness reference 5 and the color reference 6 may be installed on the body of the vehicle V.

[0107] In addition, in the example of FIG. 3, the images of the luminance sample 51 and each color sample (61, 62, 63) are formed in an area other than the output area 200. However, the arrangement of the luminance sample 51 and each color sample (61, 62, 63) does not have to be limited to such an example. In another example, at least one of the images of the luminance sample 51 and each color sample (61, 62, 63) may be formed within the output area 200. In the example of FIG. 2A, FIG. 2B, and FIG. 3, the number of color samples constituting the color reference 6 may be one, two, or four or more.

[0108] Also, in the above embodiment, the present disclosure is applied to the in-vehicle device 1. However, the application scene of the present disclosure may not be limited to the example of the in-vehicle device 1. In another example, the present disclosure may be applied to a server device. In this case, the server device may execute the above information processing on the image received from the client device. The use form of the imaging device S may not be limited to the example of being mounted on the vehicle V, and may be appropriately determined according to the embodiment. The type of the imaging device S may not be particularly limited, and may be appropriately selected according to the embodiment. In one example, the imaging device S may be an RGB camera. Note that the effect of adjusting the brightness can be more effectively exerted in a form in which the imaging device S is used in a scene where strong light may be captured. Therefore, it is preferable that the present disclosure is applied to a form in which the imaging device S is used in a place where strong light may be captured. The form of the in-vehicle device 1 according to the above embodiment is an example of a form in which the imaging device S is used in a place where strong light may be captured.

[0109] In addition, in the above embodiment, either the luminance adjustment process according to the luminance standard 5 or the color adjustment process according to the color standard 6 may be omitted. When the luminance adjustment process according to the luminance standard 5 is omitted, each component and each step related to the luminance standard 5 and the luminance adjustment process may be omitted. When the color adjustment process according to the color standard 6 is omitted, each component and each step related to the color standard 6 and the color adjustment process may be omitted.

[0110] [5 Supplementary Note] The processes and means described in the present disclosure can be freely combined and implemented as long as no technical contradiction occurs.

[0111] Furthermore, a process described as being performed by one device may be shared and executed by multiple devices. Or, a process described as being performed by different devices may be executed by one device. In a computer system, the hardware configuration for implementing each function can be flexibly changed.

[0112] The present disclosure can also be realized by supplying a computer program implementing the functions described in the above embodiments to a computer, and having one or more processors of the computer read and execute the program. Such a computer program may be provided to the computer by a non-transitory computer-readable storage medium connectable to the system bus of the computer, or may be provided to the computer via a network. Non-transitory computer-readable storage media include, for example, any type of disk, such as a magnetic disk (floppy disk, hard disk drive (HDD), etc.), an optical disk (CD-ROM, DVD disk, Blu-ray disk, etc.), a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic card, a flash memory, an optical card, and any type of medium suitable for storing electronic instructions. [Explanation of symbols]

[0113] 1...In-vehicle device (information processing device), V...Vehicle 11: control unit, 12: storage unit, 13: external interface, 14...input device, 15...output device, 16...drive, 81... program, 91... storage medium, 111: image acquisition unit; 112: value acquisition unit; 113: brightness adjustment unit; 114: color adjustment unit; 115...output section, 116: line acquisition unit; 117: distortion correction unit; 20…Images, 25...pixel value (of the brightness reference (brightness sample) image), 26...pixel value (of the color reference (color sample) image), 5...Brightness standard, 51...Brightness sample, 5B…Brightness standard, 53-55…Brightness sample, 6...Color standard, 61~63...Color sample, 6A...Color standard, 65...Color sample, 69...Frame line, S: Imaging device, T: Attachment, M…Luminance meter

Claims

1. Obtaining an image; Obtaining pixel values ​​of a luminance-based image and pixel values ​​of a color-based image captured in the acquired image; applying a brightness adjustment process to the acquired image in response to pixel values ​​of the acquired brightness reference image; and applying a color adjustment process to the captured image in response to pixel values ​​of the captured color reference image; a control unit configured to execute the luminance standard is composed of a plurality of luminance samples; Each of the plurality of luminance samples is configured to emit light at a different brightness; Applying the brightness adjustment process includes: comparing pixel values ​​of the image of the luminance swatch with a reference value corresponding to the luminance swatch; and reducing the brightness of the image if the pixel value of the brightness sample image exceeds the reference value as a result of the comparison; Including, the amount of reduction in the luminance of the image is determined according to the brightness of the luminance sample among the plurality of luminance samples, the pixel value of the image exceeding the reference value; Information processing device.

2. the luminance reference and the color reference are arranged such that an image of the luminance reference and an image of the color reference are formed in an area other than an output area of ​​the image; The information processing device according to claim 1 .

3. The image is acquired from an imaging device mounted on a vehicle.

3. The information processing device according to claim 1 or 2.

4. 1. A computer-implemented information processing method, comprising: Obtaining an image; Obtaining pixel values ​​of a luminance-based image and pixel values ​​of a color-based image captured in the acquired image; applying a brightness adjustment process to the acquired image in response to pixel values ​​of the acquired brightness reference image; and applying a color adjustment process to the captured image in response to pixel values ​​of the captured color reference image; Including, the luminance standard is composed of a plurality of luminance samples; Each of the plurality of luminance samples is configured to emit light at a different brightness; Applying the brightness adjustment process includes: comparing pixel values ​​of the image of the luminance swatch with reference values ​​corresponding to the luminance swatch; and reducing the brightness of the image when the pixel value of the brightness sample image exceeds the reference value as a result of the comparison; Including, the amount of reduction in the luminance of the image is determined according to the brightness of the luminance sample among the plurality of luminance samples, the pixel value of the image exceeding the reference value; Information processing methods.

5. A program for causing a computer to execute an information processing method, The information processing method includes: Obtaining an image; Obtaining pixel values ​​of a luminance-based image and pixel values ​​of a color-based image captured in the acquired image; applying a brightness adjustment process to the acquired image in response to pixel values ​​of the acquired brightness reference image; and applying a color adjustment process to the captured image in response to pixel values ​​of the captured color reference image; Including, the luminance standard is composed of a plurality of luminance samples; Each of the plurality of luminance samples is configured to emit light at a different brightness; Applying the brightness adjustment process includes: comparing pixel values ​​of the image of the luminance swatch with reference values ​​corresponding to the luminance swatch; and reducing the brightness of the image when the pixel value of the brightness sample image exceeds the reference value as a result of the comparison; Including, the amount of reduction in the luminance of the image is determined according to the brightness of the luminance sample among the plurality of luminance samples, the pixel value of the image exceeding the reference value; program.

Images