Image processing algorithm evaluation device

Abstract

This invention comprises: an image storage unit that stores a plurality of real images captured from a vehicle; an image generation unit that, upon input of disturbance information indicating a disturbance to a target image among the plurality of real images stored in the image storage unit, acquires and comprehends the target image, and generates a synthesis image by processing the target image on the basis of the comprehension such that the disturbance is reflected in the target image; and an image processing unit that evaluates, on the basis of the generated synthesis image, the performance of an image processing algorithm for determining the status of the vehicle.

Description

[Technical field]

[0001] The present disclosure relates to an image processing algorithm evaluation device. [Background technology]

[0002] Vehicles equipped with driving support functions such as autonomous driving are being developed. Vehicles equipped with such driving support functions are equipped with an image processing algorithm that captures the surrounding environment of the vehicle using an on-board camera or the like and judges the vehicle's situation based on the captured images.

[0003] Various performance evaluations are being carried out on image processing algorithms to realize appropriate driving support functions. For example, a technique is known in which the performance of an image processing algorithm is evaluated by performing image processing using a composite image in which an image of weather disturbance created by computer graphics is superimposed on an actual image taken from a vehicle (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-194770 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology described in Patent Document 1, the images of weather disturbances, etc. that are generated are not generated to reflect the conditions of objects, etc., captured in real-life images. For this reason, if the image processing algorithm is evaluated using the above-mentioned synthetic images, it may not be possible to obtain an appropriate evaluation.

[0006] The present disclosure has been made in consideration of the above, and aims to provide an image processing algorithm evaluation device capable of appropriately evaluating the performance of an image processing algorithm that determines the situation of a vehicle. [Means for solving the problem]

[0007] The image processing algorithm evaluation device according to the present disclosure includes an image memory unit that stores a plurality of real-life images taken from a vehicle, an image generation unit that, when disturbance information indicating a disturbance to a target image among the plurality of real-life images stored in the image memory unit is input, acquires and understands the target image and processes the target image based on the understanding so that the disturbance is reflected in the target image to generate a composite image, and an image processing unit that evaluates the performance of an image processing algorithm that determines the situation of the vehicle based on the generated composite image. Effect of the Invention

[0008] According to the present disclosure, it becomes possible to appropriately evaluate the performance of an image processing algorithm that determines the situation of a vehicle. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a functional block diagram showing an example of an image processing algorithm evaluation device according to the first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a process in the image generating unit. [Diagram 3] FIG. 3 is a flowchart showing the flow of processing for generating an image. [Figure 4] FIG. 4 is a functional block diagram illustrating an example of an image processing algorithm evaluation device according to the second embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of a reference image and a disturbance image stored in an image storage unit according to the second embodiment. [Figure 6] FIG. 6 is a conceptual diagram showing an example of the image learning unit. [Figure 7] FIG. 7 is a conceptual diagram showing another example of the image learning unit. [Figure 8] FIG. 8 is a diagram showing another example of a disturbance image stored in the image storage unit. [Figure 9] FIG. 9 is a flowchart showing the flow of processing for generating an image. [Figure 10] FIG. 10 is a conceptual diagram illustrating an example of an image learning unit according to the third embodiment. [Figure 11] FIG. 11 is a flowchart showing the flow of processing for generating an image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of an image processing algorithm evaluation device according to the present disclosure will be described with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, the components in the following embodiments include those that are replaceable and easy for a person skilled in the art, or those that are substantially the same.

[0011] [First embodiment] Fig. 1 is a functional block diagram showing an example of an image processing algorithm evaluation device according to a first embodiment. The image processing algorithm evaluation device 100 shown in Fig. 1 includes a calculation device, i.e., a CPU (Central Processing Unit), and a storage device, i.e., a memory for storing calculation contents, program information, and the like. The memory includes at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), and an external storage device such as an HDD (Hard Disk Drive). As shown in Fig. 1, the image processing algorithm evaluation device 100 includes an image storage unit 10, an image generation unit 20, and an image processing unit 30.

[0012] The image storage unit 10 stores actual images captured from a vehicle. The actual images include, for example, a front image captured in front of the vehicle, a side image captured in the side of the vehicle, a rear image captured in the rear of the vehicle, etc. The actual images also include, for example, an actual image captured on a sunny day, an actual image captured in rainy weather, an actual image captured during the day, an actual image captured at night, etc.

[0013] The image generation unit 20 generates a composite image by combining a target image input from the image storage unit 10 with a disturbance based on disturbance information input from an input unit (not shown) or the like. The image generation unit 20 has an image understanding unit 21 and an image processing unit 22. The image understanding unit 21 understands the image stored in the image storage unit 10. The image processing unit 22 processes the target image input to the image generation unit 20.

[0014] The image processing unit 30 performs image processing based on the generated composite image, and evaluates the performance of the image processing algorithm that judges the vehicle's situation. The image processing unit 30 performs image processing on the composite image using the image processing algorithm, and calculates judgment information for judging the vehicle's situation. The image processing unit 30 stores the calculated judgment information in a storage unit (not shown). The judgment information may be, for example, an approach intersection time, which is the time it takes for the vehicle to reach the position of an object in front of the vehicle from the shooting position while the vehicle is traveling. The image processing unit 30 can evaluate the performance of the image processing algorithm based on whether or not the judgment information is significantly different between when an image that does not include a disturbance is processed and when an image that includes a disturbance is processed.

[0015] Fig. 2 is a diagram showing an example of processing in the image generating unit 20. As shown in Fig. 2, a target image and disturbance information are input to the image generating unit 20. The target image may be, for example, a real image captured from the vehicle ahead. The disturbance information may be, for example, weather conditions such as fog. In this embodiment, a case where the disturbance is fog will be described as an example.

[0016] The image understanding unit 21 has calculation units 21a and 21b. The calculation unit 21a acquires an input target image. The calculation unit 21a estimates the distance from the shooting position of the target image to the object constituting the target image for each pixel through image processing. When the object shown in the target image is the sky, the distance can be estimated as infinity (indicating the sky). The calculation unit 21a generates distance information that associates the estimated distance with the pixel, and outputs the distance information.

[0017] The calculation unit 21b acquires distance information output from the calculation unit 21a. The calculation unit 21b also acquires disturbance information input via an input unit (not shown). An example of disturbance information is a foggy image generated by computer graphics. This image may have the same resolution as the target image (real-life image), for example.

[0018] The calculation unit 21b calculates the strength of disturbance for each pixel based on the distance information. For example, in a foggy environment, the influence of the fog increases as the distance from the shooting position increases. Therefore, the disturbance can be adjusted to a state close to the actual environment by adjusting the strength of disturbance for each pixel according to the distance included in the distance information. After adjusting the disturbance for each pixel, the calculation unit 21b outputs disturbance information including the adjusted disturbance.

[0019] The image processing unit 22 acquires the input target image. The image processing unit 22 acquires the disturbance information output from the calculation unit 21b. The image processing unit 22 generates a composite image in which the disturbance included in the acquired disturbance information is superimposed on the target image for each pixel. In other words, the disturbance information input to the image processing unit 22 is information in which the strength of the disturbance is adjusted for each pixel according to the distance, in response to the disturbance information input from an input unit (not shown). By superimposing such adjusted disturbance on the target image, it is possible to generate an appropriate composite image that is closer to the actual disturbance environment.

[0020] The image processing unit 30 evaluates the performance of an image processing algorithm that judges the vehicle's situation based on the composite image. Since the performance of the image processing algorithm is evaluated based on an appropriate composite image that is closer to the actual surrounding environment, a more appropriate evaluation result can be obtained compared to a case where a disturbance generated by computer graphics is simply superimposed. Also, compared to a case where the vehicle's situation is judged using a simulator that reproduces the actual surrounding environment in detail in three dimensions, an appropriate evaluation result can be obtained without using a large-scale system.

[0021] Fig. 3 is a flowchart showing the flow of processing for generating an image. As shown in Fig. 3, the image generating unit 20 acquires an input target image in the image understanding unit 21 (step S101). The image understanding unit 21 estimates the distance from the shooting position of the target image to the target object corresponding to the pixels constituting the target image (step S102), and adjusts the disturbance by calculating the strength of the disturbance according to the estimated distance (step S103). The image processing unit 22 generates a composite image by combining the attenuated disturbance with the target image based on the input target image and the strength of the disturbance calculated by the calculation unit 21b (step S104).

[0022] As described above, the image processing algorithm evaluation device 100 of this embodiment comprises an image storage unit 10 that stores a plurality of real-life images taken from a vehicle, an image generation unit 20 that, when disturbance information indicating a disturbance to a target image among the plurality of real-life images stored in the image storage unit 10 is input, acquires and understands the target image and processes the target image based on the understanding so that the disturbance is reflected in the target image to generate a composite image, and an image processing unit 30 that evaluates the performance of an image processing algorithm that determines the vehicle situation based on the generated composite image.

[0023] According to this configuration, the image generator 20 processes the target image based on the result of understanding the target image so that the disturbance is reflected in the target image to generate a composite image, so that a composite image closer to the actual surrounding environment can be generated compared to a case where a disturbance generated by computer graphics is simply superimposed. This allows the performance of the image processing algorithm for judging the vehicle situation to be appropriately evaluated.

[0024] In the image processing algorithm evaluation device 100 according to this embodiment, the image generator 20 estimates the distance from the photographing position of the target image to the target object constituting the target image, calculates the strength of disturbance due to the distance based on the estimated distance, and generates a composite image based on the calculation result. Therefore, it is possible to generate a composite image closer to the actual surrounding environment.

[0025] [Second embodiment] Next, a second embodiment will be described. Fig. 4 is a functional block diagram showing an example of an image processing algorithm evaluation device according to the second embodiment. The configuration of the image processing algorithm evaluation device 100 according to the second embodiment is the same as that of the first embodiment, and includes an image storage unit 10, an image generation unit 20, and an image processing unit 30. In this embodiment, the processing content in the image generation unit 20 is different from that of the first embodiment. In addition, a learning image storage unit 40 and an image learning unit 50 are added.

[0026] In this embodiment, the learning image storage unit 40 stores a reference image, which is a real-life image that does not include disturbance, and a disturbance image, which is a real-life image that includes disturbance, as learning images. An example of the reference image is a real-life image taken under conditions where disturbance is the least, such as during the daytime on a clear day. An example of the disturbance image is a real-life image taken under conditions where disturbance is greater than that of the reference image, such as during rainy weather, snowfall, fog, dawn, evening, or night. The learning image storage unit 40 may store a reference image and a disturbance image for the same or corresponding shooting object in association with each other. For example, the learning image storage unit 40 can store a reference image taken at a specific location during the daytime on a clear day, and disturbance images taken at the specific location during rainy weather, snowfall, fog, dawn, evening, or night, in association with each other.

[0027] Fig. 5 is a diagram showing an example of a reference image and a disturbance image stored in the learning image storage unit 40 according to the second embodiment. The reference image IM1 shown in Fig. 5 is an actual image captured of the front from a moving vehicle during the daytime on a clear day. The disturbance image IM2 shown in Fig. 5 is an actual image captured of the front from a moving vehicle during the nighttime on a clear day in the same location as the reference image IM1. The reference image IM1 and the disturbance image IM2 are stored in the learning image storage unit 40 in a corresponding state.

[0028] In this embodiment, when a target image and disturbance information indicating disturbance to the target image are input, the image learning unit 50 can generate a synthetic image for learning by learning using a neural network, for example. For example, as a method for realizing a predetermined image processing using a neural network in the image learning unit 50, a technology called Generative Adversarial Networks (GAN) or a technology called Cycle Generative Adversarial Networks (Cycle GAN) can be used.

[0029] FIG. 6 is a conceptual diagram showing an example of the image learning unit 50 (image learning unit 50A). As shown in FIG. 6, the generative adversarial network used in the image learning unit 50A is composed of two neural networks, an image generation unit 51 and an authenticity determination unit 52. The image generation unit 51 has the same configuration as the image generation unit 20 described above, and has an image understanding unit 51a and an image processing unit 51b. The image generation unit 51 generates a learning synthetic image by synthesizing a disturbance with a reference image by the same process as the process in which the image generation unit 20 generates a synthetic image based on a target image and disturbance information. The authenticity determination unit 52 determines the authenticity of the learning synthetic image generated by the image generation unit 51 based on the learning synthetic image generated by the image generation unit 51 and the disturbance image associated with the reference image. The image generation unit 51 generates a learning synthetic image so that the authenticity determination unit 52 determines that the learning synthetic image is closer to the true image. Moreover, the authenticity determination unit 52 tries to detect differences between the generated learning synthetic image and the true image. By alternately having these two networks compete with each other and progressing with learning, the image generation unit 51 can generate a learning synthetic image that is close to the true disturbance image.

[0030] FIG. 7 is a conceptual diagram showing another example (image learning unit 50B) of the image learning unit 50. As shown in FIG. 7, the cycle generative adversarial network used in the image learning unit 50B has image generation units 51 and 53 and authenticity determination units 52 and 54. The image generation units 51 and 53 have the same configuration as the image generation unit 20 described above. The image generation unit 51 has an image understanding unit 51a and an image processing unit 51b. The image generation unit 53 has image understanding units 53a and 53b. The image generation unit 51 generates a first learning synthetic image by synthesizing a disturbance with a reference image. In addition, the image generation unit 53 generates a second learning synthetic image by removing the disturbance from the disturbance image. The authenticity determination unit 52 determines the authenticity of the first learning synthetic image based on the true disturbance image. In addition, the authenticity determination unit 54 determines the authenticity of the second learning synthetic image based on the true reference image. Image generation units 51, 53 generate the first learning synthetic image and the second learning synthetic image so that the authenticity determination units 52, 54 determine that the images are closer to the true image, that is, so that the accuracy rate is higher. Furthermore, the authenticity determination units 52, 54 attempt to detect differences between the generated first learning synthetic image and the true image and the second learning synthetic image more effectively. By alternately having these two networks compete with each other and progressing with learning, the image generation unit 51 is able to generate a learning synthetic image that is closer to the true disturbance image, that is, so that the accuracy rate of authenticity is high.

[0031] In the cycle generative adversarial network, it is sufficient to have a reference image that does not include a disturbance and a disturbance image that includes a disturbance, and it is not necessary for the reference image and the disturbance image to be associated with each other. Therefore, compared to the generative adversarial network, it is easier to collect training data. In addition, when using the cycle generative adversarial network, multiple disturbance images with different levels of disturbance can be stored in the training image storage unit 40.

[0032] Fig. 8 is a diagram showing another example of disturbance images stored in the learning image storage unit 40. As shown in Fig. 8, in addition to the above-mentioned reference image IM1 and disturbance image IM2 (see Fig. 5), nighttime disturbance images IM3 and IM4 having a different degree of disturbance, i.e., a different brightness, from the disturbance image IM2 are stored in the learning image storage unit 40. The brightness (degree of disturbance) of the disturbance image IM2 shown in Fig. 5 is, for example, 100 lux, the brightness of the disturbance image IM3 shown in Fig. 8 is, for example, 10 lux, and the brightness of the disturbance image IM4 is, for example, 1000 lux.

[0033] In this manner, in a configuration in which a plurality of disturbance images with different degrees of disturbance are stored in the learning image storage unit 40, the image generating unit 20 can generate a composite image having a brightness corresponding to the specified brightness when, for example, information specifying the type of disturbance (nighttime) as well as information specifying the degree of disturbance (brightness) is input as disturbance information by using a cycle generative adversarial network. In the example shown in Fig. 8, the image generating unit 20 can generate composite images having brightnesses of, for example, 1000 lux, 100 lux, and 10 lux.

[0034] FIG. 9 is a flowchart showing the flow of the process of generating an image. As shown in FIG. 9, the image learning unit 50 acquires a learning image stored in the learning image storage unit 40 (step S201). The image generating unit 51 understands the acquired learning image in the image understanding unit 51a (step S202). The image processing unit 51b generates a learning composite image in which a disturbance is added to the learning image based on the understanding result (step S203). The authenticity determination unit 52 determines the authenticity of the learning composite image based on the true disturbance image (step S204). The image learning unit 50 makes the image generating unit 51 and the authenticity determination unit 52 compete with each other alternately to proceed with learning (step S205). The image learning unit 50 determines whether learning is completed (step S206), and if it is determined that learning is completed (Yes in step S206), proceeds to step S207. Moreover, if it is determined that the learning is not complete (No in step S206), the process from step S202 onward is repeated. In step S206, the image learning unit 50 can determine that the learning is complete, for example, when the difference between the learning synthetic image and the true disturbance image is equal to or smaller than a predetermined value, that is, when the accuracy rate is equal to or greater than a predetermined value.

[0035] After the learning is completed, when the target image and disturbance information are input, the image generating unit 20 acquires the target image from the image storage unit 10 (step S207). The image understanding unit 21 understands the image based on the learning result (step S208). The image processing unit 22 generates a composite image in which a disturbance is added to the target image based on the understanding result (step S209).

[0036] As described above, in the image processing algorithm evaluation device 100 according to this embodiment, the learning image storage unit 40 stores a reference image, which is a real-life image that does not include a disturbance, and a disturbance image, which is a real-life image that includes a disturbance, the image learning unit 50 learns using a generative adversarial network or a cycle generative adversarial network based on the reference image and the disturbance image, and the image generation unit 20 understands the target image based on the learning result of the image learning unit 50 and generates a synthetic image.

[0037] In this configuration, the image learning unit 50 learns real-life images using a generative adversarial network or a cycle generative adversarial network, and the image generating unit 20 understands the target image and generates a synthetic image based on the learning results, so that a synthetic image close to the actual surrounding environment can be generated. This makes it possible to appropriately evaluate the performance of the image processing algorithm that judges the vehicle situation.

[0038] Furthermore, the learning image storage unit 40 stores a plurality of disturbance images including the same type of disturbance but different degrees, the image learning unit 50 performs learning based on the plurality of disturbance images including disturbances of different degrees, and when disturbance information including the degree of disturbance is input, the image generating unit 20 understands the target image based on the learning result of the image learning unit 50 and generates a synthetic image such that the target image reflects the disturbance of the degree corresponding to the disturbance information. This makes it possible to appropriately generate synthetic images with different degrees of disturbance, thereby making it possible to widely evaluate the performance of image processing algorithms.

[0039] [Third embodiment] Next, a third embodiment will be described. The image processing algorithm evaluation device 100 according to the third embodiment is configured to include an image storage unit 10, an image generation unit 20, an image processing unit 30, a learning image storage unit 40, and an image learning unit 50, similar to the second embodiment. In this embodiment, the type of real-life image stored in the learning image storage unit 40 and the processing contents in the image generation unit 20 and the image learning unit 50 are different from those in the second embodiment.

[0040] The learning image storage unit 40 stores real-life images including a reference image and a disturbance image as learning images, as in the second embodiment. In this embodiment, the learning image storage unit 40 stores real-life images including different attributes. Examples of real-life images including different attributes include a plurality of real-life images with different attributes such as a real-life image taken in a downtown area, a real-life image taken in a residential area, and a real-life image taken on a mountain path. The learning image storage unit 40 can store such real-life images in association with label information indicating attributes.

[0041] In this embodiment, when a target image and disturbance information for the target image are input, the image generating unit 20 extracts label information indicating the above-mentioned attributes from the target image. The image generating unit 20 can understand the target image based on the learning result of the image learning unit 50 described later and generate a composite image reflecting the attributes. For example, when the target image is a daytime reference image of a "downtown", if "night" is input as disturbance information, it is possible to convert the image to not only darken the sky but also turn on the lights of the surrounding buildings. Also, for example, when the target image is a daytime reference image of a "mountain path", if "night" is input as disturbance information, it is possible to convert the image to not only darken the sky but also darken the surroundings as a whole.

[0042] 10 is a conceptual diagram showing an example of an image learning unit 50 (image learning unit 50C) according to the third embodiment. As shown in FIG. 10, the cycle generative adversarial network used in the image learning unit 50C has image generation units 51 and 53 and authenticity determination units 52 and 54, similar to the second embodiment. In the cycle generative adversarial network according to this embodiment, label information is input to the image generation units 51 and 53.

[0043] In this case, the image generating unit 51 generates a first composite image by combining a disturbance with a reference image. For example, if the extracted attribute is "downtown" and the input disturbance is "night", the composite image is generated so that the night disturbance is reflected in the reference image of the downtown. Also, the image generating unit 53 generates a second composite image by removing the night disturbance from the night disturbance image of the downtown. The authenticity determining unit 52 determines the authenticity of the first composite image based on the true disturbance image, and determines whether the attribute of "downtown" is appropriately reflected. Also, the authenticity determining unit 54 determines the authenticity of the second composite image based on the true reference image, and determines whether the night disturbance of the downtown is appropriately removed. Thus, in this embodiment, the authenticity determining units 52 and 54 determine the appropriateness of reflecting the attribute in addition to determining the authenticity of the composite image.

[0044] In this embodiment, the image generating units 51 and 53 generate the first and second composite images so that the authenticity determining units 52 and 54 determine that the images are closer to the true image and that the attributes are appropriately reflected. The authenticity determining units 52 and 54 also attempt to detect differences between the generated first and second composite images and the true image, and points where the attributes are not appropriately reflected. By alternately having these two networks compete with each other and progressing with learning, the image generating unit 51 becomes able to generate composite images with a high rate of correct answers for authenticity and a high rate of correct answers for attribute reflection.

[0045] FIG. 11 is a flowchart showing the flow of the process of generating an image. As shown in FIG. 11, the image learning unit 50 acquires a learning image stored in the learning image storage unit 40 (step S301). The image learning unit 50 extracts label information indicating attributes included in the acquired learning image (step S302). The image generating unit 51 understands the acquired learning image in the image understanding unit 51a (step S303). The image processing unit 51b generates a learning composite image in which a disturbance is added to the learning image based on the understanding result (step S304). The authenticity determination unit 52 determines the authenticity of the learning composite image based on the true disturbance image (step S305). The image learning unit 50 makes the image generating unit 51 and the authenticity determination unit 52 compete with each other alternately to proceed with learning (step S306). The image learning unit 50 determines whether learning is complete (step S307), and if it is determined that learning is complete (Yes in step S307), the process proceeds to step S308. If it is determined that learning is not complete (No in step S307), the process repeats the processes from step S303 onward. In step S307, the image learning unit 50 can determine that learning is complete, for example, when the difference between the learning synthetic image and the true disturbance image is equal to or less than a predetermined value, and the difference in the part related to the attributes between the learning synthetic image and the disturbance image is equal to or less than a predetermined value, that is, when the accuracy rate of the authenticity of the learning synthetic image and the accuracy rate of the part related to the attributes are equal to or greater than a predetermined value.

[0046] When the target image and disturbance information are input, the image generating unit 20 acquires the input target image (step S308). The image understanding unit 21 extracts label information indicating attributes contained in the target image (step S309) and understands the target image (step S310). The image processing unit 22 generates a composite image in which a disturbance is added to the target image based on the understanding result (step S311).

[0047] As described above, in the image processing algorithm evaluation device 100 according to this embodiment, the training image storage unit 40 stores a plurality of real-life images including different attributes in association with label information indicating the attributes, the image learning unit 50 learns using a cycle generative adversarial network so as to increase the accuracy rate regarding attributes between the generated training synthetic image and the disturbance image, and when a target image and disturbance information are input, the image generation unit 20 extracts label information indicating the attributes of the target image, understands the target image based on the learning result of the image learning unit 50, and generates a synthetic image.

[0048] With this configuration, the cycle generative adversarial network can learn to increase the accuracy rate of identifying the authenticity of the training synthetic image and to increase the accuracy rate of reflecting attributes, thereby making it possible to generate synthetic images in a manner that corresponds to the attributes of the real-life image.

[0049] The technical scope of the present invention is not limited to the above-described embodiment, and appropriate modifications can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0050] IM1 Reference Image IM2, IM3, IM4 disturbance images 10 Image storage section 20 Image Generation Unit 21 Image Understanding Department 21a,21b Arithmetic unit 22 Image Processing Department 30 Image processing section 40 Learning image memory unit 50 Image Learning Section 51,53 Image generation unit 51a, 53a Image Understanding Section 51b,53b Image processing department 52,54 Authenticity Determination Department 100 Image processing algorithm evaluation device

Claims

1. an image storage unit that stores an actual image captured from a vehicle; an image generating unit that, when disturbance information indicating a disturbance to a target image among the real-life images stored in the image storage unit is input, acquires and understands the target image, and processes the target image based on the understanding so that the disturbance is reflected in the target image to generate a composite image; an image processing unit that evaluates the performance of an image processing algorithm that determines the situation of the vehicle based on the generated synthetic image; Equipped with The image generating unit estimates a distance from a shooting position of the target image to an object constituting the target image, calculates the strength of the disturbance due to the distance based on the estimated distance, and generates the composite image based on the calculation result. Image processing algorithm evaluation device.

2. a learning image storage unit that stores, as learning images, a reference image that is the real-life image that does not include the disturbance and a disturbance image that is the real-life image that includes the disturbance; an image learning unit that generates a synthetic image for training based on the reference image and the disturbance information by the same process as the process in which the image generation unit generates the synthetic image based on the target image and the disturbance information, and learns using a generative adversarial network or a cycle generative adversarial network so as to increase a rate of accuracy between the generated synthetic image for training and the disturbance image; Further comprising: The image generating unit understands the target image based on the learning result of the image learning unit and generates the composite image.

2. The image processing algorithm evaluation device according to claim 1.

3. An image storage unit that stores an actual image taken from a vehicle; an image generating unit that, when disturbance information indicating a disturbance to a target image among the real-life images stored in the image storage unit is input, acquires and understands the target image, and processes the target image based on the understanding so that the disturbance is reflected in the target image to generate a composite image; an image processing unit that evaluates the performance of an image processing algorithm that determines the situation of the vehicle based on the generated synthetic image; Equipped with a learning image storage unit that stores, as learning images, a reference image that is the real-life image that does not include the disturbance and a disturbance image that is the real-life image that includes the disturbance; an image learning unit that generates a synthetic image for training based on the reference image and the disturbance information by the same process as the process in which the image generation unit generates the synthetic image based on the target image and the disturbance information, and learns using a generative adversarial network or a cycle generative adversarial network so as to increase a rate of accuracy between the generated synthetic image for training and the disturbance image; Further comprising: The image generation unit understands the target image based on a learning result of the image learning unit to generate the composite image; the learning image storage unit stores a plurality of the real-life images having different attributes in association with label information indicating the attributes; The image learning unit learns using the cycle generative adversarial network so as to increase a correct answer rate for the attribute between the generated learning synthetic image and the disturbance image; When the target image and the disturbance information are input, the image generation unit extracts label information indicating the attributes of the target image, understands the target image based on a learning result of the image learning unit, and generates the composite image reflecting the attributes. Image processing algorithm evaluation device.

4. the learning image storage unit stores a plurality of disturbance images including the disturbances of the same type but different degrees; the image learning unit performs learning based on a plurality of disturbance images including the disturbances of different degrees, When the disturbance information including the degree of the disturbance is input, the image generation unit understands the target image based on a learning result of the image learning unit, and generates the synthetic image such that the disturbance of a degree corresponding to the disturbance information is reflected in the target image.

4. The image processing algorithm evaluation device according to claim 2.

Images