Image evaluation method and image evaluation program

Abstract

To provide an image evaluation method and an image evaluation program capable of appropriately evaluating distortion of an under-vehicle image generated on the basis of a camera image attached to a vehicle.SOLUTION: Generating an integrated overhead image in which an overhead image that is an image obtained by overlooking the vicinity of a vehicle and a shielded image that is an image of an area shielded by the vehicle when the vehicle is overlooked are integrated, and evaluating an image deviation between the overhead image and the shielded image, wherein the overhead image and the shielded image include an image obtained by imaging a same first graphic that extends across the overhead image and the shielded image in the integrated overhead image when there is no image deviation.SELECTED DRAWING: Figure 17

Description

[Technical Field]

[0001] The present disclosure relates to an image evaluation method and an image evaluation program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a technology has been put into practical use in which images captured by multiple cameras on a vehicle are combined to generate an overhead image of the vehicle's surroundings.

[0003] For example, Patent Document 1 discloses a device that estimates parameters representing the vehicle's posture from images captured by each camera based on linear features in the longitudinal direction of the vehicle, such as white lines painted on the road surface, and evaluates image deviations at the image boundaries to correct an overhead image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6371185 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the above-described conventional technology, it is difficult to evaluate distortion of the under-vehicle image occluded by the vehicle with sufficient accuracy.

[0006] Non-limiting examples of the present disclosure contribute to providing an image evaluation method and an image evaluation program that can appropriately evaluate distortion in an under-vehicle image generated based on an image from a camera attached to the vehicle. [Means for solving the problem]

[0007] An image evaluation method according to one embodiment of the present disclosure includes the steps of generating an integrated overhead image by integrating an overhead image, which is an image of an area near a vehicle, and an occlusion image, which is an image of an area that is blocked by the vehicle when viewed from above, and evaluating the image misalignment between the overhead image and the occlusion image, wherein the overhead image and the occlusion image include images of the same first figure that straddles the overhead image and the occlusion image in the integrated overhead image when there is no image misalignment.

[0008] An image evaluation program according to one embodiment of the present disclosure causes a computer to perform the following steps: generating an integrated overhead image by integrating an overhead image, which is an image of the vicinity of a vehicle, with an occlusion image, which is an image of the area blocked by the vehicle when viewed from above; and evaluating the image misalignment between the overhead image and the occlusion image; wherein the overhead image and the occlusion image include images of the same first figure that is configured to straddle the overhead image and the occlusion image in the integrated overhead image when there is no image misalignment.

[0009] An image evaluation method according to one embodiment of the present disclosure includes the steps of generating an integrated overhead image by integrating an overhead image, which is an image of an area near a vehicle, and an occlusion image, which is an image of an area that is blocked by the vehicle when viewed from above, and evaluating the image deviation between the overhead image and the occlusion image using a second image, wherein the second image includes a first region outside the boundary of the occlusion image, a second region outside the first region, and a third region that is smaller than the occlusion image, and in the step of evaluating the image deviation, the image deviation is evaluated based on whether the third region borders the first region, and the first region, the second region, and the third region are colored in different colors.

[0010] An image evaluation program according to one embodiment of the present disclosure causes a computer to perform the following steps: generating an integrated overhead image by integrating an overhead image, which is an image of the area near a vehicle, with an occlusion image, which is an image of the area that is blocked by the vehicle when viewed from above; and evaluating the image deviation between the integrated overhead image and the occlusion image using a second image, wherein the second image includes a first region outside the boundary of the occlusion image, a second region outside the first region, and a third region that is smaller than the occlusion image; and in the step of evaluating the image deviation, the image deviation is evaluated based on whether the third region borders the first region, and the first region, the second region, and the third region are colored in different colors.

[0011] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0012] According to an embodiment of the present disclosure, it is possible to provide an image evaluation method and an image evaluation program that can appropriately evaluate distortion in an under-vehicle image generated based on an image from a camera attached to the vehicle.

[0013] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram illustrating an example of an evaluation device according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of a pattern used in a first evaluation method of an overhead image according to the first embodiment; [Figure 3]FIG. 10 is a diagram showing an example of a pattern used in a first evaluation method of an overhead image according to the first embodiment; [Figure 4] FIG. 10 is a diagram showing an example of a pattern used in a first evaluation method of an overhead image according to the first embodiment; [Figure 5] FIG. 10 is a diagram showing an example of a pattern used in a first evaluation method of an overhead image according to the first embodiment; [Figure 6] FIG. 10 is a diagram showing an example of a pattern used in a second evaluation method of an overhead image according to the first embodiment; [Figure 7] FIG. 10 is a diagram showing an example of a pattern used in a second evaluation method of an overhead image according to the first embodiment; [Figure 8] FIG. 10 is a diagram showing an example of a pattern used in a second evaluation method of an overhead image according to the first embodiment; [Figure 9] FIG. 10 is a diagram showing an example of a pattern used in a third evaluation method for an overhead image according to the first embodiment; [Figure 10] FIG. 10 is a diagram showing an example of a pattern used in a third evaluation method for an overhead image according to the first embodiment; [Figure 11] FIG. 10 is a diagram showing an example of a pattern used in a fourth evaluation method of an overhead image according to the first embodiment; [Figure 12] FIG. 10 is a diagram showing an example of a pattern used in a fourth evaluation method of an overhead image according to the first embodiment; [Figure 13] FIG. 10 is a diagram showing an example of a pattern used in a fourth evaluation method of an overhead image according to the first embodiment; [Figure 14] FIG. 10 is a diagram showing an example of a pattern used in a fifth evaluation method of an overhead image according to the first embodiment; [Figure 15] FIG. 10 is a diagram showing an example of a pattern used in a fifth evaluation method of an overhead image according to the first embodiment; [Figure 16] FIG. 10 is a diagram showing an example of a pattern used in a sixth evaluation method of an overhead image according to the first embodiment; [Figure 17] FIG. 10 is a block diagram illustrating an example of an evaluation device according to a second embodiment. [Figure 18] FIG. 10 is a diagram showing an example of a pattern used in a first evaluation method of a masked image according to a second embodiment; [Figure 19] FIG. 10 is a diagram showing an example of a pattern used in a second evaluation method of a masked image according to a second embodiment; [Figure 20] FIG. 10 is a diagram showing an example of a pattern used in a second evaluation method of a masked image according to a second embodiment; [Figure 21] FIG. 10 is a diagram showing an example of a pattern used in a third evaluation method of a masked image according to the second embodiment; [Figure 22] FIG. 10 is a diagram showing an example of a pattern used in a third evaluation method of a masked image according to the second embodiment; [Figure 23] FIG. 10 is a diagram showing an example of a pattern used in a fourth evaluation method of a masked image according to the second embodiment; [Figure 24] FIG. 10 is a diagram showing an example of a pattern used in a fourth evaluation method of a masked image according to the second embodiment; [Figure 25] FIG. 10 is a diagram showing an example of visualization of the evaluation result of the integrated overhead image by the image evaluation unit according to the second embodiment. [Figure 26] FIG. 10 is a diagram showing an example of a pattern used in a fifth evaluation method of a masked image according to the second embodiment; [Figure 27] FIG. 10 is a diagram showing an example of a pattern used in a fifth evaluation method of a masked image according to the second embodiment; [Figure 28] FIG. 10 is a diagram showing an example of a pattern used in a fifth evaluation method of a masked image according to the second embodiment; [Figure 29] FIG. 10 is a diagram showing an example of a pattern used in a sixth evaluation method of a masked image according to the second embodiment; [Figure 30] FIG. 10 is a diagram showing an example of a pattern used in a sixth evaluation method of a masked image according to the second embodiment; [Figure 31] FIG. 10 is a diagram showing an example of a pattern used in a sixth evaluation method of a masked image according to the second embodiment; [Figure 32] FIG. 10 is a diagram showing an example of a pattern used in a seventh evaluation method of a masked image according to the second embodiment; [Figure 33] FIG. 10 is a diagram showing an example of a pattern used in a seventh evaluation method of a masked image according to the second embodiment; [Figure 34]FIG. 10 is a diagram showing an example of a pattern used in an eighth evaluation method of a masked image according to the second embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or redundant explanation of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following explanation and to facilitate understanding by those skilled in the art.

[0016] The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0017] (First embodiment) <Evaluation equipment> The evaluation device 1 will be described with reference to Fig. 1. As shown in Fig. 1, the evaluation device 1 includes an overhead image generation unit 11 and an image evaluation unit 12.

[0018] The overhead image generating unit 11 generates an overhead image P1 including a pattern described below based on the camera image 21. The pattern included in the overhead image P1 is a captured image of a pattern formed on the floor surface. When generating the overhead image P1, for example, the overhead image generating unit 11 refers to camera installation information 22, distortion information 23, vehicle movement position information 24a, and vehicle attitude information 24b. The overhead image generating unit 11 transmits the overhead image P1 to the image evaluating unit 12.

[0019] The image evaluation unit 12 evaluates the overhead-view image P1 received from the overhead-view image generation unit 11 by referring to the floor surface information 31 and the vehicle shape information 32. For example, the image evaluation unit 12 determines whether or not the overhead-view image P1 is distorted based on the degree of distortion of the pattern included in the overhead-view image P1. The image evaluation unit 12 may also calculate parameters for correcting (calibrating) the distortion of the overhead-view image P1 and add them to the camera installation information 22 and the distortion information 23 to generate a corrected image Q1 in which the distortion of the overhead-view image P1 has been corrected.

[0020] Furthermore, the image evaluation unit 12 feeds back the evaluation result to the overhead image generation unit 11. The overhead image generation unit 11 may generate the overhead image P1 by referring to the evaluation result fed back from the image evaluation unit 12 in addition to the camera image 21 and other information.

[0021] For example, the evaluation device 1 may be equipped with a storage device (not shown) such as a memory, and parameters for correcting distortion in the overhead image P1 and a corrected image Q1 obtained by correcting the distortion in the overhead image P1 may be stored in the device.

[0022] <Various information> The camera installation information 22, the distortion information 23, the vehicle movement position information 24a, and the vehicle attitude information 24b are each information acquired by a sensor provided on the vehicle V. The camera installation information 22 and the distortion information 23 may be stored in advance in a storage device.

[0023] The camera image 21 is an image captured by a camera attached to the vehicle V. For example, the vehicle V is equipped with four cameras, which capture images of the entire periphery of the vehicle V, including floor surface information 31. The overhead image P1 generated based on these images includes the floor surface information 31.

[0024] The floor surface information 31 is information about the road surface on which the vehicle V is traveling, and includes the slope and unevenness of the road surface, the road surface material (asphalt, concrete, soil, etc.), the road surface condition (dry, wet, frozen, etc.), and patterns painted on the road surface (white lines on the road, parking and crosswalk markings, etc.).

[0025] The camera mounting information 22 is information about the position and orientation of a camera mounted on the vehicle V, and includes coordinates (x, y, z) that represent the camera position and Euler angles (rx, ry, rz) that represent the camera orientation. The camera mounting information 22 also includes vehicle shape information 32.

[0026] The vehicle shape information 32 includes information on the physical shape and dimensions of the vehicle V, such as the overall length, overall width, overall height, wheelbase (the distance between the front and rear wheels), tread (the distance between the left and right wheels), vehicle height, the shape and curves of the body, and the positions of windows and doors. This information is set in advance and may therefore be stored in a storage device in advance.

[0027] The distortion information 23 includes information relating to distortion of the lens of the camera attached to the vehicle V. The distortion information 23 includes parameters that indicate the characteristics of the camera lens and the degree of distortion.

[0028] The vehicle movement position information 24a and the vehicle attitude information 24b are information based on vehicle steering information 34. The vehicle steering information 34 includes, for example, information such as the steering angle and acceleration of the vehicle, and data from a gyro sensor.

[0029] The vehicle movement position information 24a includes, for example, information regarding the initial position of the vehicle V, the movement distance of the vehicle V from the initial position, and the movement direction of the vehicle V.

[0030] The vehicle attitude information 24b includes, for example, information regarding the direction in which the vehicle V is facing (yaw angle) and the degree to which the vehicle V is tilted relative to the ground (pitch angle and roll angle).

[0031] In the first embodiment, a case will be described in which the vehicle V is not actually driven, but the driving of the vehicle V and the generation of the overhead image P1 are simulated by MILS (Model In the Loop Simulation) to evaluate the overhead image P1. However, the present invention is not limited to this, and the vehicle V may actually be driven to generate the overhead image P1, and the overhead image P1 may be evaluated.

[0032] <Evaluation method for overhead image P1> The following describes a method for evaluating the overhead image P1 by the image evaluation unit 12. The image evaluation unit 12 according to the first embodiment evaluates the overhead image P1 using a plurality of methods. Below, the methods for evaluating the overhead image P1 by the image evaluation unit 12 will be described in order.

[0033] (First evaluation method for overhead image P1) A first evaluation method for the overhead image P1 according to the first embodiment will be described with reference to FIGS.

[0034] The overhead image P1 includes an occluded area 2, a camera boundary line 3, and a pattern A11.

[0035] The shaded area 2 is an area covered by a moving or parked vehicle V. In other words, the shaded area 2 is a space in which the vehicle V is present at the time of image evaluation, and may hereinafter be referred to as a parking space.

[0036] The camera boundary line 3 is a boundary line of the camera image 21 captured by a camera attached to the vehicle V. In this embodiment, four cameras are used. Therefore, the overhead image P1 is composed of four camera images 21a, 21b, 21c, and 21d. Note that the overhead image P1 may be composed of four or more camera images.

[0037] Pattern A11 is a pattern for evaluating distortion between the four camera images 21a to 21d. Pattern A11 defines a quadrangular area in overhead image P1. Pattern A11 is a quadrangle that is larger than occluded area 2 and has sides perpendicular to camera boundary line 3.

[0038] Pattern A11 may be any polygon including a side perpendicular to camera boundary line 3, and overhead image P1 can be evaluated using such a polygon. Pattern A11a may also be a circular or elliptical pattern including a curve, as shown in Fig. 3. Pattern A11 may also be a pattern including a curve and having a side perpendicular to camera boundary line 3.

[0039] The image evaluation unit 12 analyzes the overhead image P1 to detect white. For example, the image evaluation unit 12 detects white lines on the road surface. The image evaluation unit 12 also acquires the positions of the white lines on each of the camera images 21a to 21d (for example, x and y coordinates with the center of the overhead image P1 as the origin).

[0040] Next, the image evaluation unit 12 compares the positions of the white lines on the camera boundary line 3 in each of the camera images 21a to 21d, and calculates the difference in the positions as the amount of distortion.

[0041] The image evaluation unit 12 acquires the position of the white line from the center of the part detected as the white line or the center of the part where each value of RGB (red, green, and blue color components) is maximum.

[0042] For example, the width of the white line is preferably about 8 pixels or 5 cm so that it can be detected by the image evaluation unit 12. The size of the pattern A11 is preferably about twice the overall length of the vehicle V.

[0043] Based on the calculated amount of distortion, the image evaluation unit 12 determines that the overhead image P1 shown in FIGS. 2 and 3 is undistorted, and determines that the overhead image P1 shown in FIGS. 4 and 5 is distorted.

[0044] In the overhead image P1 shown in Fig. 4, camera image 21a is misaligned with respect to camera images 21b and 21d, so image evaluation unit 12 determines that there is distortion. In addition, in the overhead image P1 shown in Fig. 5, pattern A11, which should be rectangular, is distorted and rounded, so it is also determined that there is distortion.

[0045] In the first evaluation method for the overhead image P1, the image evaluation unit 12 compares the positions of the white lines of the rectangular pattern A11 or the curved pattern A11a at the camera boundary line 3 of each of the camera images 21a to 21d, calculates the difference in position as the amount of distortion, and evaluates the distortion of the overhead image P1 generated from each of the camera images 21a to 21d.

[0046] Furthermore, the evaluation device 1 may be provided with a display (not shown) such as a liquid crystal display, and may display parameters for correcting distortion in the overhead-view image P1 and a corrected image Q1 obtained by correcting the distortion in the overhead-view image P1 on the display. In this case, the user can view the image displayed on the display and check and evaluate the distortion of the overhead-view image P1.

[0047] Furthermore, although the pattern A11 is black on a white background, it may be white on a black background, and may be any color as long as the image evaluation unit 12 can evaluate the distortion of the overhead image P1.

[0048] (Second evaluation method for overhead image P1) A second evaluation method for the overhead image P1 according to the first embodiment will be described with reference to FIGS.

[0049] The pattern A12 shown in FIG. 6 is a pattern for evaluating distortion between the four camera images 21a to 21d. The pattern A12 is composed of four diamonds. The longer of the two diagonals of each diamond of the pattern A12 is perpendicular to the camera boundary line 3, as with the pattern A11 of the first embodiment. However, the pattern A12 differs from the pattern A11 of the first embodiment in that it has a non-constant thickness in the direction perpendicular to the diagonal. This allows the image evaluation unit 12 to accurately calculate the amount of distortion in the direction parallel to the camera boundary line 3.

[0050] 7 is also made up of triangles with sides perpendicular to the camera boundary line 3, like the pattern A12. This allows the image evaluation unit 12 to accurately calculate the amount of distortion in the direction parallel to the camera boundary line 3.

[0051] 8 is an example of an overhead image P1 determined to be distorted by the image evaluation unit 12. Two diamond-shaped portions included in the camera image 21a are misaligned in directions perpendicular and parallel to the camera boundary line 3, so the image evaluation unit 12 determines that there is distortion.

[0052] In the second evaluation method for overhead image P1, pattern A12 and pattern A13 have thickness in a direction parallel to camera boundary line 3. Therefore, image evaluation unit 12 can accurately calculate the amount of distortion in directions perpendicular and parallel to camera boundary line 3.

[0053] (Third evaluation method for overhead image P1) A third evaluation method for the overhead image P1 according to the first embodiment will be described with reference to FIGS.

[0054] The overhead image P1 shown in Fig. 9 includes a pattern in which a pattern A11 and a pattern A12 are concentrically combined, and the overhead image P1 shown in Fig. 10 includes a pattern in which two patterns A12 are concentrically combined.

[0055] In the third evaluation method for the overhead image P1, a plurality of patterns are formed on the overhead image P1, which allows the image evaluation unit 12 to accurately calculate the amount of distortion.

[0056] It should be noted that a plurality of the above-described patterns A11, A11a, A12, and A13 may be appropriately combined, and three or more patterns may be combined as long as the space in the overhead image P1 allows.

[0057] (Fourth evaluation method for overhead image P1) A fourth evaluation method for the overhead image P1 according to the first embodiment will be described with reference to FIGS.

[0058] 11 is a rectangle that is concentric with and larger than the shielded area 2, with a cross formed inside. The vertical and horizontal lines included in the four sides and cross of the pattern A14 are parallel to the vertical and horizontal lines included in the four sides of the shielded area 2 when the vehicle V is parked in a parking space and there is no rotation or distortion in the overhead image P1.

[0059] Therefore, by detecting the parallelism of the vertical and horizontal lines included in the four sides and crosshairs of pattern A14 with respect to the vertical and horizontal lines included in the four sides of shielded area 2, image evaluation unit 12 can evaluate the rotation of overhead image P1, as shown in Fig. 12. For example, image evaluation unit 12 calculates the rotation angle of overhead image P1 based on pattern A14.

[0060] 13, the image evaluation unit 12 can evaluate the reduction of the overhead image P1. Similarly, the image evaluation unit 12 can evaluate the enlargement of the overhead image P1. For example, the image evaluation unit 12 calculates the enlargement and reduction rates of the overhead image P1 based on the pattern A14. Here, it is assumed that information on the size of the pattern A14 relative to the shielded area 2 when the overhead image P1 is neither enlarged nor reduced is set in advance.

[0061] In this way, the fourth evaluation method for the overhead image P1 can calculate not only the amount of distortion of the overhead image P1 but also the amount of rotation and magnification, allowing the image evaluation unit 12 to accurately calculate the distortion. Note that the quadrangle of the pattern A14 may be either a square or a rectangle.

[0062] (Fifth evaluation method for overhead image P1) A fifth evaluation method for the overhead image P1 according to the first embodiment will be described with reference to FIGS.

[0063] 14 shows an example of a simulation in which a stopped vehicle V drives straight and parks in a parking space (shielded area 2). For example, a running simulation of the vehicle V is performed using the above-mentioned MILS.

[0064] At time t=0, a vehicle V is located outside the shielded area 2. At time t=0, the vehicle V is stationary.

[0065] Then, the vehicle V is driven, and the driving condition of the vehicle V is determined based on the floor surface information 31. For example, the desired speed v and steering angle st of the vehicle V are determined based on the unevenness of the road surface, the road surface condition, the weather, the positional relationship between the parking space and the vehicle V, etc. This determination is made multiple times at predetermined time intervals between time t=0 and time T1.

[0066] At time t=T1, parking of the vehicle V is completed and the vehicle V overlaps the shielded area 2. At this time, the overhead image generating unit 11 synthesizes the multiple camera images 21 to generate an overhead image P1.

[0067] The image evaluation unit 12 evaluates the distortion of the generated overhead image P1 using the above-described first to fourth evaluation methods for the overhead image P1. Note that the overhead image generation unit 11 may synthesize the overhead image P1 while the vehicle V is traveling, and the image evaluation unit 12 may evaluate the distortion of the overhead image P1.

[0068] Unlike FIG. 14, FIG. 15 is an example of a simulation in which a stopped vehicle V turns and parks in a parking space (shielded area 2).

[0069] At time t=0, a vehicle V is located outside the shielded area 2. At time t=0, the vehicle V is stationary.

[0070] Then, the vehicle V is driven, and the driving condition of the vehicle V is determined based on the floor surface information 31. For example, the desired speed v and steering angle st of the vehicle V are determined based on the unevenness of the road surface, the road surface condition, the weather, the positional relationship between the parking space and the vehicle V, etc. This determination is made multiple times at predetermined time intervals between time t=0 and time T2.

[0071] At time t=T2, parking of the vehicle V is completed and the vehicle V overlaps with the shielded area 2. At this time, the overhead image generating unit 11 synthesizes the multiple camera images 21 to generate an overhead image P1.

[0072] The image evaluation unit 12 evaluates the distortion of the generated overhead image P1 using the above-described first to fourth evaluation methods. Note that the overhead image generation unit 11 may synthesize the overhead image P1 while the vehicle V is traveling, and the image evaluation unit 12 may evaluate the distortion of the overhead image P1.

[0073] At time t=T1 or time t=T2, the image evaluation unit 12 may evaluate distortion of the overhead image P1 when the vehicle V passes through the shielded area 2. In this case, unlike when the vehicle V is stopped, the image evaluation unit 12 can evaluate distortion of the overhead image P1 due to the moving speed, turning, etc. of the vehicle V.

[0074] In the fifth evaluation method of the overhead image P1, the image evaluation unit 12 can calculate the amount of distortion of the overhead image P1 even when the vehicle V is not stopped and is moving, thereby assisting the vehicle V in its travel.

[0075] (Sixth evaluation method of overhead image P1) A sixth evaluation method for the overhead image P1 according to the first embodiment will be described with reference to Fig. 16. Fig. 16 is a diagram showing an example of a pattern used in the sixth evaluation method for the overhead image P1 according to the first embodiment.

[0076] In the sixth evaluation method for the overhead image P1, the image evaluation unit 12 evaluates the overhead image P1 generated by a real vehicle test in which the vehicle V is actually driven.

[0077] The sixth evaluation method for the overhead image P1 is an example of a case where a real-vehicle test is conducted in which the overhead image P1 is corrected while the vehicle V is actually traveling at a driving school or test course where the road surface is modeled after a crosswalk at a scramble intersection, as shown in Figure 16.

[0078] In the sixth evaluation method for the overhead image P1, lines A16 and A17 are added to a road surface on which a pattern A15 simulating a pedestrian crossing at a scramble intersection has already been painted, and a real-vehicle test is performed by running a vehicle V. By adding a specific pattern to an already existing pattern in this way, the effort required for preparing for the real-vehicle test can be reduced.

[0079] Instead of the lines A16 and A17, the above-mentioned patterns A11, A11a, A12, and A13, or a combination of these patterns, may be drawn on the road surface. Also, any pattern may be drawn depending on the content of the actual vehicle test.

[0080] The patterns A11, A11a, A12, A13, A14, and A15 are examples of the first graphic.

[0081] (Second embodiment) In the first embodiment, a case has been described in which distortions of the camera images 21a to 21d constituting the overhead image P1 are evaluated. In the second embodiment, a case will be described in which deviations of the masked image P3 in the integrated overhead image P2 obtained by integrating the distortion-corrected overhead image P1 and the masked image P3 in the distortion-free overhead image P1 of the camera images 21a to 21d are evaluated.

[0082] <Evaluation equipment> An evaluation device 10 according to the second embodiment will be described with reference to Fig. 17. Note that components that are substantially the same as those in the first embodiment will be given the same reference numerals, and detailed descriptions thereof may be omitted.

[0083] As shown in FIG. 17, the evaluation device 10 includes an overhead image generation unit 11, an image evaluation unit 12, a shielded image generation unit 16, and an integrated overhead image generation unit 17.

[0084] Shielding image generation unit 16 references vehicle movement position information 24a and vehicle attitude information 24b and generates shielding image P3 based on overhead image P1 generated in advance by overhead image generation unit 11. In this way, because the overhead image P1 generated in advance is used to generate shielding image P3, shielding image generation unit 16 can generate shielding image P3 even if the road surface below the vehicle is blocked by the vehicle when viewed from above.

[0085] The integrated overhead image generating unit 17 integrates the overhead image P1 generated by the overhead image generating unit 11 and the shielded image P3 generated by the shielded image generating unit 16 to generate an integrated overhead image P2 including a pattern described below. The pattern included in the integrated overhead image P2 is an image of a pattern formed on the floor surface. The image evaluating unit 12 evaluates the integrated overhead image P2 generated by the integrated overhead image generating unit 17 with reference to floor surface information 31 and vehicle shape information 32. The other configurations are the same as those described in the first embodiment.

[0086] Also, as in the first embodiment, the image evaluation unit 12 feeds back the evaluation result to the overhead image generation unit 11. The overhead image generation unit 11 may generate the overhead image P1 by referring to the evaluation result fed back from the image evaluation unit 12 in addition to the camera image 21 and other information.

[0087] In the second embodiment, too, a case will be described in which the vehicle V is not actually driven, but the driving of the vehicle V and the generation of the integrated overhead image P2 are simulated by MILS (Model In the Loop Simulation) to evaluate the integrated overhead image P2. However, the present invention is not limited to this, and the vehicle V may actually be driven to generate the integrated overhead image P2, and the integrated overhead image P2 may be evaluated.

[0088] <Method for evaluating occluded image P3> The following describes a method for evaluating the integrated overhead image P2 by the image evaluation unit 12. The image evaluation unit 12 according to the second embodiment evaluates the integrated overhead image P2 using a plurality of methods. The methods for evaluating the integrated overhead image P2 by the image evaluation unit 12 will be described below in order.

[0089] (First evaluation method of occluded image P3) A first evaluation method for the shielded image P3 according to the second embodiment will be described with reference to Fig. 18. Fig. 18 is a diagram showing an example of a pattern used in the first evaluation method for the shielded image P3 according to the second embodiment.

[0090] The integrated overhead image P2 includes an occlusion image P3, a camera boundary line 3, and a pattern B11.

[0091] The pattern B11 is a pattern for evaluating distortion between the 44 camera images 21a to 21d. The pattern B11 includes a first rectangle B11a that is larger than the occluded image P3 and a second rectangle B11b that is smaller than the occluded image P3. Each side of the first rectangle B11a and the second rectangle B11b is parallel to each side of the occluded image P3 and the integrated overhead image P2. The pattern B11 also includes a diagonal line B11c that is common to the first rectangle B11a and the second rectangle B11b.

[0092] The parallelism and distance between the first rectangle B11a and the second rectangle B11b are compared to determine whether the masked image P3 has been scaled or rotated. For example, if the sides of the first rectangle B11a and the second rectangle B11b are parallel, their ratio is constant, and the angles of the diagonals B11c are the same, the image evaluation unit 12 determines that the masked image P3 has not been scaled or rotated. Note that information on the ratio of the sides of the first rectangle B11a and the second rectangle B11b when there is no scaling is assumed to be set in advance.

[0093] In the first evaluation method for the masked image P3, the image evaluation unit 12 determines whether the masked image P3 has been enlarged, reduced, or inverted based on the pattern B11.

[0094] (Second evaluation method of occluded image P3) A second evaluation method for the masked image P3 according to the second embodiment will be described with reference to FIGS.

[0095] The pattern B12 shown in FIG. 19 includes a first quadrangle B12a that is larger than the shielding image P3, and a line segment B12b that crosses the shielding image P3 and the first quadrangle B12a.

[0096] The image evaluation unit 12 compares the position of the portion of the line segment B12b that crosses the masked image P3 with the position of the other portion to evaluate the distortion in the vertical direction of the masked image P3. The image evaluation unit 12 can also determine that there is no distortion in the portion of the integrated overhead image P2 other than the masked image P3 by detecting the parallelism of the lines of the first quadrangle B12a.

[0097] 20 includes a first rectangle B13a that is larger than the blocked image P3, and line segments B13b, B13c, and B13d that intersect the blocked image P3 and the first rectangle B13a. Unlike the pattern B12, the pattern B13 includes three line segments B13b, B13c, and B13d, which increases the accuracy of the image evaluation unit 12 in determining the distortion of the blocked image P3 compared to the pattern B12.

[0098] In the second evaluation method for the masked image P3, the image evaluation unit 12 determines the vertical deviation of the masked image P3 based on the pattern B12 or the pattern B13.

[0099] (Third evaluation method of occluded image P3) A first evaluation method for the masked image P3 according to the second embodiment will be described with reference to FIGS.

[0100] The pattern B14 shown in FIG. 21 includes a first quadrangle B14a that is larger than the shielding image P3, and a line segment B14b that crosses the shielding image P3 and the first quadrangle B14a.

[0101] The image evaluation unit 12 compares the position of the portion of the line segment B14b that vertically crosses the masked image P3 with the position of the other portion to evaluate the distortion in the left-right direction of the masked image P3. The image evaluation unit 12 can also determine that there is no distortion in the portion of the integrated overhead image P2 other than the masked image P3 by detecting the parallelism of the lines of the first quadrangle B14a.

[0102] 22 includes a first rectangle B15a that is larger than the blocked image P3, and line segments B15b, B15c, and B15d that run vertically through the blocked image P3 and the first rectangle B15a. Unlike the pattern B14, the pattern B15 includes three line segments B15b, B15c, and B15d, which increases the accuracy of the image evaluation unit 12 in determining the distortion of the blocked image P3 compared to the pattern B14.

[0103] In the third evaluation method for the masked image P3, the image evaluation unit 12 can evaluate the deviation of the masked image P3 in the left-right direction based on the pattern B14 or the pattern B15.

[0104] (Fourth evaluation method of occluded image P3) A fourth evaluation method for the masked image P3 according to the second embodiment will be described with reference to FIGS.

[0105] The pattern B16 shown in FIG. 23 includes a first quadrangle B16a larger than the shielding image P3, a line segment B16b crossing the shielding image P3 and the first quadrangle B16a, and a line segment B16c crossing the shielding image P3 and the first quadrangle B16a.

[0106] The image evaluation unit 12 can simultaneously determine the distortion in the left-right and up-down directions of the shading image P3 by comparing the positions of the part of the line segment B16b that crosses the shading image P3 with the other parts, and the positions of the part of the line segment B16b that crosses the shading image P3 with the other parts.

[0107] The pattern B17 shown in Figure 24 includes a first rectangle B17a that is larger than the shielding image P3, line segments B17b, B17c, and B17d that intersect the shielding image P3 and the first rectangle B17a, and line segments B17e, B17f, and B17g that intersect the shielding image P3 and the first rectangle B17a.

[0108] The pattern B17 has a larger number of line segments that cross the first quadrangle B17a and line segments that run vertically therethrough than the pattern B16, and therefore the accuracy with which the image evaluation unit 12 determines the distortion of the masked image P3 is improved.

[0109] In the fourth evaluation method for the masked image P3, the image evaluation unit 12 can simultaneously evaluate the vertical deviation and the horizontal deviation of the masked image P3 based on the pattern B14 or the pattern B15.

[0110] The patterns B11, B12, B13, B14, B15, B16, and B17 are examples of the first graphic.

[0111] (Visualization of integrated overhead image P2) With reference to FIG. 25, visualization of the evaluation result of the integrated overhead image P2 by the image evaluation unit 12 will be described.

[0112] 25(b) indicates the amount of distortion (amount of deviation) in the up-down direction of the integrated overhead image P2 calculated by the image evaluation unit 12 for nine regions obtained by dividing the integrated overhead image P2 shown in FIG. 25(a) into three regions vertically and horizontally. In the value C1, the amount of distortion in the up direction is indicated as a positive value, and the amount of distortion in the down direction is indicated as a negative value.

[0113] 25(c) indicates the amount of distortion (amount of deviation) in the left-right direction of the integrated overhead image P2 calculated by the image evaluation unit 12 for nine regions obtained by dividing the integrated overhead image P2 shown in FIG. 25(a) into three regions vertically and horizontally. In the value C2, the amount of distortion in the right direction is indicated as a positive value, and the amount of distortion in the left direction is indicated as a negative value.

[0114] This allows the user to visually recognize the amount of distortion in the integrated overhead image P2. Furthermore, since the numerical values ​​C1 and C2 numerically indicate the amount of distortion in the nine regions of the integrated overhead image P2, the user can visually recognize the amount of local distortion in the integrated overhead image P2. Note that the number of divisions of the integrated overhead image P2 may be changed depending on the required accuracy and the processing power of the computer that calculates the amount of distortion.

[0115] (Vectorization) A vector D shown in FIG. 25(f) is a diagram obtained by the image evaluation unit 12 vectorizing the numerical values ​​C1 and C2.

[0116] For example, the image evaluation unit 12 displays the vector for nine regions obtained by dividing the integrated overhead image P2 vertically and horizontally into three, with the value indicated by the value C1 as the y component of the vector and the value indicated by the value C2 as the x component of the vector, thereby allowing the user to more intuitively understand the amount of distortion.

[0117] (Heat map) The heat maps E1 and E2 shown in FIGS. 25(d) and 25(e) are heat maps created by the image evaluation unit 12 based on the numerical values ​​C1 and C2.

[0118] The image evaluation unit 12 creates a heat map E1 based on the numerical value indicated by the numerical value C1. Since the numerical value C1 indicates the amount of distortion in the vertical direction, the heat map E1 is a heat map that indicates the amount of distortion in the vertical direction.

[0119] The heat map E1 is a diagram that represents the magnitude of numerical values ​​using color shading and hue, and the image evaluation unit 12 determines the color shading based on the numerical value shown in the numerical value C1. For example, areas where the image has shrunk due to distortion are colored darker, and areas where the image has stretched are colored lighter.

[0120] The heat map E1 represents the magnitude of the numerical values ​​by the shade of color or hue, and the image evaluation unit 12 determines the shade of color based on the numerical value indicated by the numerical value C1. For example, areas where the image has shrunk due to distortion are dark in color, and areas where the image has stretched are light in color.

[0121] (Fifth evaluation method for occluded image P3) A fifth evaluation method for the masked image P3 according to the second embodiment will be described with reference to FIGS.

[0122] Patterns B18a and B18b have a first region S11 painted in a first color (e.g., white) outside the boundary of the occluded image P3, second regions S21 and S22 painted in a second color (e.g., green) inside and outside the first region S11, and a third region S32 painted in a third color (e.g., red) inside the second region S21 and outside the second region S22.

[0123] Furthermore, pattern B18b has a first region S12 on the innermost side that is painted in a first color (e.g., white), a second region S22 on the outside of first region S12 that is painted in a second color (e.g., green), and a third region S32 on the outside of second region S22 that is painted in a third color (e.g., red).

[0124] Here, the widths of the second regions S21 and S22 and the size of the third region S31 are determined by the magnitude of the allowable distortion of the masked image P3. For example, the greater the allowable distortion of the masked image P3, the greater the widths of the second regions S21 and S22 and the smaller the size of the third region S31.

[0125] The image evaluation unit 12 evaluates the image misalignment between the integrated overhead image P2 and the masked image P3 using the second image. Specifically, the image evaluation unit 12 evaluates the misalignment between the integrated overhead image P2 and the masked image P3 based on whether the third region S31 contacts the first region S11.

[0126] The second image includes a first region S11 outside the boundary of the integrated overhead image P2, a second region S21 or S22 outside the first region, and a third region S31 smaller than the blocked region 2.

[0127] When the image evaluation unit 12 makes a judgment, the image evaluation unit 12 may refer to either the second image including the first region S11, the second region S21, and the third region S31, or the second image including the first region S11, the second region S22, and the third region S31. Furthermore, when a user makes a judgment by visual inspection, the evaluation result referring to the second image including the second region S21 is easier for the user to judge.

[0128] The first regions S11 and S12, the second regions S21 and S22, and the third regions S31 and S32 may be any colors, but it is preferable that the colors have different RGB values ​​in terms of distinguishability. The degree of distortion of the masked image P3 is expressed by the positional relationship of the regions with different colors.

[0129] For example, in the integrated overhead image P2' shown in Figure 27(b), the shielded image P3 is shifted to the right, so the third region S31 contacts the first region S11 in the area F1. In this case, an NG judgment is made by the image evaluation unit 12. On the other hand, in the integrated overhead image P2 shown in Figure 27(a), although the shielded image P3 is shifted to the right, the third region S31 does not contact the first region S11, so an OK judgment is made.

[0130] 28(b), the masked image P3 is shifted upward, so the third region S31 contacts the first region S11 in area F2. In this case, the image evaluation unit 12 judges the image as NG. On the other hand, in the integrated overhead image P2 shown in FIG. 28(a), the masked image P3 is shifted upward, but the third region S31 does not contact the first region S11, so the image evaluation unit 12 judges the image as OK.

[0131] For example, if a third region S31 painted in a third color (e.g., red) contacts a first region S11 painted in a first color (e.g., white), the image evaluation unit 12 will make an NG judgment. The integrated overhead image P2 may be displayed on a display (not shown) provided in the evaluation device 1, and in this case, a user may make a visual judgment.

[0132] (Sixth evaluation method for occluded image P3) A sixth evaluation method for the masked image P3 according to the second embodiment will be described with reference to FIGS.

[0133] Patterns B19a and B19b are lattice-like patterns and are arranged in two locations, one on the left and one on the right, across the shielded image P3 and the other portion of the integrated overhead image P2 other than the shielded image P3. Patterns B19a and B19b have portions that overlap with the shielded image P3 and portions that do not overlap with the shielded image P3. Image evaluation unit 12 detects corners of patterns B19a and B19b from the integrated overhead image P2 and obtains the coordinates of the corners of patterns B19a and B19b. Image evaluation unit 12 calculates the amount of distortion of the shielded image P3 based on the obtained coordinates of the corners of patterns B19a and B19b.

[0134] It is desirable that the patterns B19a and B19b are provided so as to be located near the tires of the vehicle V. This is because the distortion of the blocked image P3 can be evaluated more accurately near the tires where parameters for deriving the amount of movement of the vehicle V are obtained.

[0135] 30 is a diagram showing an example of a method for determining distortion of the masked image P3. In the following, a case where the pattern B19b is used to determine distortion will be described, but the same method can also be used to determine distortion when the pattern B19a is used.

[0136] The image evaluation unit 12 calculates the deviations in the front-back and left-right directions by referring to the coordinates of the corners of the pattern B 19b. For example, the image evaluation unit 12 uses an image recognition algorithm to obtain the coordinates of the corners of the pattern B 19b.

[0137] The image evaluation unit 12 acquires the coordinates of the top, bottom, right, and left edges of the pattern B19b. If the boundary of the masked image region exists in the vertical direction for the pattern B19b, the image evaluation unit 12 acquires the coordinates of the top, bottom, and right edges of the portion that overlaps with the masked image P3 and the top, bottom, and left edges of the portion that does not overlap with the masked image P3.

[0138] Then, the image evaluation unit 12 calculates the difference between the upper ends of the patterns B19b and the difference between the lower ends of the patterns B19b to calculate the amount of distortion in the up-down direction (front-to-back direction of the vehicle V) of the shielded image P3. Note that in Fig. 30, the difference between the upper ends is -10 pixels for the front-to-back deviation (top), and the difference between the lower ends is -17 pixels for the front-to-back deviation (bottom), so the average is -13.5 pixels.

[0139] The image evaluation unit 12 also acquires the horizontal size of pattern B19b and sets this as the true value. The image evaluation unit 12 then calculates the horizontal width (measured value) of pattern B19b using the difference in coordinates between the right and left edges of pattern B19b, and calculates the amount of horizontal distortion of pattern B19b by comparing this with the true value. The image evaluation unit 12 then determines whether the amounts of vertical and horizontal distortion are equal to or less than predetermined thresholds, and if they are equal to or less than the thresholds, makes a judgment of OK, and if they are greater than the thresholds, makes a judgment of NG. In FIG. 30, the true value is 200 pixels and the measured value is 189 pixels, so the horizontal deviation is 11 pixels.

[0140] Furthermore, when determining the amount of distortion of the masked image P3, patterns B20a and B20b shown in FIG. 31 may be used instead of the grid patterns B19a and B19b. These patterns B20a and B20b are obtained by replacing part of the grid pattern shown in FIG. 29 with a geometric pattern having a specific pattern. By using patterns B20a and B20b, the image evaluation unit 12 can determine the amount of distortion by pattern determination rather than by numerical determination. For example, patterns B20a and B20b may be any image that allows pattern determination, such as a two-dimensional code.

[0141] (Seventh evaluation method for occluded image P3) A seventh evaluation method for the masked image P3 according to the second embodiment will be described with reference to FIGS.

[0142] 32 shows an example of a simulation in which a stopped vehicle V drives straight and parks in a parking space (shielded area 2). For example, a running simulation of the vehicle V is performed using the above-mentioned MILS.

[0143] At time t=0, a vehicle V is located outside the shielded area 2. At time t=0, the vehicle V is stationary.

[0144] Then, the vehicle V is driven, and the driving condition of the vehicle V is determined based on the floor surface information 31. For example, the desired speed v and steering angle st of the vehicle V are determined based on the unevenness of the road surface, the road surface condition, the weather, the positional relationship between the parking space and the vehicle V, etc. This determination is made multiple times at predetermined time intervals between time t=0 and time T3.

[0145] At time t=T3, vehicle V completes parking and overlaps with the masked area 2. At this time, multiple camera images 21 are combined together to generate overhead image P1 by overhead image generation unit 11, and masked image generation unit 16 generates masked image P3. Using the generated overhead image P1 and masked image P3, integrated overhead image generation unit 17 generates integrated overhead image P2.

[0146] The image evaluation unit 12 evaluates the distortion of the generated integrated overhead image P2 using the above-described first to sixth evaluation methods for the occluded image P3. Note that the integrated overhead image generation unit 17 may synthesize the integrated overhead image P2 while the vehicle V is traveling, and the image evaluation unit 12 may evaluate the distortion of the integrated overhead image P2.

[0147] Unlike FIG. 32, FIG. 33 is an example of a simulation in which a stopped vehicle V turns and parks in a parking space (shielded area 2).

[0148] At time t=0, a vehicle V is located outside the shielded area 2. At time t=0, the vehicle V is stationary.

[0149] Then, the vehicle V is driven, and the driving condition of the vehicle V is determined based on the floor surface information 31. For example, the desired speed v and steering angle st of the vehicle V are determined based on the unevenness of the road surface, the road surface condition, the weather, the positional relationship between the parking space and the vehicle V, etc. This determination is made multiple times at predetermined time intervals between time t=0 and time T4.

[0150] At time t=T4, vehicle V completes parking and overlaps with the masked area 2. At this time, multiple camera images 21 are combined together to generate overhead image P1 by overhead image generation unit 11, and masked image generation unit 16 generates masked image P3. Using the generated overhead image P1 and masked image P3, integrated overhead image generation unit 17 generates integrated overhead image P2.

[0151] The image evaluation unit 12 evaluates the distortion of the generated integrated overhead image P2 using the above-described first to sixth evaluation methods for the occluded image P3. Note that the integrated overhead image generation unit 17 may synthesize the integrated overhead image P2 while the vehicle V is traveling, and the image evaluation unit 12 may evaluate the distortion of the integrated overhead image P2.

[0152] At time t=T3 or time t=T4, a simulation may be performed in which vehicle V passes through shielded area 2, and image evaluation unit 12 evaluates distortion of integrated overhead image P2 when vehicle V passes through shielded area 2. In this case, unlike when vehicle V is stopped, image evaluation unit 12 can evaluate distortion of integrated overhead image P2 due to the moving speed, turning, etc. of vehicle V.

[0153] In the seventh evaluation method for the occluded image P3, the image evaluation unit 12 can calculate the amount of distortion of the integrated overhead image P2 even when the vehicle V is not stopped and is moving, thereby assisting the vehicle V in its travel.

[0154] (Eighth evaluation method for occluded image P3) With reference to FIG. 34, an eighth evaluation method for the masked image P3 according to the second embodiment will be described.

[0155] In the eighth evaluation method for the occluded image P3, a real-world test is conducted in which the integrated overhead image P2 is corrected while the vehicle V is actually driven on a driving school or test course where the road surface is made of brick cobblestones or where a pattern resembling brick cobblestones is painted on the road surface.

[0156] In this vehicle test, a vehicle V is driven using a brick cobblestone pattern B21 painted on the road surface, which is separated into left and right halves by covering the center of the pattern, as shown in Figure 34. By adding a specific pattern to an already existing pattern in this way, the effort required for preparing for the vehicle test can be reduced.

[0157] In the eighth evaluation method for the occluded image P3, a mat B22 is laid on a road surface on which the pattern B21 has already been simulated, the pattern B21 is divided, and a real vehicle test is performed by running a vehicle V. By adding a specific pattern to the already existing pattern in this way, the effort required for preparing for the real vehicle test can be reduced.

[0158] As in the first embodiment, the actual vehicle test may be performed by painting any pattern on the road surface on which the pattern A15 simulating the scramble intersection used in the actual vehicle test shown in FIG. 16 is painted.

[0159] The items described in the above embodiments may be combined as appropriate unless they are contradictory or unless it is explicitly stated that they cannot be combined.

[0160] In the above-described embodiments, the notation "... part" used for each component may be replaced with other notations such as "... circuitry," "... assembly," "... device," "... unit," or "... module."

[0161] <Summary of the embodiment> An image evaluation method according to one embodiment of the present disclosure includes the steps of generating an integrated overhead image by integrating an overhead image, which is an image of an area near a vehicle, and an occlusion image, which is an image of an area that is blocked by the vehicle when viewed from above, and evaluating the image misalignment between the overhead image and the occlusion image, wherein the overhead image and the occlusion image include images of the same first figure that straddles the overhead image and the occlusion image in the integrated overhead image when there is no image misalignment.

[0162] With this configuration, it is possible to evaluate the deviation of the masked image P3 in the overhead image P1 without distortion of each of the camera images 21a to 21d, or in the integrated overhead image P2 obtained by integrating the overhead image P1 with the masked image P3 after correcting the distortion.

[0163] The present disclosure can be realized in software, hardware, or software in conjunction with hardware.

[0164] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the above-described embodiments may be combined in any manner without departing from the spirit of the disclosure.

[0165] Although specific examples of the present disclosure have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and alterations of the specific examples exemplified above. [Industrial Applicability]

[0166] An embodiment of the present disclosure is useful for an image evaluation method and an image evaluation program. [Explanation of symbols]

[0167] 1 Evaluation device 2 Shield area 3 Camera Boundary 10 Evaluation equipment 11. Bird's-eye view image generation unit 12 Image evaluation section 16. Occluded image generation unit 17 Integrated bird's-eye view image generation unit 21 camera images 22 Camera installation information 23 Distortion information 24a Vehicle movement location information 24b Vehicle attitude information 31 Floor information 32 Vehicle shape information 34 Vehicle steering information P1 overhead view image P2 Integrated overhead image P3 occlusion image

Claims

1. generating an integrated overhead image by integrating an overhead image, which is an image of an area near the vehicle, and an occlusion image, which is an image of an area that is blocked by the vehicle when viewed from above; evaluating an image misalignment between the overhead image and the occluded image; Including, the overhead image and the occlusion image include an image of the same first figure that straddles the overhead image and the occlusion image in the integrated overhead image when there is no misalignment of the images; Image evaluation methods.

2. The image evaluation method described in claim 1, wherein the overhead image further includes an image of a first rectangle larger than the occlusion image, and the occlusion image further includes an image of a second rectangle smaller than the occlusion image and having sides parallel to each side of the first rectangle.

3. The image evaluation method according to claim 2 , wherein the first figure is a diagonal line between the first rectangle and the second rectangle.

4. 2. The image evaluation method of claim 1, wherein the overhead image further includes an image of a first rectangle larger than the occluded image, and the first figure includes straight lines parallel to two sides of the first rectangle corresponding to the longitudinal direction of the vehicle.

5. 2. The image evaluation method of claim 1, wherein the overhead image further includes an image of a first rectangle larger than the occlusion image, and the first figure includes straight lines parallel to two sides of the first rectangle corresponding to the left-right direction of the vehicle.

6. 6. The image evaluation method according to claim 4, wherein the first figure has a plurality of straight lines parallel to two sides of the first rectangle.

7. 2. The image evaluation method of claim 1, wherein the overhead image further includes an image of a first rectangle larger than the occluded image, and the first figure includes a first straight line parallel to two sides of the first rectangle corresponding to the left-right direction of the vehicle, and a second straight line parallel to two sides of the first rectangle corresponding to the left-right direction of the vehicle.

8. The image evaluation method according to claim 7 , wherein the first straight line and the second straight line each include a plurality of straight lines parallel to each other.

9. The image evaluation method according to claim 1 , further comprising the step of displaying the amount of misalignment of the images as a vector or a heat map.

10. The image evaluation method according to claim 1 , wherein the first graphic includes a lattice pattern graphic.

11. The image evaluation method according to claim 10 , wherein the first graphic further includes a geometric design having a specific pattern.

12. 2. The image evaluation method according to claim 1, wherein the evaluation of the image misalignment is performed while the vehicle is traveling.

13. 2. The image evaluation method according to claim 1, wherein the first graphic includes a pattern of brick cobblestones separated into left and right by covering the center.

14. the overhead image is a composite of a plurality of camera images captured by a plurality of cameras, and the method further includes a step of evaluating a deviation of the plurality of camera images when the plurality of camera images are composited; 2. The image evaluation method according to claim 1, wherein the plurality of camera images includes an image of the same second figure that is configured to straddle a boundary line between the plurality of camera images in the overhead image when there is no misalignment between the plurality of camera images.

15. The image evaluation method according to claim 14 , wherein the second graphic includes a rectangle.

16. The image evaluation method according to claim 14 , wherein the second graphic includes a graphic at least a part of which is formed by a curve.

17. The image evaluation method according to claim 14 , wherein the second graphic includes a graphic in which each side of a quadrangle is replaced with a rhombus or a triangle.

18. The image evaluation method according to claim 14 , wherein the second graphic includes a plurality of concentric graphics.

19. 19. The image evaluation method according to claim 14, wherein the second graphic includes a graphic having, inside a quadrangle, a crosshair made up of two straight lines parallel to sides of the quadrangle.

20. The image evaluation method according to claim 14, wherein the evaluation of the image misalignment is performed while the vehicle is traveling.

21. The image evaluation method according to claim 14, wherein the second graphic includes a pattern of a pedestrian crossing at a scramble intersection.

22. generating an integrated overhead image by integrating an overhead image, which is an image of an area near the vehicle, and an occlusion image, which is an image of an area that is blocked by the vehicle when viewed from above; evaluating an image misalignment between the overhead image and the occluded image; on the computer, the overhead-view image and the occlusion image include an image of the same first figure that is configured to straddle the overhead-view image and the occlusion image in the integrated overhead-view image when there is no misalignment of the images; Image evaluation program.

23. generating an integrated overhead image by integrating an overhead image, which is an image of an area near the vehicle, and an occlusion image, which is an image of an area that is blocked by the vehicle when viewed from above; evaluating an image misalignment between the overhead image and the occluded image using a second image; Including, the second image includes a first region outside a boundary of the occlusion image, a second region outside the first region, and a third region smaller than the occlusion image; In the step of evaluating the misalignment of the images, the misalignment of the images is evaluated based on whether the third region contacts the first region; The first area, the second area, and the third area are colored in different colors. Image evaluation methods.

24. generating an integrated overhead image by integrating an overhead image, which is an image of an area near the vehicle, and an occlusion image, which is an image of an area that is blocked by the vehicle when viewed from above; evaluating an image misalignment between the integrated overhead image and the occluded image using a second image; on the computer, the second image includes a first region outside a boundary of the occlusion image, a second region outside the first region, and a third region smaller than the occlusion image; In the step of evaluating the misalignment of the images, the misalignment of the images is evaluated based on whether the third region contacts the first region; The first area, the second area, and the third area are colored in different colors. Image evaluation program.

Images