DEVICE FOR DETECTING A MOUNTING ANGLE FOR AN IN-VEHICLE CAMERA, DEVICE FOR CALIBRATING A MOUNTING ANGLE, AND METHOD FOR DETECTING A MOUNTING ANGLE

Abstract

Device (120, 220) for detecting a mounting angle, which detects a mounting angle in which an in-vehicle camera (10) for detecting an image of the ground around a vehicle is mounted on the vehicle, wherein the device for detecting a mounting angle comprises the following: a unit (101) for obtaining a captured image, which obtains a captured image of the ground on which a target marker is marked with two mutually parallel straight lines of equal width; a unit (109) for converting to a bird's-eye view which receives a pre-defined value of the mounting angle of the vehicle's in-vehicle camera (10) relative to the vehicle and converts the captured image into a bird's-eye view image assuming that the vehicle's in-vehicle camera (10) is mounted according to the pre-defined value, wherein the bird's-eye view image is an image in which the ground appearing in the captured image appears as if it were seen from above; a mounting angle search unit (110) which, by changing the predefined value of the mounting angle used in the conversion to a bird's-eye view, searches for the mounting angle in which both the parallelism of the two straight parts of the target marker in the bird's-eye view image and the difference in width between the two straight parts of the target marker in the bird's-eye view image are within predefined tolerance ranges; and a mounting angle determination unit (113) which determines the mounting angle found by the search as the mounting angle of the vehicle-internal camera (10) relative to the vehicle.

Description

[Technical field]

[0001] The present disclosure relates to a technique for capturing a mounting angle, wherein an in-vehicle camera is mounted on the vehicle to capture an image around the vehicle, and for calibrating the mounting angle. [State of the art]

[0002] Various driver assistance technologies have been implemented to allow a driver to easily assess the situation around the vehicle by displaying an image captured by an in-vehicle camera on a screen. One example of a technology used to assist with reversing is as follows: An in-vehicle camera is typically mounted at the rear of the vehicle, and during reversing, a rear-view image is displayed on a monitor, overlaying the captured image onto the screen to show a predicted trajectory of the vehicle.

[0003] An example of a technique used to allow a driver to easily assess the situation around a vehicle is as follows. In-vehicle cameras are mounted, in particular, on the front, rear, left, and right sides of a vehicle. Images captured by these in-vehicle cameras are transformed into a bird's-eye view (in which the vehicle's surroundings appear as seen from above), and this bird's-eye view image is displayed on a screen.

[0004] The techniques mentioned above are based on the assumption that the vehicle's internal cameras are mounted at the correct angles. Let us assume that there is a deviation in the mounting angle of an internal camera. In this case, for example, during reversing, a discrepancy occurs between the actual path and the predicted path, which is displayed by superimposing a rear-view image.Furthermore, if there is a deviation in the mounting angle, when the images captured by the in-vehicle cameras at the front, rear, right, and left of the vehicle are converted into a bird's-eye view image and displayed, a positional offset occurs, for example, between the image from the left in-vehicle camera in bird's-eye view and the image from the front in-vehicle camera in bird's-eye view. The same or a similar problem can occur between the right in-vehicle camera and the front in-vehicle camera, between the left in-vehicle camera and the rear in-vehicle camera, and between the right in-vehicle camera and the rear in-vehicle camera.

[0005] The mounting angle of the vehicle's in-vehicle camera relative to the vehicle was set at the time of delivery from the factory. However, there are instances where the mounting angle needs to be readjusted, for example, if a misalignment occurs due to prolonged use, or if the in-vehicle camera is removed for repairs.

[0006] Therefore, the following technique was presented (JP 2011-182 236 A). Specifically, in the technique disclosed in JP 2011-182 236 A, a vehicle is moved into a frame, such as a parking lot, which has a well-known shape, and the viewpoints of images captured by the vehicle's in-vehicle cameras at the front, rear, left, and right of the vehicle are converted. Thus, in the technique disclosed in JP 2011-182 236 A, the mounting angle of each in-vehicle camera relative to the vehicle is determined. Assume that the mounting angles of the in-vehicle cameras are correct. In this case, the bird's-eye view image obtained by converting the images captured by the in-vehicle cameras exhibits no positional offset, even at image connection points.

[0007] By focusing on this point, the technique disclosed in JP 2011-182 236 A determines the mounting angles of the vehicle's in-vehicle cameras relative to the vehicle in such a way that no positional offset occurs between the bird's-eye view images obtained by converting the images captured by the vehicle's in-vehicle cameras. In this way, if the mounting angle of each in-vehicle camera can be determined, the mounting angle can be adjusted to fall within a tolerance range; thus, the mounting angle can be calibrated.

[0008] WO 2015 / 190 066 A1 and EP 2 597 614 A1 provide further technological background in this field. [Summary of the invention] [Technical problem]

[0009] The proposed technique described above relies on the assumption that the vehicle's internal cameras are mounted at the front, rear, left, and right of the vehicle, and that images of the entire surroundings of the vehicle are captured using these internal cameras. If the internal camera is not mounted on any of the front, rear, left, or right sides of the vehicle (for example, if the internal cameras are only mounted at the front and rear), the mounting angles of the internal cameras relative to the vehicle cannot be detected or calibrated to correct them.

[0010] The present disclosure aims to provide a technique whereby, if a vehicle does not have an in-vehicle camera at the front, rear, right and left, the mounting angle of the in-vehicle camera relative to the vehicle can nevertheless be detected and calibrated to a correct mounting angle. [Solution to the problem]

[0011] A device for detecting the mounting angle according to one aspect of the technology of the present disclosure comprises the features listed in claim 1. Advantageous embodiments are the subject of the dependent claims.

[0012] Thus, according to the technology of the present disclosure, even if a vehicle does not have the vehicle's internal camera on each of the front, rear, left and right parts, the mounting angle of the vehicle's internal camera relative to the vehicle can be detected and calibrated to a correct mounting angle.

[0013] The target marker has two straight lines that are parallel to each other and of equal width. For example, the target marker could be a graphic such as a parking space or parking area, which has two side lines that are parallel to each other and of equal width, but no transverse line that intersects the side lines. Using the technique described in the present disclosure, the mounting angle of an in-vehicle camera 10 can be detected and calibrated to a correct mounting angle using such a target marker. [Brief explanation of the characters] Fig. Figure 1 shows a diagram of a vehicle in which an in-vehicle camera is mounted. Fig. Figure 2 is a block diagram showing the internal structure of a device for calibrating a mounting angle according to the first embodiment. Fig. Figure 3 is a flowchart showing a process for calibrating a mounting angle, performed by a device for calibrating a mounting angle according to the first embodiment. Fig. Figure 4 is a flowchart showing a process for calibrating a mounting angle, which is performed by a device for calibrating a mounting angle according to the first embodiment. Fig. Figure 5 shows the state in which a vehicle is positioned relative to side lines in a parking lot or parking space. Fig. Figure 6 shows the case of driving straight ahead or moving a vehicle in a straight line relative to a parking space. Fig. Figure 7A shows the case of capturing straight center lines and straight boundary lines of side lines from a captured image of a parking space. Fig. Figure 7B shows the case of capturing straight center lines and straight boundary lines of side lines from a captured image of a parking space. Fig. Figure 7C shows the case of capturing straight center lines and straight boundary lines of side lines from a captured image of a parking space. Fig. Figure 8 shows the case of collecting the captured straight center lines and straight boundary lines of side lines from numerous recorded images. Fig. 9A shows one reason why, by accumulating or collecting the detection results of straight center lines and straight boundary lines of the side lines, it can be determined whether a vehicle is aligned parallel to the side lines. Fig. 9B shows one reason why, by accumulating the detection results of straight center lines and straight boundary lines of the side lines, it can be determined whether a vehicle is aligned parallel to the side lines. Fig. Figure 10A shows the case of searching for a mounting angle of an in-vehicle camera, focusing on the parallelism of two captured side lines of a parking space and on a difference in width between the two side lines in a bird's-eye view image. Fig. Figure 10B shows the case of searching for a mounting angle of an in-vehicle camera, focusing on the parallelism of two captured side lines of a parking space and on a difference in width between the two side lines in a bird's-eye view image. Fig. Figure 10C shows the case of searching for a mounting angle of an in-vehicle camera, focusing on the parallelism of two captured side lines of a parking space and on a difference in width between the two side lines in a bird's-eye view image. Fig. Figure 10D shows the case of searching for a mounting angle of an in-vehicle camera, focusing on the parallelism of two captured side lines of a parking space and on a difference in width between the two side lines in a bird's-eye view image. Fig. Figure 11 shows one reason why, before searching for a mounting angle for an in-vehicle camera, it is checked that the orientation of a vehicle is parallel to the side lines of a parking space. Fig. Figure 12 shows the relationship between a vehicle-based coordinate system and a camera-based coordinate system. Fig. Figure 13 shows a method for converting a vehicle-based coordinate system into an analytical vehicle-based coordinate system. Fig. Figure 14 shows a method for converting an analytical vehicle-based coordinate system into a camera-based coordinate system. Fig. Figure 15 shows the case of converting a camera-based coordinate system into an image plane coordinate system. Fig. Figure 16 shows the conversion to a bird's-eye view. Fig. Figure 17 shows the case of performing a conversion of a recorded image into a bird's-eye view while changing the angle of an in-vehicle camera relative to a parking space. Fig. 18A indicates that the mounting angle of an in-vehicle camera can be calibrated even in a parking space that has no cross line intersecting two side lines. Fig. 18B indicates that the mounting angle of an in-vehicle camera can be calibrated even in a parking space that has no cross line intersecting two side lines. Fig. Figure 19 is a block diagram showing the internal structure of a device for calibrating a mounting angle according to the second embodiment. Fig. Figure 20 is a flowchart showing a process for calibrating a mounting angle, which is carried out by a device for calibrating a mounting angle according to the second embodiment. Fig. Figure 21A shows the case of finding a mounting angle for an in-vehicle camera by converting a representative recorded image into a bird's-eye view. Fig. Figure 21B shows the case of finding a mounting angle for an in-vehicle camera by converting a representative recorded image into a bird's-eye view. Fig. Figure 22A shows a variation of the search for a mounting angle of an in-vehicle camera, focusing on the positions of two side lines of a parking space and on the endpoints of the side lines. Fig. Figure 22B shows a variation of the search for a mounting angle of an in-vehicle camera, focusing on the positions of two side lines of a parking space and on the endpoints of the side lines. Fig. Figure 22C shows a variation of the search for a mounting angle of an in-vehicle camera, focusing on the positions of two side lines of a parking space and on the endpoints of the side lines. Fig. Figure 22D shows a variation of the search for a mounting angle of an in-vehicle camera, focusing on the positions of two side lines of a parking space and on the endpoints of the side lines. [Description of the embodiments]

[0014] The technology explained above in the present disclosure is described below by means of specific embodiments as examples with reference to the figures. A. First embodiment: A-1. Device setup:

[0015] Fig. Figure 1 shows an exemplary diagram of a vehicle 1 on which an internal camera 10 is mounted. In the example shown in the figure, the internal camera 10 is mounted at an angle to the rear of the vehicle 1, thus capturing a view behind the vehicle 1 from a slightly elevated angle. The image captured by the internal camera 10 is displayed on a monitor 6 by an image generation device 20. This allows the driver to visually check the image displayed on the monitor 6. Consequently, the view behind the vehicle 1 can be inspected.

[0016] In the following description, it is assumed that the vehicle's internal camera 10 is mounted at the rear of the vehicle 1. However, this is not a limitation. The vehicle's internal camera 10 can also be mounted at the front of the vehicle 1.

[0017] The image generation device 20 includes, for example, a distortion correction function and an image conversion function for an image captured by the vehicle's in-vehicle camera 10. The distortion correction function converts the image captured by the vehicle's in-vehicle camera 10, which is taken from an oblique angle above, into an image that is easily viewable by removing distortions caused by an optical system of the vehicle's in-vehicle camera 10. The resulting image (the captured image after distortion correction) is then displayed on the monitor 6. The image conversion function converts the captured image after distortion correction into a bird's-eye view image. The resulting image (the bird's-eye view image in which the vehicle's surroundings appear as if viewed (directly) from above) is then displayed on the monitor 6.The image generation device 20 includes, for example, a storage unit 21, such as a memory. The storage unit 21 stores, for example, data for removing distortion due to the optical system (distortion correction data) and data on the mounting angle of the vehicle's internal camera 10 at the time of conversion to a bird's-eye view. When it receives the image captured by the vehicle's internal camera 10, the image generation device 20 converts the image using the data read from the storage unit 21. This generates the image to be displayed on the monitor 6. In this way, a driver can visually check the image displayed on the monitor 6. Accordingly, the situation around the vehicle 1 can be easily assessed.

[0018] Here, the conversion of the captured image to a bird's-eye view is based on the assumption that the vehicle's internal camera 10 is mounted at the correct angle on the vehicle 1. The mounting angle of the vehicle's internal camera 10 was set to the correct angle when the vehicle 1 was shipped from the factory. However, a misalignment of the mounting angle can occur for some reason, such as after prolonged use. If such a misalignment of the mounting angle occurs, the bird's-eye view image will be distorted.

[0019] Thus, in the present embodiment, the vehicle 1 comprises a device 100 for calibrating the mounting angle, or a mounting angle calibration device 100, which calibrates the mounting angle of the vehicle's internal camera 10 relative to the vehicle 1. The device 100 for calibrating the mounting angle according to the present embodiment is connected to the image generation device 20. The device 100 for calibrating the mounting angle receives the captured image from the image generation device 20 after distortion correction. Based on the captured image it has received, the device 100 for calibrating the mounting angle searches for a suitable adjustment or mounting angle of the vehicle's internal camera 10. A method for searching for the mounting angle of the vehicle's internal camera 10 is described later.If a suitable mounting angle is successfully found, the mounting angle calibration device 100 changes the mounting angle value stored in the memory unit 21 of the image generation device 20 to the value of the mounting angle found by the search. Thus, the mounting angle calibration device 100 calibrates the mounting angle of the vehicle's in-vehicle camera 10.

[0020] In order for the device 100 to calibrate the mounting angle and find a suitable mounting angle for the vehicle's internal camera 10 by searching, the vehicle 1 must move a suitable distance or route while the steering wheel 2 is in a straight position (in a steering state for straight-line travel). Thus, in the present embodiment, for example, a steering angle sensor 3, which detects the steering angle of the steering wheel 2, and a distance sensor 4, which detects the distance traveled by the vehicle 1, are connected to the device 100 for calibrating the mounting angle. When calibrating the mounting angle of the vehicle's internal camera 10, the device 100 for calibrating the mounting angle monitors, for example, the steering state of the steering wheel 2 and the distance traveled by the vehicle 1.Consequently, the device 100 for calibrating the mounting angle issues a warning using the following procedure if the steering wheel 2 is not in the steering position for straight-ahead driving or if the vehicle 1 has not moved (if the condition for finding a suitable mounting angle is not met). More precisely, for example, the monitor 6 is used to display a warning image, or a speaker 5 is used to emit a warning tone. Thus, the device 100 for calibrating the mounting angle requires a recalibration of the mounting angle of the vehicle's in-vehicle camera. A-2. Internal structure of the device 100 for calibrating the mounting angle:

[0021] Fig. Figure 2 shows an exemplary block diagram illustrating the internal structure of the device 100 for calibrating the mounting angle according to the present embodiment. As in Fig. As illustrated in Figure 2, the device 100 for calibrating the mounting angle according to the present embodiment comprises a unit 101 for receiving a captured image, a detection unit 102 for a straight center line, a detection unit 103 for a straight boundary line, a storage unit 104 for a captured straight line, a parallel determination unit 105, a vehicle monitoring unit 106, and an alarm output unit 107. Furthermore, the device 100 for calibrating the mounting angle comprises: a receiving unit 108 for a predefined value; a unit 109 for converting to a bird's-eye view; a mounting angle search unit 110, which includes a bird's-eye view image evaluation unit 111 and a unit 112 for changing a predefined value; a mounting angle determination unit 113; and a mounting angle update unit 114.It should be noted that the section of the device 100 for calibrating the mounting angle excludes the mounting angle update unit 114 (which is indicated by the dashed line in . Fig. 2 enclosed section), corresponds to a device 120 for detecting the mounting angle of the present embodiment.

[0022] The units mentioned above are abstract concepts used to describe the internal structure of the device 100 for calibrating the mounting angle, with a focus on its function of detecting and calibrating the mounting angle of the vehicle's in-vehicle camera 10. Thus, the above description does not indicate that the device 100 for calibrating the mounting angle is necessarily physically subdivided into the respective units mentioned above. The units mentioned above can be provided, for example, by the following method.In particular, each of the aforementioned units can, for example, be implemented as a program executed by a central processing unit (CPU) contained in a microcomputer, or it can be implemented as an electronic component with a highly integrated circuit (LSI), memory, and the like. Furthermore, these can be combined to implement the aforementioned units. In the present embodiment, the mounting angle calibration device 100 comprises a microcomputer, which mainly includes a CPU, memory (for example, a non-volatile, computer-readable physical storage medium such as read-only memory (ROM), and random access memory (RAM)). Thus, each of the aforementioned units is implemented as a program executed by the CPU.

[0023] The unit 101 for obtaining a captured image receives an image from the image generation device 20, which is obtained by removing distortions caused by the optical system of the vehicle's internal camera 10 from the captured image (the captured image after distortion correction). After detecting the mounting angle of the vehicle's internal camera 10, the unit 101 for obtaining a captured image receives an image of the ground on which a predefined target marker is marked or indicated.

[0024] A target marker is a graphic consisting of two straight lines drawn on the ground, parallel to each other and of equal width, for example, around / within a parking space. For instance, a parking space, which designates a parking spot for a vehicle in a parking lot or similar, includes side lines that indicate the boundaries between the parked vehicle and vehicles on either side of it. The side lines of the parking space correspond to two straight lines that are parallel to each other and of equal width. Thus, the parking space can be used as a representative target marker. Specific examples of target markers are described later.

[0025] The straight centerline detection unit 102 detects two straight line segments of the target marker contained in the image captured by unit 101 to obtain a captured image, and detects a straight centerline corresponding to the centerline of each of the straight line segments. The target marker comprises two straight line segments. Thus, the straight centerline detection unit 102 detects one straight centerline for each of the straight line segments, i.e., a total of two straight centerlines. The straight centerline detection unit 102 stores the two detected straight centerlines in the detected straight line storage unit 104.

[0026] The straight boundary line capture unit 103 captures two straight line segments of the target marker contained in the image acquired by unit 101 to obtain a captured image, and captures straight boundary lines, or in short, straight boundary lines, corresponding to boundary lines on either side of each of the straight line segments. The target marker comprises two straight line segments. Thus, the straight boundary line capture unit 103 captures two straight boundary lines for each of the straight line segments, i.e., a total of four straight boundary lines. The straight boundary line capture unit 103 stores the four captured straight boundary lines in the captured straight line storage unit 104. Therefore, the numerous straight center lines and straight boundary lines captured from the numerous captured images are stored in the captured straight line storage unit 104.In other words, the storage unit 104 for detected straight lines corresponds to a predefined storage area, such as a memory contained in the device 100 for calibrating the mounting angle.

[0027] Specific examples in which two straight center lines and four straight boundary lines are captured from the target marker will be described later.

[0028] The parallel determination unit 105 reads and analyzes the numerous straight center lines and straight boundary lines stored in the memory unit 104 for detected straight lines. Based on the analysis result, the parallel determination unit 105 determines whether the orientation of vehicle 1 is parallel to the two straight segments of the target marking drawn on the ground. A procedure for determining whether the orientation is parallel to the two straight line segments is described with reference to other figures.

[0029] In order for the parallel determination unit 105 to correctly determine the orientation of vehicle 1 relative to the two straight line segments of the target marking, the numerous recorded images must be preserved in the state in which vehicle 1 is moving straight ahead by at least a predetermined distance.

[0030] Thus, in the present embodiment, the vehicle monitoring unit 106 monitors the state of the vehicle 1 during the period in which the unit 101 receives a captured image. The vehicle monitoring unit 106 is connected to the steering angle sensor 3 and the distance sensor 4. The vehicle monitoring unit 106 monitors the steering position of the steering wheel 2, the distance traveled by the vehicle 1, and the like. If the steering wheel 2 is turned or the distance traveled by the vehicle 1 is insufficient during the period in which the numerous captured images are received, the vehicle monitoring unit 106 determines that the captured image must be reacquired. Consequently, the vehicle monitoring unit 106 discards the acquisition results for the straight center lines and the straight boundary lines that are stored in the memory unit 104 for captured straight lines.Furthermore, the vehicle monitoring unit 106 outputs the determination result to the alarm output unit 107, indicating that the captured image must be recaptured.

[0031] The alarm output unit 107 is connected to the loudspeaker 5, the monitor 6, and the like. When it receives a result from the vehicle monitoring unit 106 indicating that the captured image needs to be recaptured, the alarm output unit 107 issues a corresponding warning via the loudspeaker 5, the monitor 6, and the like.

[0032] If it is determined that the orientation of vehicle 1 is not parallel to the two straight parts of the target marker, the parallel determination unit 105 discards the detection results of the straight center lines and the straight boundary lines that are stored in the memory unit 104 for detected straight lines. The parallel determination unit 105 outputs the detection result, indicating that the detected image must be recaptured, to the alarm output unit 107. The alarm output unit 107 issues a corresponding warning via the loudspeaker 5, the monitor 6, and the like.

[0033] If, on the other hand, the parallel determination unit 105 determines that the vehicle 1 is oriented parallel to the two straight parts of the target marking, the unit 108 receives a predetermined value of the mounting angle of the vehicle's internal camera 10 from the image generation device 20 in order to obtain a predetermined value. The mounting angle of the vehicle's internal camera 10 is stored in the storage unit 21 in the image generation device 20.

[0034] When the parallel determination unit 105 determines that the orientation of vehicle 1 is parallel to the two straight segments of the target marker, the bird's-eye view conversion unit 109 also converts the captured image into a bird's-eye view image as follows. Specifically, the bird's-eye view conversion unit 109 reads the captured image obtained by the capture unit 101. At this point, the bird's-eye view conversion unit 109 reads the captured image that is the last among the captured images obtained by the capture unit 101.The unit 109 for converting to a bird's-eye view converts the recorded image into a bird's-eye view image under the assumption that the mounting angle of the vehicle's internal camera 10 and the mounting angle read by the storage unit 21 in the image generation device 20 match each other.

[0035] The bird's-eye view image evaluation unit 111 assesses the parallelism of the two straight line segments of the target marker and the difference in width between them by analyzing the bird's-eye view image resulting from the conversion by unit 109. If both the parallelism and the difference in width are outside predefined tolerance ranges, the bird's-eye view image evaluation unit 111 outputs this evaluation result to unit 112 to modify a predefined value.Consequently, to change a predefined value based on the input evaluation result, unit 112 modifies the predefined value of the mounting angle of the vehicle's in-vehicle camera 10, which it receives from the storage unit 21 in the image generation device 20, and outputs a value after the change to unit 109 for conversion to the bird's-eye view.

[0036] Thus, Unit 109 performs the bird's-eye view conversion again. Specifically, Unit 109 performs the bird's-eye view conversion again using the mounting angle after the change for the captured image, thereby generating a new bird's-eye view image. The bird's-eye view image evaluation unit 111 evaluates the parallelism of the two straight-line segments of the target marker and the difference in width between the two straight-line segments on the new bird's-eye view image resulting from the conversion by Unit 109.If the bird's-eye view image evaluation unit 111 determines that both the parallelism and the difference in width are not within the predefined tolerance ranges, the unit 112 changes the value of the mounting angle of the vehicle's in-vehicle camera 10 again to change the predefined value.

[0037] In this way, the bird's-eye view image evaluation unit 111 and the unit 112 for changing the predefined value repeat the operations described above until a predefined condition is met. Specifically, the unit 112 for changing the predefined value changes the mounting angle of the in-vehicle camera 10 based on the evaluation result of the bird's-eye view image. The bird's-eye view image evaluation unit 111 then converts the captured image back into a bird's-eye view image using the mounting angle after the change and performs the evaluation again. In other words, the bird's-eye view image evaluation unit 111 and the unit 112 for changing the predefined value change the predefined value of the mounting angle of the in-vehicle camera 10 to be used in the conversion to a bird's-eye view and re-evaluate the captured image.In this way, the mounting angle at which the two straight lines of the target marker in the bird's-eye view image meet predefined criteria is found by the search. Thus, the bird's-eye view image evaluation unit 111 and the unit 112 for changing the predefined value correspond to the mounting angle search unit 110.

[0038] If it is determined that the two straight lines of the target marker in the bird's-eye view image meet the predefined criteria, the bird's-eye view image evaluation unit 111 outputs the value of the mounting angle used to the mounting angle determination unit 113, which was used at that time during the conversion to the bird's-eye view. Consequently, the mounting angle determination unit 113 determines the input value of the mounting angle as a correct value of the mounting angle (an actual value of the mounting angle). Thus, according to the present embodiment, the device 120 for detecting the mounting angle can detect a correct mounting angle of the vehicle's in-vehicle camera 10.

[0039] Furthermore, the mounting angle update unit 114 receives the correct mounting angle of the vehicle's internal camera 10, which was determined by the mounting angle determination unit 113, and writes the received mounting angle to the storage unit 21 in the image generation device 20. The mounting angle update unit 114 thus updates the value of the mounting angle stored in the storage unit 21 in the image generation device 20 to the correct value. As described above, the device 100 performs the processes of the aforementioned units to calibrate the mounting angle according to the present embodiment. This completes the calibration of the mounting angle of the vehicle's internal camera 10, which is to be used by the image generation device 20 when generating an image (an image to be displayed on the monitor 6).

[0040] The processes performed by the device 100 for calibrating the mounting angle according to the present embodiment, which has the internal structure described above when calibrating the mounting angle of the vehicle's internal camera 10, are described below by means of specific examples. A-3. Mounting angle calibration procedure:

[0041] The Fig. 3 and Fig. Figure 4 illustrates a flowchart of the mounting angle calibration process performed by the mounting angle calibration device 100 according to the present embodiment. Note that the present process is performed by the CPU contained within the mounting angle calibration device 100. Specifically, the CPU in the mounting angle calibration device 100 reads a program for the mounting angle calibration process from a predefined memory location of the memory or the like (a non-volatile, computer-readable physical storage medium) and executes the program.

[0042] As in Fig. Figure 3 shows an example of how, in the process of calibrating the mounting angle according to the first embodiment, the device 100 for calibrating the mounting angle first provides an instruction for fulfilling the conditions required for calibrating the mounting angle (the conditions relating to the alignment of the vehicle 1 and the steering wheel position 2). More precisely, the loudspeaker 5 and the monitor 6 are used to issue an instruction that the vehicle 1 should be aligned parallel to the side lines of the parking space, and furthermore, that the steering wheel 2 should be set to the steering position for driving or moving in a straight line or for driving straight ahead (step S100).

[0043] For example, assume that, as in Fig. Figure 5 illustrates an example where a parking space T with two parallel side lines a and b of equal width and a transverse line c perpendicular to these side lines a and b is painted on the ground. In this case, the device 100 calibrates the mounting angle by aligning the vehicle 1 parallel to the two side lines a and b and also emits a sound from the loudspeaker 5 indicating that the steering wheel 2 in the vehicle should be positioned in a straight-line position (in the state where the steering wheel 2 is straight, without being turned left or right). Furthermore, the device 100 displays an image on the monitor 6 showing a corresponding instruction.

[0044] In the present embodiment, the parking space T corresponds to the “target marking” and the two side lines a and b correspond to “the two straight parts of the target marking”. Furthermore, the target marking need not necessarily be the parking space shown by way of example in Figure 5, as long as it is a graphic that has two parallel straight parts of the same width.

[0045] By following the instructions, a driver moves vehicle 1 so that it is aligned parallel to the side lines a and b of parking space T. Note that the driver does not need to align vehicle 1 exactly parallel to the side lines a and b of parking space T.

[0046] When the instruction is given that the conditions regarding the orientation of the vehicle 1 and the steering wheel position 2 should meet the predefined conditions, the device 100 for calibrating the mounting angle by means of the loudspeaker 5 and the monitor 6 gives the instruction that the vehicle 1 should be moved or driven a predefined distance (e.g., two meters) in a straight line or straight ahead (step S101 in Fig. 3). Fig. Figure 6 illustrates the case in which vehicle 1 is moved straight ahead relative to parking space T by a predetermined distance L. Following the instruction, the driver moves vehicle 1 straight ahead by the predetermined distance L relative to parking space T. Note that it is sufficient that the distance by which vehicle 1 is moved straight ahead is not less than the predetermined distance L; the distance does not have to be exactly the predetermined distance L.

[0047] The device 100 for calibrating the mounting angle receives the recorded image of the parking space T from the image generation device 20 (step S102 in Fig. 3) At this time, the image captured by the image-generating device 20 is the captured image after distortion correction (the image obtained from the captured image by removing distortions due to the optical system of the vehicle's in-vehicle camera 10).

[0048] The device 100 for calibrating the mounting angle analyzes the captured image and detects straight center lines and straight boundary lines of the side lines a and b of the parking space T. The device 100 then stores this information in a predefined memory area (the memory unit 104 for detected straight lines). Fig. 2) of the memory or the like, the slopes of the captured straight center lines and straight boundary lines, the intersection positions of the captured straight center lines and the intersection positions of the captured straight boundary lines or the boundary line intersection positions (step S103).

[0049] The Fig. 7A, Fig. 7B and Fig. Figure 7C illustrates the case of capturing the straight center lines and the straight boundary lines of the side lines a and b of parking space T by analyzing the recorded image. As in Fig. As shown in Figure 7A, the side lines a and b of the parking space T and the transverse lines c appear perpendicular to the side lines a and b in the captured image. Since the distortions caused by the optical system of the vehicle's internal camera 10 were removed from the captured image, the left and right side lines a and b in the image are straight lines with predefined widths.

[0050] Thus, the device 100 for calibrating the mounting angle detects a straight center line which is an extension of the center line of each of the left and right side lines a and b. Fig. Figure 7B shows an example of a straight center line bc, which is captured from the lateral line b in the captured image, and a straight center line ac, which is captured from the lateral line a in the captured image.

[0051] Furthermore, the device 100 for calibrating the mounting angle for each of the left and right side lines a and b detects two straight boundary lines which are extensions of the inner boundary line and the outer boundary line of the parking space T. Fig. Figure 7B shows an example of an inner straight boundary line bi, an outer straight boundary line bo, and a straight center line bc, which are captured from the side line b in the captured image, and an inner straight boundary line ai, an outer straight boundary line ao, and a straight center line ac, which are captured from the side line a in the captured image.

[0052] Three straight lines, which are the straight center line bc, the inner straight boundary line bi, and the outer straight boundary line bo of the side line b, and three straight lines, which are the straight center line ac, the inner straight boundary line ai, and the outer straight boundary line ao of the side line a, are captured from one image. Thus, a total of six straight lines are captured from one image.

[0053] After detecting the six straight lines as described above, the device 100 for calibrating the mounting angle detects the slope of each of the straight lines. Fig. 7B illustrates the case of capturing a slope kbi of the inner straight boundary line bi, which is captured from the side line b contained in the captured image. Furthermore, it illustrates Fig. 7B the case of detecting a slope kai of the inner straight boundary line ai, which is detected from the side line a contained in the captured image. Furthermore, the device 100 for calibrating the mounting angle detects slopes kbc, kbo, kac and kao of the respective straight lines, namely the straight center line bc, the straight boundary line bo, the straight center line ac and the straight boundary line ao (which are not shown in the figures).

[0054] Furthermore, the device 100 for calibrating the mounting angle detects the intersection positions of the straight center lines bc and ac. The intersection position means the coordinate position of the intersection point between the straight center line and a predefined edge, which is determined in the captured image. Fig. Figure 7C shows an example of defining an edge DL (hereinafter referred to as "a test edge DL") for testing at a position between the lower and upper edges of the captured image, parallel to the lower edge. In this case, the mounting angle calibration device 100 detects the coordinate position of the intersection point between the test edge DL and the straight center line bc as an intersection position pbc of the straight center line bc. The mounting angle calibration device 100 detects the coordinate position of the intersection point between the test edge DL and the straight center line ac as an intersection position pac of the straight center line ac.

[0055] Furthermore, the device 100, used to calibrate the mounting angle, detects a boundary line intersection position of each of the four straight boundary lines bo, bi, ao, and ai. The boundary line intersection position represents the coordinate position of the intersection point between the straight boundary line and the predefined test edge DL, which is specified in the captured image. As shown in Fig. As illustrated in Figure 7C, the device 100 for calibrating the mounting angle detects boundary line intersection positions pbo, pbi, pao and pai of the respective straight boundary lines bo, bi, ao and ai.

[0056] The present embodiment illustrates by way of example the case in which a straight line parallel to the bottom edge of the captured image is set as the test edge DL, but this is not limiting. For example, the test edge DL can be a straight line set between the left and right edges of the captured image and parallel to either the left or right edge. Alternatively, any of the top, bottom, left, or right edges of the captured image can be selected and used as the test edge DL. As yet another example, test edges DL can be used that are provided separately for the two straight center lines bc and ac and the four straight boundary lines bo, bi, ao, and ai.

[0057] As described above, the device 100 performs the following processing in step S103 for calibrating the mounting angle. Fig. 3. In particular, the device 100 for calibrating the mounting angle detects the slope, the axis intercept position, etc., of each of the two detected straight center lines. Furthermore, the device 100 for calibrating the mounting angle detects the slope, the boundary line intersection position, etc., of each of the four straight boundary lines. The device 100 for calibrating the mounting angle stores the detection results in memory (the memory unit for the detected straight lines 104 in Fig. 2).

[0058] The device 100 for calibrating the mounting angle determines whether the steering wheel 2 is held in the straight-ahead position (step S104). As described above, the device 100 for calibrating the mounting angle instructs the driver of vehicle 1 to move the steering wheel 2 into the straight-ahead position and to drive vehicle 1 straight ahead (steps S100 and S101). Thus, in the processing step S104, a judgment of "yes" is made if the driver does not turn the steering wheel 2.

[0059] If the judgment "Yes" is made in processing step S104, the mounting angle calibration device 100 determines whether the results of the aforementioned acquisition process have been saved for a predetermined number of captured images (for example, 100 images) (step S107). Note that in this acquisition process, the captured images are analyzed, and the slopes and intersection positions of the straight center lines, as well as the slopes and boundary line intersection positions of the straight boundary lines, are recorded. Subsequently, in this process, the acquisition results for captured straight-line data are stored in the acquisition unit 104. Initially, only one captured image is processed. Thus, the judgment "No" is made in step S107.

[0060] If the judgment “No” is made in the processing in step S107, the device 100 for calibrating the mounting angle determines whether the travel distance of the vehicle 1 has reached the pre-defined distance L (step S108).

[0061] If it is determined that the journey distance of vehicle 1 has not reached the predefined distance L (step S108: No), the device 100 for calibrating the mounting angle receives another recorded image of the parking space T (step S102) and analyzes the recorded image. The device 100 then detects the slopes and the intersection positions of the straight center lines with the slopes and the boundary line intersection positions of the straight boundary lines and stores the detected results in the memory unit 104 for detected straight lines (step S103).

[0062] Thus, each time a captured image is obtained, the slopes and intersection positions of the straight center lines and the slopes and boundary line intersection positions of the straight boundary lines are collected in the memory of device 100 for calibrating the mounting angle. More precisely, the slopes and intersection positions of the two straight center lines and the slopes and boundary line intersection positions of the four straight boundary lines are collected in the memory unit 104 for captured straight lines each time a captured image is obtained. Fig. Figure 8 schematically illustrates the case of accumulating slopes kbc and kac and intersection positions pbc and pac of the straight center lines bc and ac with slopes kbo, kbi, kao and kai and boundary line intersection positions pbo, pbi, pao and pai of straight boundary lines bo, bi, ao and ai in the memory of the device 100 for calibrating the mounting angle.

[0063] As described above, the device 100 for calibrating the mounting angle determines that the steering wheel 2 is not held in the steering angle position for straight-ahead driving (step S104 in Fig. 3: No), if the driver turns the steering wheel 2 during the time period in which the detection results of the straight center lines and the straight boundary lines are accumulated.

[0064] If the judgment “No” is made in step S104 of the processing, the device 100 for calibrating the mounting angle discards the detection results for the straight center lines and the straight boundary lines (the data shown as an example in Fig. 8 are shown), which are collected in memory (step S105). Subsequently, the device 100 for calibrating the mounting angle issues the warning and the instruction that the steering wheel 2 should be held in the steering position for straight-ahead driving using the speaker 5 and the monitor 6 (step S106).

[0065] By returning to the beginning of the processing, the device 100 for calibrating the mounting angle again issues the instruction that the vehicle 1 should be aligned parallel to the side lines a and b of the parking space T and that the steering wheel 2 should be in the straight-ahead position (step S100). The device 100 for calibrating the mounting angle then repeats the processing described above in steps S102 to S108. As in Fig. Figure 8 shows, by way of example, the acquisition results for the straight center lines and the straight boundary lines are accumulated in the memory in the device 100 for calibrating the mounting angle.

[0066] If it is determined that the vehicle 1's travel distance has reached the predetermined distance L (step S108: Yes) before the acquisition results for the predetermined number of captured images have been accumulated (step S107: No), the device 100 performs the following process to calibrate the mounting angle. Specifically, the loudspeaker 5 and the monitor 6 are used to issue an instruction that the direction of travel of vehicle 1 should be reversed and that vehicle 1 should move straight ahead by a predetermined distance L (step S109). For example, if vehicle 1 has been traveling in reverse, an instruction is issued that vehicle 1 should move forward by the predetermined distance L while the steering wheel 2 is held in the straight-ahead position.If vehicle 1 has been moving forward, an alternative instruction is issued that vehicle 1 should travel back the predetermined distance L while the steering wheel 2 is held in the position for driving straight ahead.

[0067] After receiving another captured image of parking space T (step S102), the device 100 continues the process of accumulating the straight center lines and straight boundary lines that were captured by analyzing the captured image (steps S103 to S108) to calibrate the mounting angle.

[0068] In device 100 for calibrating the mounting angle, the straight center lines and straight boundary lines captured by the recorded images are accumulated by repeating such processes. Ultimately, the number of recorded images in which the straight center lines and straight boundary lines were captured and accumulated reaches the predetermined number. Thus, in the process described in S107, the judgment "Yes" is made.

[0069] Once the accumulation of the straight center lines and straight boundary lines, acquired from the predetermined number of captured images, is complete, the device 100 can determine, for calibrating the mounting angle, whether the vehicle 1 is aligned parallel to the side lines a and b of the parking space T as instructed. A method for determining the orientation of the vehicle 1 is described below. First, the basic concept for determining the orientation of the vehicle 1 is explained with reference to the Fig. 9A and Fig. 9B described.

[0070] It is assumed that the orientation of vehicle 1 is parallel to the side lines a and b of parking space T. If vehicle 1 moves while the steering wheel 2 is held in the straight-ahead position, the relative positional relationship between vehicle 1 and parking space T changes. Fig. Figure 9A shows an example of the relative positional relationship between vehicle 1 and parking space T when the position of parking space T changes relative to the position of vehicle 1 (in the state where the position of vehicle 1 is fixed).

[0071] As in Fig. Figure 9A illustrates that the axis of parking space T in the direction of movement corresponds to the axis of vehicle 1 in one direction of its actual movement. Vehicle 1 moves straight ahead while the steering wheel 2 is held in the straight-ahead position. Therefore, the axis of vehicle 1 in its direction of movement corresponds to the axis of vehicle 1 in the direction in which vehicle 1 is oriented. The driver steers vehicle 1 so that it is aligned parallel to the side lines a and b of parking space T. Consequently, the axis of parking space T in its direction of travel is in Fig. 9A parallel to the two side lines a and b. In other words, the direction of movement of the parking space T corresponds to that of the two side lines a and b. Therefore, if the parking space T moves relative to vehicle 1, the side lines a and b of the parking space T move as shown in Fig. 9A illustrates on the respective extensions of sidelines a and b.

[0072] If vehicle 1 is oriented parallel to the side lines a and b of parking space T, then the straight center lines ac, which are captured from the side lines a in the numerous images taken while vehicle 1 is moving straight ahead, correspond to the same straight line. Similarly, the straight center lines bc, which are captured from the side lines b, are the same straight line. The outer straight boundary lines ao, which are captured from the side lines a, become the same straight line, and the inner straight boundary lines ai, which are captured from the side lines a, become the same straight line. Similarly, the outer straight boundary lines bo, which are captured from the side lines b, become the same straight line, and the inner straight boundary lines bi, which are captured from the side lines b, become the same straight line.

[0073] In contrast, it is assumed that the orientation of vehicle 1 is not parallel to the side lines a and b of the parking space T. Fig. Figure 9B illustrates the relative positional relationship between vehicle 1 and parking space T when the position of parking space T changes relative to the position of vehicle 1. As shown in Fig. Figure 9B shows an example of how the positions of the side lines a and b change relative to vehicle 1 when the position of the parking space T changes.

[0074] In the case where the orientation of vehicle 1 is not parallel to the side lines a and b of parking space T, then, if the captured images are obtained while vehicle 1 is moving straight ahead, the straight center line ac, captured from side line a in each of the captured images, is a different straight line. Similarly, the straight center line bc, captured from side line b in each of the captured images, is a different straight line. Both the outer straight boundary line ao and the inner straight boundary line ai, captured from side line a in each of the captured images, are different straight lines, and each of the outer straight boundary line bo and the inner straight boundary line bi, captured from side line b in each of the captured images, is a different straight line.

[0075] In light of the foregoing, the orientation of vehicle 1 can be determined by the following procedure. Specifically, the mounting angle calibration device 100 stores the straight center lines and straight boundary lines captured from each of the recorded images. Using the straight line data stored therein, the mounting angle calibration device 100 determines whether the straight center lines in the recorded images are the same straight line. Similarly, the mounting angle calibration device 100 determines whether the straight boundary lines in the recorded images are the same straight lines. If the straight lines in the recorded images can be considered the same straight line, the mounting angle calibration device 100 determines that the orientation of vehicle 1 is parallel to the side lines a and b of the parking space T.

[0076] In the process of calibrating the mounting angle in the Fig. 3 and Fig. 4 such a principle is used to determine that the orientation of vehicle 1 is parallel to the side lines a and b of the parking space T.

[0077] This is particularly evident in steps S110 and S111 in Fig. 4 shown. First, the device 100 determines the mounting angle for calibration when the accumulation of straight center lines and straight boundary lines from the pre-defined number of captured images is complete (step S107 in Fig. 3: yes), whether the scatter of the slopes and the intersection positions of the accumulated straight centerlines is within a predefined tolerance range (step S110). In other words, the device 100 for calibrating the mounting angle determines whether the scatter of the slopes and the intersection positions of the accumulated straight centerlines is within the predefined tolerance range if the accumulation of the straight centerlines and the straight boundary lines satisfies a predefined condition.

[0078] This finding is made as follows. As above based on Fig. As described in section 8, the slopes and intersection positions of each of the straight centerlines bc and ac are accumulated. Thus, during processing in step S110, the slopes kbc of the straight centerline bc are read, and their standard deviation σkbc is calculated. Furthermore, the slopes kac of the straight centerline ac are read, and their standard deviation σkac is calculated. Similarly, during processing in step S110, the intersection positions pbc of the straight centerline bc are read, and their standard deviation σpbc is calculated. Furthermore, the intersection positions pac of the straight centerline ac are read, and their standard deviation σpac is calculated.

[0079] In step S110, the processing determines whether the calculated variance σkbc, σkac, σpbc, and σpac lie within a predefined tolerance range. Note that the tolerance range (e.g., the upper and lower limits) is predefined for each value of the calculated variance.

[0080] Consequently, the device 100 for calibrating the mounting angle determines whether the variation of the bevels and the boundary line intersection positions of the straight boundary lines are within the predefined tolerance range (step S111), if it is determined that any variation of the bevels and the intersection positions of the straight center lines is within the predefined tolerance range (step S110 in Fig. 4: yes). In other words, the device 100 for calibrating the mounting angle determines whether the scatter of the slopes and the boundary line intersection positions from the accumulated straight boundary lines are within the specified tolerance range, if both the slope and the intersection position of the straight center line are within the predefined tolerance range.

[0081] This determination is made as follows. Similar to the straight center line as above, based on... Fig. As described in section 8, for each of the straight boundary lines bo, bi, ao, and ai, the slopes and intersection positions of the boundary lines with the straight boundary line are accumulated. Thus, in processing step S111, the slopes kbo and kbi of the straight boundary lines bo and bi are read, and their respective variances σkbo and σkbi are calculated. Similarly, in processing step S111, the slopes kao and kai of the straight boundary lines ao and ai are read, and their respective variances σka and σkai are calculated.

[0082] Furthermore, in the processing step S111, the boundary line intersection positions pbo, pbi, pao and pai of the straight boundary lines bo, bi, ao and ai are read, and their corresponding dispersion σpbo, σpbi, σpao and σpai is calculated.

[0083] In the processing step S111, it is determined whether the calculated dispersion σkbo, σkbi, σka, σkai, σpbo, σpbi, σpao and σpai lies within the predefined tolerance range.

[0084] If, as a result, it is determined that any variation in the slopes and the boundary line intersection positions of the straight boundary lines lies within the predefined tolerance range (step S111 in Fig. 4: Yes), the device 100 for calibrating the mounting angle determines that the straight center lines bc and ac in the recorded images are the same straight line (their positions in the images coincide). Similarly, the straight boundary lines in the recorded images are determined to be the same straight line. More precisely, the device 100 for calibrating the mounting angle determines that the positional relationship between the vehicle 1 and the parking space T in the Fig. The state shown in Figure 9A is present when both the slope and the intersection position of the straight center line are within the predefined tolerance range, and the slope and the intersection position of the straight boundary line are also within the predefined tolerance range. Therefore, in this case, the orientation of vehicle 1 can be determined to be parallel to the side lines a and b of parking space T.

[0085] In contrast, if it is determined that the dispersion of the slopes and / or the intersection positions of the straight center lines is not within the predefined tolerance range (step S110 in Fig. 4: no), the device 100 for calibrating the mounting angle determines that the orientation of the vehicle 1 is not parallel to the side lines a and b of the parking space T.

[0086] Similarly, the device 100 for calibrating the mounting angle can determine that the orientation of the vehicle is not parallel to the side lines a and b of the parking space T if it is determined that the scatter of the slopes and / or the boundary line intersection positions of the straight boundary lines are not within the predefined tolerance range (step S111: No).

[0087] If, during processing in step S110 or step S111 as described above, the judgment “No” is made, the device 100 for calibrating the mounting angle outputs information via the loudspeaker 5 and the monitor 6 indicating that the orientation of the vehicle 1 is not parallel to the side lines a and b of the parking space T (step S112). The device 100 for calibrating the mounting angle discards the detection results of the straight center lines and the straight boundary lines (which are in Fig. 8 exemplary data shown), which are accumulated in memory (step S113). In addition, the device 100 calibrates the mounting angle by returning to the beginning of the processing, the processing described above in step S100 in Fig. 3 to step S111 in Fig. 4 through.

[0088] As described above, in the present embodiment, the orientation of the vehicle 1 is determined to be parallel to the side lines a and b of the parking space T when the following two conditions are met. One of the conditions, i.e., the first condition, is that the variation in the slopes and the intersection positions of the straight center lines lies within the predefined tolerance range. The second condition is that the variation in the slopes and the intersection positions of the straight boundary lines lies within the predefined tolerance range. However, if either of the aforementioned first and second conditions is met, the device 100 for calibrating the mounting angle can determine that the orientation of the vehicle 1 is parallel to the side lines a and b of the parking space T.

[0089] If, for example, the variation in the slopes and intersection positions of the straight center lines lies within the predefined tolerance range, the device 100 for calibrating the mounting angle can determine that the orientation of the vehicle 1 is parallel to the side lines a and b of the parking space T. Thus, it is sufficient for the device 100 for calibrating the mounting angle to accumulate the slopes and intersection positions of the straight center lines; the device 100 for calibrating the mounting angle does not need to accumulate the slopes and intersection positions of the straight boundary lines.

[0090] If the variation in the slopes and intersection positions of the straight boundary lines is within the predefined tolerance range, the device 100 for calibrating the mounting angle can determine that the orientation of the vehicle 1 is parallel to the side lines a and b of the parking space T. Therefore, it is sufficient for the device 100 to accumulate the slopes and intersection positions of the straight boundary lines; the device 100 for calibrating the mounting angle does not need to accumulate the slopes and intersection positions of the straight center lines.

[0091] If the device 100 has successfully confirmed, by the procedure described above, that the alignment of the vehicle 1 is parallel to the side lines a and b of the parking space T (step S111 in Fig. 4: Yes), the device 100 receives the captured image for calibrating the mounting angle (step S114). The captured image obtained at this time can be a new one obtained from the image-generating device 20. Alternatively, a most recent image obtained from the image-generating device 20 can be used from a predetermined number of captured images obtained for accumulating the straight center lines and the straight boundary lines.

[0092] The device 100 for calibrating the mounting angle receives the predefined value of the mounting angle of the vehicle's internal camera 10, which is stored in the memory unit 21 in the image generation device 20 (step S115 in Fig. 4) As described above, the image generation device 20 has a function for converting an image captured by the vehicle's internal camera 10 into a bird's-eye view image and for outputting the bird's-eye view image to the monitor 6. To convert the image captured by the vehicle's internal camera 10 into a bird's-eye view image, the value of the mounting angle of the vehicle's internal camera 10 is required. Therefore, the preset value of the mounting angle of the vehicle's internal camera 10 is stored in the memory unit 21 in the image generation device 20. Thus, in processing step S115, this predefined value of the mounting angle is obtained from the image generation device 20.

[0093] The device 100 for calibrating the mounting angle converts the received captured image into a bird's-eye view image (step S116 in Fig. 4) The mounting angle of the vehicle's internal camera 10, which is used at this time during the conversion to the bird's-eye view, is the pre-defined value of the mounting angle obtained from the image generation device 20.

[0094] The device 100 for calibrating the mounting angle determines, by analyzing the obtained bird's-eye view image, whether the parallelism of the side lines a and b of the parking space T, which appears in the bird's-eye view image, is within the predefined tolerance range (step S117).

[0095] For example, let's assume that the recorded image is the one in Fig. The image shown in 10A is shown. Furthermore, it is assumed that the one in Fig. The bird's-eye view image shown in 10B is obtained by converting the captured image into a bird's-eye view. As shown in Fig. As illustrated in Figure 10B, parking space T in the bird's-eye view image has a distorted shape. If the mounting angle of the vehicle's in-vehicle camera 10, used in the bird's-eye view conversion, is correct, parking space T in the resulting bird's-eye view image will have the actual shape of parking space T (a shape where the two side lines a and b are parallel and of equal width). Therefore, the fact that parking space T has a distorted shape in the bird's-eye view image means that the mounting angle of the vehicle's in-vehicle camera 10, used in the bird's-eye view conversion, differs from the actual mounting angle.

[0096] Thus, in the processing step S117 in Fig. 4. The image is analyzed from a bird's-eye view, the sidelines a and b of parking space T are detected, and the straight centerline bc of sideline b and the straight centerline ac of sideline a are detected. Furthermore, during processing in step S117, it is determined whether the parallelism of the straight centerline bc of sideline b and the straight centerline ac of sideline a lies within the predefined tolerance range. Note that the tolerance range (e.g., the upper limit and the lower limit) is predefined.

[0097] If it is determined that the parallelism of the straight center line bc of the side line b to the straight center line ac of the side line a is not within the predefined tolerance range (step S117 in 4: no), the device 100 for calibrating the mounting angle consequently determines that the parking space T in the bird's-eye view image has a distorted shape and the mounting angle of the vehicle's in-vehicle camera 10, which is used in the conversion to the bird's-eye view, is not correct.

[0098] The device 100 for calibrating the mounting angle changes the predefined value of the mounting angle (step S119) and, using the mounting angle after the change, reconverts the captured image into a bird's-eye view image (step S116). Subsequently, by analyzing the resulting bird's-eye view image, the device 100 for calibrating the mounting angle detects the straight center lines of the side lines a and b of the parking space T in the bird's-eye view image and determines whether the parallelism of the detected straight center lines is within the predefined tolerance range (step S117).

[0099] Although details are described later, the mounting angle of the in-vehicle camera 10 comprises the following angles. Specifically, the mounting angle includes a pitch direction angle (hereinafter referred to as the "pitch direction angle θp"), a yaw direction angle (hereinafter referred to as the "yaw direction angle θy"), and a roll direction angle (hereinafter referred to as the "roll direction angle θr"). The pitch direction is the direction in which the angle of the optical axis of the in-vehicle camera 10 changes vertically relative to the vehicle 1. The yaw direction is the direction in which the angle of the optical axis of the in-vehicle camera 10 changes horizontally relative to the vehicle 1. The roll direction is the direction in which the in-vehicle camera 10 rotates about its center about the optical axis.

[0100] In the processing in step 119, which is carried out after the judgment “No” in the processing in step S117 in Fig. 4 was felled, preferably mainly the predefined values ​​of the pitch direction angle θp and the yaw direction angle θy are changed in order to change the predefined value of the mounting angle of the vehicle's internal camera 10.

[0101] As described above, the device 100 for calibrating the mounting angle changes the mounting angle of the vehicle's in-vehicle camera 10 and repeats the conversion of the captured image to a bird's-eye view until the predefined condition is met (steps S119 and S116 in Fig. 4) For example, let us assume that from this we can derive the following: Fig. The image shown in 10C is from a bird's-eye view. As in Fig. As illustrated in Figure 10C, the sidelines a and b of parking space T are parallel in the bird's-eye view image, but parking space T has a distorted shape. Furthermore, the width wb of sideline b is greater than the width wa of sideline a; sideline b and sideline a are not the same width (wb > wa). Fig. 10C). Even if the parallelism of the two side lines a and b is within the predefined tolerance range, it is therefore not possible to determine, based on this result, that the mounting angle of the vehicle's internal camera 10 is correct.

[0102] If it is thus determined that the parallelism of the side lines a and b of the parking space T lies within the predefined tolerance range (step S117 in Fig. 4: Yes), the device 100 for calibrating the mounting angle detects the width wb of side line b and the width wa of side line a and determines whether the difference in width between them is within a predefined tolerance range (step S118).

[0103] If, therefore, it is determined that the difference in width between the two sidelines a and b is not within the predefined tolerance range (step S118 in Fig. 4: No), the device 100 for calibrating the mounting angle determines that the mounting angle of the vehicle's in-vehicle camera 10 used in the bird's-eye view is incorrect and changes the predefined value of the mounting angle again (step S119). The device 100 for calibrating the mounting angle then converts the captured image back into a bird's-eye view image using the modified mounting angle (step S116). Subsequently, the device 100 for calibrating the mounting angle analyzes the resulting bird's-eye view image, detects the side lines a and b of the parking space T in the bird's-eye view image, and determines whether the parallelism of the detected side lines a and b is within the predefined tolerance range and whether the difference in width between the detected side lines a and b is within the predefined tolerance range (steps S117 and S118).

[0104] In the processing in step 119, which is carried out after the judgment “No” has been made in the processing in step S118, preferably mainly the preset value of the roll direction angle θr is changed in order to change the preset value of the mounting angle of the vehicle's internal camera 10.

[0105] As described above, the device 100 for calibrating the mounting angle changes the mounting angle of the vehicle's in-vehicle camera 10 and repeats the conversion of the captured image to the bird's-eye view image until the predefined condition is met (steps S119 and S116 in Fig. 4) Let us assume that, for example, the following can be derived from this: Fig. The image shown in 10D is from a bird's-eye view. In the Fig. In the 10D image shown from a bird's-eye view, the side lines a and b of parking space T are parallel. Furthermore, the width wb of side line b is equal to the width wa of side line a ("wb = wa" in ). Fig. 10D). Thus, the parking space T is determined to be undistorted in shape. This result means that the mounting angle of the vehicle's internal camera 10 (the actual mounting angle of the vehicle's internal camera 10 relative to the vehicle 1) is correct, which is used in the conversion to the bird's-eye view image.

[0106] Thus, in step S120, the device 100 for calibrating the mounting angle determines the value of the mounting angle at the time of conversion to the bird's-eye view image as an updated value of the mounting angle (a correct value of the mounting angle) when a bird's-eye view image is obtained where the parallelism of the side lines a and b of the parking space T is within the predefined tolerance range and the difference in width between the side lines a and b of the parking space T is within the predefined tolerance range (step S117: Yes and step S118: Yes).

[0107] The device 100 for calibrating the mounting angle writes the determined update value of the mounting angle to the memory unit 21 of the image generation device 20, thereby updating the preset value of the mounting angle stored in the memory unit 21 (step S121 in Fig. 4) Then the one in the Fig. 3 and Fig. The process shown in step 4 for calibrating the mounting angle is complete.

[0108] Note that in the above-described process for calibrating the mounting angle, the processing of step S100 in Fig. 3 to step S120 in Fig. 4 corresponds to a mounting angle detection process for detecting a mounting angle of the vehicle's internal camera 10.

[0109] As described in detail, in the device 100 for calibrating the mounting angle (the process for calibrating the mounting angle), which includes the device 120 for capturing the mounting angle according to the present embodiment, it is confirmed that the orientation of the vehicle 1 is parallel to the side lines a and b of the parking space T (step S110: yes and step S111: yes) before the captured image is converted into a bird's-eye view image and a correct mounting angle is found by searching (before the processing of steps S116 to S119 in Fig. 4 is carried out). Thus, the device 100 for calibrating the mounting angle according to the present embodiment can easily detect a correct mounting angle of the vehicle's internal camera 10 relative to the vehicle 1. A supplementary description of this point follows.

[0110] As described above, in the process of calibrating the mounting angle, the captured image, as shown in Fig. The image shown in 10A is converted into a bird's-eye view image, and the mounting angle of the vehicle's internal camera 10 is searched for until a bird's-eye view image is obtained, as shown in Fig. 10D is shown, which has a shape that shows the parking space T without distortion.

[0111] The mounting angle thus obtained is the angle of the optical axis of the vehicle's internal camera 10 relative to the side lines a and b of the parking space T. For example, assume that, as in Fig. Figure 11 shows, by way of example, that the optical axis CL of the vehicle's internal camera 10 is inclined by an angle θ in the yaw direction relative to the side lines a and b of the parking space T. Simultaneously, the shape of the parking space T is distorted when the captured image is converted into a bird's-eye view image, assuming that the optical axis CL of the vehicle's internal camera 10 is not inclined. Therefore, in the process of calibrating the mounting angle, the angle in the yaw direction is changed so that a bird's-eye view image, including the parking space T, with an undistorted shape can be obtained. Thus, the angle θ is found by searching for it in the process of calibrating the mounting angle.

[0112] The angle θ found by searching in this way is, however, an angle at which the optical axis CL of the vehicle's internal camera 10 is inclined relative to the side lines a and b of the parking space T. Therefore, this is irrelevant to the angle at which the vehicle's internal camera 10 is mounted on the vehicle 1. More precisely, the mounting angle of the vehicle's internal camera 10 relative to the vehicle 1 cannot be calculated, even if the angle of the optical axis CL of the vehicle's internal camera 10 relative to the side lines a and b of the parking space T is known, if the angle of the vehicle 1's axis in the direction in which the vehicle 1 is oriented relative to the side lines a and b of the parking space T is unknown. Fig. 11 is the reason why vehicle 1 is shown with dashed lines, that the orientation of vehicle 1 has not been determined.

[0113] Assume that the orientation of vehicle 1 is parallel to the side lines a and b of parking space T. In this case, the orientation of vehicle 1 and the orientation of the side lines a and b of parking space T are the same. Therefore, the angle of the optical axis CL of the vehicle's internal camera 10 relative to the side lines a and b of parking space T can be directly interpreted as the angle of vehicle 1 (the mounting angle of the vehicle's internal camera 10 relative to vehicle 1). Therefore, the mounting angle of the vehicle's internal camera 10 relative to vehicle 1 can be easily determined.

[0114] For this reason, in the above-described process for calibrating the mounting angle, the captured image is converted into a bird's-eye view image after it has been confirmed that the orientation of vehicle 1 is parallel to the side lines a and b of the parking space T (S111 in Fig. 4: yes). Subsequently, a correct mounting angle of the vehicle's internal camera 10 relative to the vehicle 1 is found by searching (steps S116 to S119).

[0115] Furthermore, the above-described process for calibrating the mounting angle is based on the assumption that the mounting angle of the vehicle's internal camera 10 can be changed and that the captured image can be converted into a bird's-eye view image based on the mounting angle (different mounting angles) after the change. The reason why this process (conversion to a bird's-eye view) is possible is described below.

[0116] First, various coordinate systems are described that are used to convert the captured image into a bird's-eye view image.

[0117] Fig. Figure 12 illustrates various coordinate systems used in the conversion to a bird's-eye view. Coordinate systems used in the conversion to a bird's-eye view according to the present embodiment include, for example, an orthogonal coordinate system for the vehicle 1 and an orthogonal coordinate system for the vehicle's in-vehicle camera 10. The orthogonal coordinate system for the vehicle 1 is expressed using a coordinate axis Xo (hereinafter referred to as the "Xo-axis"), a coordinate axis Yo (hereinafter referred to as the "Yo-axis"), and a coordinate axis Zo (hereinafter referred to as the "Zo-axis"). The Xo-axis is a coordinate axis of the vehicle 1 that is perpendicular to its forward and backward (or longitudinal) direction and extends in the horizontal direction. The Yo-axis is a coordinate axis of the vehicle 1 that extends in the longitudinal direction.The zo-axis is a coordinate axis of vehicle 1, which lies at a right angle to the longitudinal direction and extends vertically, or upwards. Hereinafter, coordinates in the orthogonal coordinate system based on vehicle 1 are referred to as "vehicle-based coordinates," and a coordinate value of the vehicle-based coordinates is expressed as (xo, yo, zo).

[0118] The orthogonal coordinate system based on the in-vehicle camera 10 is expressed using a coordinate axis CX (hereinafter referred to as the "CX axis"), a coordinate axis CY (hereinafter referred to as the "CY axis"), and a coordinate axis CZ (hereinafter referred to as the "CZ axis"). The CY axis is a coordinate axis extending along the optical axis of the in-vehicle camera 10. The CX axis is a coordinate axis perpendicular to the CY axis and extending horizontally. The CZ axis is a coordinate axis perpendicular to both the CY axis and the CX axis and extending upwards. Hereinafter, coordinates in the orthogonal coordinate system based on the in-vehicle camera 10 are referred to as "camera-based coordinates," and a coordinate value of the camera-based coordinates is expressed as (cx, cy, cz).

[0119] The relationship between the vehicle-based coordinates (xo, yo, zo) and the camera-based coordinates (cx, cy, cz) changes according to the mounting position of the vehicle-internal camera 10 relative to the vehicle 1. Therefore, in the present embodiment, the following new orthogonal coordinate system is introduced to facilitate the conversion to a bird's-eye view. More precisely, the new orthogonal coordinate system is obtained by shifting the orthogonal coordinate system based on the vehicle 1 such that the origin O of the orthogonal coordinate system based on the vehicle 1 corresponds to the origin CO of the orthogonal coordinate system based on the vehicle-internal camera 10.The coordinate axes of this new orthogonal coordinate system comprise: a coordinate axis parallel to the Xo-axis as the X-axis; a coordinate axis parallel to the Yo-axis as the Y-axis; and a coordinate axis parallel to the Zo-axis as the Z-axis. Hereinafter, coordinates in this new orthogonal coordinate system are referred to as "analytical vehicle-based coordinates," and a coordinate value of the analytical vehicle-based coordinates is expressed as (x, y, z).

[0120] With such analytical vehicle-based coordinates, the rotation of the vehicle's internal camera 10 in the pitch direction corresponds to the rotation about the X-axis; the rotation of the vehicle's internal camera 10 in the roll direction corresponds to the rotation about the Y-axis; and the rotation of the vehicle's internal camera 10 in the yaw direction corresponds to the rotation about the Z-axis.

[0121] Furthermore, the coordinate value (xo, yo, zo) of the vehicle-based coordinates and the coordinate value (x, y, z) of the analytical vehicle-based coordinates are determined according to the [document / reference]. Fig. Equation (1) shown in Figure 13 is easily convertible. Note that [tx, ty, tz] in Equation (1) specifies the xo-coordinate, the yo-coordinate and the zo-coordinate of the vehicle-based coordinates where the origin CO of the camera-based coordinates is located.

[0122] In this way, the coordinate values ​​(xo, yo, zo) of the vehicle-based coordinates and the coordinate values ​​(x, y, z) of the analytical vehicle-based coordinates can be converted into each other. The analytical vehicle-based coordinate values ​​(x, y, z) are primarily used for analysis.

[0123] The coordinate values ​​(x, y, z) of the analytical vehicle-based coordinates and the coordinate values ​​(cx, cy, cz) of the camera-based coordinates are determined according to the [reference to be added]. Fig. Equation (2) shown in Figure 14 is linked. Note that in equation (2) the following holds: [P] represents a rotation matrix for rotating the analytical vehicle-based coordinates in the pitch direction; [R] represents a rotation matrix for rotating the analytical vehicle-based coordinates in the roll direction; and [Y] represents a rotation matrix for rotating the analytical vehicle-based coordinates in the yaw direction. The angle θp in the rotation matrix [P] is the rotation angle in the pitch direction of the camera-based coordinates relative to the analytical vehicle-based coordinates. The angle θr in the rotation matrix [R] is the rotation angle of the camera-based coordinates with respect to the analytical vehicle-based coordinates in the roll direction. The angle θy in the rotation matrix [Y] is the rotation angle of the camera-based coordinates with respect to the analytical vehicle-based coordinates in the yaw direction.

[0124] The coordinate value (x, y, z) of the analytical vehicle-based coordinates for the parking space T drawn on the ground is converted into the coordinate value (cx, cy, cz) of the camera-based coordinates according to equation (2) as in Fig. 14 shown converted.

[0125] The coordinate value (cx, cy, cz) of parking space T, obtained from the conversion to camera-based coordinates, can be converted into a coordinate value of the captured image. Hereinafter, the coordinate value of the captured image is referred to as the "image plane coordinates," and a coordinate value of the image plane coordinates is expressed as (u, v).

[0126] Fig. Figure 15 illustrates a procedure for converting the coordinate value (cx, cy, cz) of parking space T, which results from the conversion to camera-based coordinates, into the coordinate value (u, v) of the image plane coordinates. As in Fig. As illustrated in Figure 15, in the present conversion process a plane perpendicular to the optical axis (the coordinate axis CY) of the vehicle's internal camera 10 (hereinafter referred to as the "image plane") is set at a suitable distance SL from the origin CO of the camera-based coordinates. Thus, the image captured by the vehicle's internal camera 10 can be considered an image obtained by projecting a recording object in the camera-based coordinates onto the image plane.

[0127] Thus, in the present conversion process, a coordinate space is assumed that includes: a coordinate axis U parallel to the coordinate axis CX of the camera-based coordinates, and a coordinate axis V parallel to the coordinate axis CZ of the camera-based coordinates, with the origin located at the intersection between the image plane and the coordinate axis CY of the camera-based coordinates. In this case, the coordinate value (cx, cy, cz) of the parking space T drawn on the ground can be determined according to the Fig. Equation (3) shown in 15 can be converted into the coordinate value (u, v) on the image plane.

[0128] Fig. Figure 16 shows, by way of example, the conversion relationship of the coordinate values ​​described above and the conversion relationship of the coordinate values ​​for conversion to a bird's-eye view. In particular, the coordinate value (xo, yo, zo) of the vehicle-based coordinates and the coordinate value (x, y, z) of the analytical vehicle-based coordinates are shown according to the Fig. Equation (1) shown in 13 can be transformed into each other. The coordinate value (x, y, z) of the analytical vehicle-based coordinates can be determined according to the equation shown in Fig. Equation (2) shown in Figure 14 can be converted into the coordinate value (cx, cy, cz) of the camera-based coordinates. The coordinate value (cx, cy, cz) of the camera-based coordinates can be determined according to the equation shown in Figure 14. Fig. Equation (3) shown in Figure 15 can be converted into the coordinate value (u, v) of the image plane coordinates. The coordinate value (uv) of the image plane coordinates corresponds to the coordinate value of the captured image.

[0129] Thus, the coordinate values ​​(x, y, z) of the analytical vehicle-based coordinates and the coordinate values ​​(u, v) of the image plane coordinates have a one-to-one correspondence. Therefore, in the present conversion process, the coordinate values ​​(u, v) of the image plane coordinates can be converted as indicated by the dashed arrow. Fig. Figure 16 shows how the coordinate values ​​(x, y, z) of the analytical vehicle-based coordinates can be reversed. The bird's-eye view is an image viewed from a predefined height (above) along the Z-axis of the analytical vehicle-based coordinates. Thus, the bird's-eye view is obtained by deleting the Z-axis component from the coordinate values ​​(x, y, z) of the analytical vehicle-based coordinates.

[0130] In light of the above, it suffices to perform a series of conversion operations, such as the bird's-eye view conversion, in which the coordinate value (u, v) of the image plane coordinates is inversely converted into the coordinate value (x, y, z) of the analytical vehicle-based coordinates, and the Z-axis component is deleted from the resulting coordinate value (x, y, z) of the analytical vehicle-based coordinates. The bird's-eye view conversion can be performed, for example, by a bird's-eye view conversion module with modules that carry out these operations. Hereinafter, the coordinate value (x, y) obtained by deleting the Z-axis component from the coordinate value (x, y, z) of the analytical vehicle-based coordinates is expressed as the coordinate value (x, y) of bird's-eye view coordinates, which constitute a coordinate system after the bird's-eye view conversion.

[0131] As in equation (2) in Fig. As shown in Figure 14, the conversion from the coordinate value (x, y, z) of the analytical vehicle-based coordinates to the coordinate value (cx, cy, cz) of the camera-based coordinate depends on the angle θp of the vehicle-internal camera 10 in the pitch direction, the angle θr of the vehicle-internal camera 10 in the roll direction, and the angle θy of the vehicle-internal camera 10 in the yaw direction. Similarly, the conversion from the image plane coordinate (u, v) of the captured image to the coordinate value (x, y) of the coordinates from the bird's-eye view depends on the angle θp of the vehicle-internal camera 10 in the pitch direction, the angle θr of the vehicle-internal camera 10 in the roll direction, and the angle θy of the vehicle-internal camera 10 in the yaw direction.

[0132] As in Fig. As shown in Figure 17 as an example, the module for converting to a bird's-eye view is therefore designed to be able to adjust the angle θp of the vehicle's internal camera 10 in the pitch direction, the angle θr of the vehicle's internal camera 10 in the roll direction and the angle θy of the vehicle's internal camera 10 in the yaw direction.

[0133] During processing in step S116 in the Fig. In the process shown in section 4 for adjusting the mounting angle, the recorded image is converted into a bird's-eye view image using such a module.

[0134] The above describes the case in which the parking space T has two side lines a and b and a transverse line c perpendicular to these side lines a and b (see Fig. 5) The transverse line c of parking space T is in the Fig. 3 and Fig. The operation shown in Figure 4 is not used, although the processing is carried out using the side lines a and b of the parking space T. Thus, in the operation to calibrate the mounting angle according to the present embodiment, the mounting angle of the vehicle's in-vehicle camera 10 can be detected and calibrated to a correct mounting angle, even if the parking space T does not have the transverse line c.

[0135] If the parking space T as in Fig. If, as shown in example 18A, the cross line c is not present, a recorded image like the one in Fig. Figure 18B illustrates this. The device 100 for calibrating the mounting angle according to the present embodiment performs the above-described process for calibrating the mounting angle even in this case, if the two side lines a and b can be detected. Thus, the mounting angle of the vehicle's internal camera 10 can be appropriately calibrated. B. Second embodiment

[0136] In the first embodiment, a setup was described in which the image received by the image generation device 20 is converted into a bird's-eye view image and the side lines a and b of the parking space T are captured from the obtained bird's-eye view image.

[0137] As described above, in the process for calibrating the mounting angle according to the first embodiment, the straight center lines and the straight boundary lines of the side lines a and b, which are captured from the numerous recorded images, are accumulated before the recorded images are converted into bird's-eye view images. This confirms in the process of calibrating the mounting angle that the orientation of vehicle 1 is parallel to the side lines a and b of the parking space T.

[0138] In the present embodiment, instead of converting the image received by the image generation device 20 into a bird's-eye view, an image for conversion into a bird's-eye view is generated using the accumulated straight center lines and straight boundary lines. The present embodiment is described below, highlighting the differences from the first embodiment. B-1. Internal structure of the device 200 for calibrating the mounting angle

[0139] Fig. Figure 19 shows an exemplary block diagram illustrating the internal structure of a device 200 for calibrating the mounting angle according to the present embodiment. The device 200 for calibrating the mounting angle according to the present embodiment differs from the device 100 for calibrating the mounting angle according to the first embodiment in that it includes a unit 201 for generating a representative captured image. Thus, the device 200 for calibrating the mounting angle differs in that the unit 109 for converting to a bird's-eye view receives an image undergoing conversion to a bird's-eye view from the unit 201 for generating a representative captured image, instead of the unit 101 for obtaining a captured image.The internal structure of the device 200 for calibrating the mounting angle according to the present embodiment is briefly described below, with a focus on these differences.

[0140] As in Fig. As shown by way of example in Figure 19, the device 200 for calibrating the mounting angle according to the present embodiment, as in the first embodiment, comprises a unit 101 for obtaining captured images, a unit 102 for detecting straight center lines, a unit 103 for detecting straight boundary lines, a storage unit 104 for straight lines, a parallel determination unit 105, a vehicle monitoring unit 106, and an alarm output unit 107. Furthermore, the device 200 for calibrating the mounting angle comprises: a unit 108 for obtaining a predefined value; a unit 109 for converting to a bird's-eye view; a mounting angle search unit 110, which includes a bird's-eye view image evaluation unit 111 and a unit 112 for changing a predefined value; a mounting angle determination unit 113; and a mounting angle update unit 114.In addition, the unit 201 for generating a representative recorded image is included in the device 200 for calibrating the mounting angle.

[0141] Note that the section of device 200 for calibrating the mounting angle excludes the mounting angle update unit 114 (the section indicated by the dashed line in Fig. 19 enclosed), corresponds to a device 220 for detecting the mounting angle according to the present embodiment.

[0142] The unit 101 for receiving a captured image, the detection unit 102 for straight center lines, the unit 103 for detecting straight boundary lines, the storage unit 104 for detected straight lines, the parallel determination unit 105, the vehicle monitoring unit 106, and the alarm output unit 107 are essentially the same as in the first embodiment. Therefore, their description is omitted here.

[0143] Unit 201, used to generate a representative captured image, receives the determination result from parallel determination unit 105. If the determination result shows that the orientation of vehicle 1 is parallel to the two straight segments of the target marking, the representative image generation unit 201 reads the capture results of the straight center lines and the straight boundary lines stored in storage unit 104 for captured straight lines. Thus, unit 201 generates a representative captured image. The representative captured image generated at this time comprises two straight center lines and four straight boundary lines, representing the two straight segments of the target marking, which are captured from the numerous captured images.

[0144] When it receives the determination result from the parallel determination unit 105, indicating that the orientation of the vehicle 1 is parallel to the two straight parts of the target marking, the unit 108 receives the predefined value of the mounting angle of the vehicle's internal camera 10 from the image generation device 20. Subsequently, the unit 108 outputs the received predefined value to the unit 109 for conversion into a bird's-eye view.

[0145] When it receives the mounting angle of the vehicle's internal camera 10 from unit 108 to obtain a predetermined value, unit 109 converts the representative captured image into a bird's-eye view, which unit 201 has generated for converting to a bird's-eye view.

[0146] The bird's-eye view image evaluation unit 111 analyzes a bird's-eye view image obtained by converting the representative captured image into a bird's-eye view. Based on the analysis result, the bird's-eye view image evaluation unit 111 determines whether the two straight center lines and the four straight boundary lines contained in the representative captured image have been correctly converted into straight lines representing the two straight portions of the target marker. Consequently, if it is determined that the lines have not been converted correctly, the unit 112 modifies the predefined value of the mounting angle of the vehicle's in-vehicle camera 10, which it receives from the storage unit 21 of the image generation device 20.

[0147] Thus, unit 109 performs the bird's-eye view conversion again. Specifically, using the mounting angle after the change, unit 109 converts the bird's-eye view back into the representative captured image, thereby generating a new bird's-eye view image.

[0148] Thus, in the present embodiment, the bird's-eye view image evaluation unit 111 and the unit 112 for changing the predefined value of the mounting angle search unit 110 also correspond.

[0149] When it is determined that the two straight center lines and the four straight boundary lines contained in the representative captured image have been correctly converted, the bird's-eye view image evaluation unit 111 outputs the value of the mounting angle to the mounting angle determination unit 113, which is used in the conversion to the bird's-eye view at that time. Consequently, the mounting angle determination unit 113 and the mounting angle update unit 114 perform essentially the same processes as the mounting angle determination unit 113 and the mounting angle update unit 114 according to the first embodiment.In particular, the mounting angle determination unit 113 determines as a correct value of the mounting angle (an actual value of the mounting angle) the value of the mounting angle received by the bird's-eye view image evaluation unit 111 (the value of the mounting angle used for the conversion to the bird's-eye view, in which the lines are correctly converted). Furthermore, the mounting angle update unit 114 receives the correct mounting angle of the vehicle's in-vehicle camera 10, determined by the mounting angle determination unit 113, and writes the received mounting angle to the storage unit 21 in the image generation device 20. The mounting angle update unit 114 updates the value of the mounting angle stored in the storage unit 21 in the image generation device 20 to the correct value of the mounting angle.As described above, the device 200 performs the operations of the aforementioned units for calibrating the mounting angle according to the present embodiment. This completes the calibration of the mounting angle of the vehicle's internal camera 10, which can be used by the image generation device 20 to generate an image (an image to be displayed on the monitor 6). B-2. Mounting angle calibration procedure:

[0150] Fig. Figure 20 illustrates a flowchart showing the mounting angle calibration process (the second half) performed by the device 200 to calibrate the mounting angle according to the present embodiment. The flowchart in the first half of the mounting angle calibration process according to the present embodiment is essentially the same as the flowchart of the first half of the mounting angle calibration process shown in Figure 20. Fig. Figure 3 is shown. Therefore, its illustration is omitted. Note that the present process is performed by the CPU contained in the device 200 for calibrating the mounting angle. In particular, the CPU in the device 200 for calibrating the mounting angle reads a program for the mounting angle calibration process from a predefined memory area of ​​the memory (a non-volatile, computer-readable physical storage medium) or the like, and executes the program. In the following description, the following applies: Fig. 3 for the first half of the mounting angle calibration process, and for the second half of the mounting angle calibration process we refer to Fig. 20. The mounting angle calibration process according to the present embodiment is briefly described below.

[0151] In the mounting angle calibration process according to the present embodiment, as in the first embodiment, the device 200 for calibrating the mounting angle first issues an instruction that the vehicle 1 should be aligned parallel to the side lines a and b of the parking space T and that the steering wheel 2 should be in the steering position for straight-ahead driving. Subsequently, the device 200 for calibrating the mounting angle issues an instruction to move the vehicle 1 straight ahead by a predetermined distance L (according to steps S100 and S101 in Fig. 3) The device 200 for calibrating the mounting angle then receives a recorded image of the parking space T from the image generation device 20. The device 200 for calibrating the mounting angle detects from the received recorded image the slopes of the straight center lines and the straight boundary lines, the intersection positions of the straight center lines and the intersection positions of the straight boundary lines of the side lines a and b of the parking space T and stores the detected slopes, center line intersection positions and boundary line intersection positions in the memory (according to steps S102 and S103).

[0152] The device 200 for calibrating the mounting angle determines whether the steering wheel 2 is held in the straight-ahead position. The device 200 for calibrating the mounting angle determines whether the acquisition results of a predetermined number of captured images have been stored. The device 200 for calibrating the mounting angle determines whether the vehicle 1's travel distance has reached the predetermined distance L (according to steps S104, S107, and S108). Consequently, the device 200 for calibrating the mounting angle discards the straight centerline and straight boundary line acquisition results stored in memory if it determines that the steering wheel 2 is not held in the straight-ahead position.The device 200 then issues a warning and an instruction to hold the steering wheel 2 in the straight-ahead position (according to steps S105 and S106) for calibrating the mounting angle. The processing then restarts. Furthermore, when it is determined that the distance traveled by vehicle 1 has reached the predetermined distance L, the device 200 issues an instruction to reverse the direction of travel of vehicle 1 and move it straight ahead for a predetermined distance L (according to step S109). Afterward, the device 200 receives a new captured image and stores (according to steps S102 and S103) the straight center lines and straight boundary lines captured from the image in its memory.

[0153] In the device 200 for calibrating the mounting angle, the acquisition results of the straight center lines and straight boundary lines are accumulated by repeating such processes. Finally, the number of captured images from which the acquisition results of the straight center lines and straight boundary lines have been accumulated reaches the predetermined number (according to S107: yes). Thus, in the device 200 for calibrating the mounting angle, the accumulation of the straight center lines and straight boundary lines acquired from the captured images is completed.

[0154] When the accumulation of straight center lines and straight boundary lines, acquired from the predetermined number of captured images, is completed during the process of calibrating the mounting angle according to the present embodiment, the following process is carried out. In particular, the device 200 determines the mounting angle calibration process as described in Fig. 20. By way of example, it is shown whether the scatter of the slopes and the intersection positions of the accumulated straight center lines lies within a predefined tolerance range (step S150). This determination is the same as in the first embodiment (the processing in step S110 in Fig. 4) In particular, as above, based on Fig. In step S150, as described in section 8, the slopes kbc of the straight centerline bc are read, and their standard deviation σkbc is calculated. Furthermore, the slopes kac of the straight centerline ac are read, and their standard deviation σkac is calculated. Similarly, in step S150, the intersection positions pbc of the straight centerline bc are read, and their standard deviation σpbc is calculated. Furthermore, the intersection positions pac of the straight centerline ac are read, and their standard deviation σpac is calculated.

[0155] Furthermore, in processing step S150, it is determined whether the calculated dispersion σkbc, σkac, σpbc and σpac lies within the predefined tolerance range.

[0156] If it is determined that the scatter of the bevels and the intersection positions of the straight center lines is within the predefined tolerance range (step S150 in 20: yes), then the device 200 for calibrating the mounting angle consequently determines whether the scatter of the bevels and the boundary line intersection positions of the straight boundary lines are within a predefined tolerance range (step S151). This determination is also the same as in the first embodiment (the processing in step S111 in Fig. 4) In particular, as above, based on Fig. As described in section 8, in processing step S151, the slopes kbo, kbi, kao, and kai of the straight boundary lines bo, bi, ao, and ai are read, and their respective variances σkbo, σkbi, σkao, and σkai are calculated. Similarly, in processing step S151, the boundary line intersection positions pbo, pbi, pao, and pai of the straight boundary lines bo, bi, ao, and ai are read, and their corresponding variances σpbo, σpbi, σpao, and σpai are calculated.

[0157] Furthermore, in processing step S151, it is determined whether the calculated dispersion σkbo, σkbi, σka, σkai, σpbo, σpbi, σpao and σpai lies within the predefined tolerance range.

[0158] If it is determined that the dispersion of the slopes and / or the intersection positions of the straight centerlines are not within the predefined tolerance range (step S150 in Fig. 20: No), consequently the device 200 for calibrating the mounting angle determines that the orientation of vehicle 1 is not parallel to the side lines a and b of the parking space T. If it is determined that the scatter of the slopes and / or the boundary line intersection positions of the straight boundary lines are not within the predefined tolerance range (step S151: no), consequently the device 200 for calibrating the mounting angle determines that the orientation of vehicle 1 is not parallel to the side lines a and b of the parking space T.

[0159] As described above, the mounting angle calibration device 200 outputs information indicating that the orientation of vehicle 1 is not parallel to the side lines a and b of parking space T (step S152) if the "no" judgment is made in the sequence in step S150 or step S151. The mounting angle calibration device 200 discards the detection results of the straight center lines and the straight boundary lines (the data exemplified in Fig. 8 are shown), which were accumulated in the memory (step S153). Subsequently, the device 200 for calibrating the mounting angle restarts a process of the above-described procedure by returning to the start of the processing (according to step S100 of the Fig. 3) returns.

[0160] If, on the other hand, it is determined that any dispersion of the bevels and the intersection positions of the straight center lines lies within the predefined tolerance range (step S150: Yes) and any dispersion of the bevels and the boundary line intersection positions of the straight boundary lines lies within the predefined tolerance range (step S151: Yes), the mounting angle calibration device 200 determines that the straight center lines in the captured images are the same straight line (their positions in the images are identical). Similarly, the straight boundary lines in the captured images are determined to be the same straight line.In particular, if both the slope and the intersection position of the straight center line are within the predetermined tolerance range, and both the slopes and the intersection positions of the straight boundary line are within the predetermined tolerance range, the device 200 for calibrating the mounting angle determines that the positional relationship between the vehicle 1 and the parking space T is as shown in the diagram. Fig. The vehicle is in the state shown in Figure 9A. Therefore, in this case, the orientation of vehicle 1 can be determined to be parallel to the side lines a and b of parking space T.

[0161] If, in the mounting angle calibration process according to the first embodiment, the orientation of the vehicle 1 is determined to be parallel to the side lines a and b of the parking space T (step S111 in Fig. 4: yes), a captured image is obtained for conversion into a bird's-eye view (step S114).

[0162] In contrast, in the process for calibrating the mounting angle according to the present embodiment, a representative recorded image is generated for conversion to a bird's-eye view (step S154 in Fig. 20). The representative image taken represents the predetermined number of images taken and is used to verify that the orientation of vehicle 1 is parallel to the side lines a and b of the parking space T.

[0163] The representative captured image is described in detail. It is assumed that it has been confirmed that the orientation of vehicle 1 is parallel to the side lines a and b of parking space T. This can be interpreted to mean that the straight center lines and the straight boundary lines overlap between the captured images (their positions match), which are taken from a predetermined number of captured images (the one in Fig. 9A (shown condition).

[0164] In such a state, as occurs, for example, in the Fig. 21A and Fig. As illustrated in Figure 21B, a representative straight centerline tbc, representing the straight centerlines bc captured from the predetermined number of captured images, and a representative straight centerline tac, representing the straight centerlines ac captured from the predetermined number of captured images, can be determined. For example, the representative straight centerline tbc is determined as follows. Specifically, the mean or median value of each of the obliques kbc and the intersection positions pbc of the multiple straight centerlines bc captured from the predetermined number of captured images is calculated. Each straight line, its oblique, and its intersection position are the respective mean or median value. The calculated straight line is designated as the representative straight centerline tbc.

[0165] Similarly, a representative straight boundary line tbo representing the numerous straight boundary lines bo, a representative straight boundary line tbi representing the numerous straight boundary lines bi, a representative straight boundary line tao representing the numerous straight boundary lines ao, and a representative straight boundary line tai representing the numerous straight boundary lines ai can be determined.

[0166] The Fig. 21A and Fig. Figure 21B shows, by way of example, the representative straight center lines tbc and tac and representative straight boundary lines tbo, tbi, tao, and tai determined in this manner. The representative straight center lines tbc and tac and the representative straight boundary lines tbo, tbi, tao, and tai represent parts of the side lines a and b of the parking space T, which appear in the predefined number of captured images. Thus, the representative captured image, which is processed in step S154 in Fig. 20 is generated, an image like the one in Fig. 21A shown.

[0167] The representative image described above includes the two representative straight center lines tbc and tac and the four representative straight boundary lines tbo, tbi, tao and tai.

[0168] However, the representative captured image may, for example, only include the four representative straight boundary lines tbo, tbi, tao, and tai. In this case, it is sufficient for the device 200 to accumulate the slopes and boundary line intersection positions of the straight boundary lines for calibrating the mounting angle; the device 200 for calibrating the mounting angle does not need to accumulate the slopes and intersection positions of the straight center lines.

[0169] The device 200 for calibrating the mounting angle receives the predefined value of the mounting angle of the vehicle's internal camera 10, which is stored in the memory unit 21 in the image generation device 20 (step S155 in Fig. 20). The device 200 for calibrating the mounting angle converts the generated representative image into a bird's-eye view image (step S156). As in Fig. As shown in Figure 21A, the representative captured image comprises the two representative straight center lines tbc and tac and the four representative straight boundary lines tbo, tbi, tao, and tai. Thus, after conversion to a bird's-eye view, the representative captured image is a bird's-eye view image, as shown in Figure 21A. Fig. 21B shown.

[0170] The device 200 for calibrating the mounting angle analyzes the obtained bird's-eye view image, detects the side lines a and b of the parking space T in the bird's-eye view image, and determines whether the parallelism of the detected side lines a and b lies within a predefined tolerance range (step S157). For example, in the processing in step S157, the parallelism of the two representative straight center lines tbc and tac in the image is determined. Fig. The image shown in 21B is calculated from a bird's-eye view. In step S157 of the processing, a determination is made as to whether the calculated parallelism lies within the predefined tolerance range.

[0171] For example, suppose that the representative captured image does not include the two representative straight center lines tbc and tac. Therefore, if the bird's-eye view image does not include the two representative straight center lines tbc and tac, the following determination can be made. More precisely, it can be determined whether the parallelism of the four representative straight boundary lines tbo, tbi, tao, and tai in the bird's-eye view image lies within the predefined tolerance range.

[0172] Consequently, the device 200 for calibrating the mounting angle determines whether the difference in width between the side lines a and b of the parking space T is within a predefined tolerance range (step S158) if it is determined that the parallelism of the side lines a and b of the parking space T is within the predefined tolerance range (step S157 in Fig. 20: yes). In the processing in step S158, the distance between the two representative straight boundary lines tbo and tbi and the distance between the two representative straight boundary lines tao and tai in the bird's-eye view image are calculated, for example in Fig. Figure 21B is shown. In the processing step S158, a determination is made as to whether the difference between the two calculated distances lies within a predefined tolerance range.

[0173] If it is determined that the parallelism of the side lines a and b of the parking space T is not within the predefined tolerance range (step S157 in Fig. 20: No), the device 200 for calibrating the mounting angle determines that the mounting angle of the vehicle's in-vehicle camera 10, used in the bird's-eye view conversion, is incorrect. Even if it determines that the difference in width between the side lines a and b of the parking space T is not within the predefined tolerance range (step S158: no), the device 200 for calibrating the mounting angle determines that the mounting angle of the vehicle's in-vehicle camera 10, used in the bird's-eye view conversion, is incorrect.

[0174] The device 200 for calibrating the mounting angle changes the predefined value of the mounting angle (step S159) and, using the modified mounting angle, reconverts the representative captured image into a bird's-eye view image (step S156). Subsequently, by analyzing the resulting bird's-eye view image, the device 200 determines whether the parallelism between the side lines a and b of the parking space T in the bird's-eye view image is within the predefined tolerance range and whether the difference in width between the side lines a and b of the parking space T is within the predefined tolerance range (steps S157 and S158). The device 200 for calibrating the mounting angle repeats the aforementioned determination until the condition is met.Thus, a correct mounting angle of the vehicle's internal camera 10 relative to the vehicle 1 is found by searching.

[0175] If it is determined that the parallelism of the side lines a and b of the parking space T in the bird's-eye view image is within the predefined tolerance range, and the difference in width between the side lines a and b of the parking space T is within the predefined tolerance range (step S157: Yes and step S158: Yes in Fig. 20), the device 200 for calibrating the mounting angle determines as the updated value of the mounting angle (a correct value of the mounting angle) the value of the mounting angle that is used in the conversion to the bird's-eye view at that time (step S160).

[0176] The device 200 for calibrating the mounting angle writes the determined update value of the mounting angle to the storage unit 21 in the image generation device 20, thereby updating the preset value of the mounting angle stored in the storage unit 21 (step S161). Then the process for calibrating the mounting angle according to the present embodiment is completed.

[0177] It should be noted that the processing of step S100 is also included in the above-described process for calibrating the mounting angle. Fig. 3 to step S160 in Fig. 20 corresponds to the mounting angle detection process for detecting a mounting angle of the vehicle's internal camera 10.

[0178] As described in detail above, the mounting angle calibration device 200 (the mounting angle calibration process), which includes the mounting angle detection device 220 according to the present embodiment, converts the representative detected image with two representative straight center lines and four representative straight boundary lines into a bird's-eye view image. Therefore, the mounting angle calibration device 200 can directly calculate the representative straight center lines and the representative straight boundary lines from the bird's-eye view image.Thus, in the device 200 for calibrating the mounting angle according to the present embodiment, the parking space T is captured in the bird's-eye view image, and even if the side lines a and b of the parking space T have not been captured, it can be easily determined whether the two side lines a and b have been correctly converted into a bird's-eye view image.

[0179] Furthermore, the device 200 for calibrating the mounting angle converts the representative captured image, which represents the predetermined number of captured images, into a bird's-eye view image. Therefore, according to the present embodiment, the device 200 for calibrating the mounting angle is not affected by noise contained in the captured image. Thus, a correct mounting angle of the vehicle's internal camera 10 relative to the vehicle 1 can be reliably determined. C. Variation:

[0180] Each of the embodiments described above can be modified as follows. As above, based on Fig. As described in 10, there is an offset in the positions of the endpoints of the two sidelines a and b (see Fig. 10C), if the widths of the two sidelines a and b are not equal. Conversely, if the widths of the two sidelines a and b are equal, there is no offset in the positions of the endpoints of the two sidelines a and b (see Fig. 10D).

[0181] Thus, in the device for calibrating the mounting angle according to the present modification, the mounting angle of the vehicle's internal camera 10 can be found by searching for the positions of the endpoints of the two side lines a and b instead of the difference in width between the two side lines a and b.

[0182] For example, let's assume that the recorded image is the one in Fig. The image shown in 22A is shown. Furthermore, it is assumed that converting this captured image into a bird's-eye view results in the bird's-eye view image shown in Fig. Figure 22B shows that in this case, the straight center line bc of side line b and the straight center line ac of side line a are not parallel in the bird's-eye view image. Therefore, it is possible to determine that the mounting angle used in the bird's-eye view conversion is incorrect.

[0183] It is therefore assumed that a continuous search for the mounting angle of the vehicle's in-vehicle camera 10 leads to the bird's-eye view image shown in 22C. In this case, the straight center line bc of side line b and the straight center line ac of side line a are parallel in the bird's-eye view image. However, there is an offset between the position of an endpoint Tpb of side line b and the position of an endpoint Tpa of side line a. Therefore, it is possible to determine that the mounting angle used in the bird's-eye view conversion is incorrect.

[0184] Furthermore, it is assumed that a continuous search for the mounting angle of the vehicle's internal camera 10 leads to the result in Fig. The image shown in Figure 22D is from a bird's-eye view. In this bird's-eye view image, the straight midline bc of sideline b and the straight midline ac of sideline a are parallel, and there is no offset between the position of the endpoint Tpb of sideline b and the position of the endpoint Tpa of sideline a. In this case, it is possible to determine that the mounting angle used in the bird's-eye view conversion is correct.

[0185] Thus, the device for calibrating the mounting angle is used to search for the mounting angle of the vehicle's internal camera 10 according to the given variation until a bird's-eye view image is obtained in which the side lines a and b of the parking space T are parallel and which shows no offset between the positions of the endpoints of the side lines a and b of the parking space T. In this way, a correct mounting angle of the vehicle's internal camera 10 on the vehicle 1 can be determined.

[0186] In the device described above for calibrating the mounting angle according to the present modification, the mounting angle of the vehicle's internal camera 10 is determined based on the parallelism of the side lines a and b of the parking space T and the positions of the endpoints of the side lines a and b of the parking space T. Thus, in the mounting angle calibration process as described above, the following is used: Fig. 18 describes how the mounting angle of the vehicle's internal camera 10 is detected even if the parking space T does not have the transverse line c. This allows the mounting angle to be calibrated to a correct mounting angle.

[0187] Although the embodiments and their variations have been described above, the technology of the present disclosure is not limited to the content described above. The technology of the present disclosure can be applied in various forms without deviating from its teachings. [List of reference symbols] 1 vehicle 10 ... In-vehicle camera 20 ... Image generating device 100 ... Device for calibrating the mounting angle 101 ... Unit for preserving a captured image 102 ... Unit for capturing a straight center line 103 ... Unit for detecting a straight boundary line 104 ... Storage unit for captured straight lines 105 ... Parallel determination unit 106 ... Vehicle monitoring unit 107 ... Alarm output unit 108 ... unit for maintaining a predetermined value 109 ... Unit for conversion to bird's-eye view 110 ... Mounting angle determination unit 111 ... Bird's-eye view image evaluation unit 112 ... Unit for changing the pre-defined value 113 ... Mounting angle search unit 114 ... Mounting angle update unit 120 ... Device for detecting the mounting angle 200 ... Device for calibrating the mounting angle 201 ... Unit for generating a representative recorded image 220 ... Device for detecting the mounting angle

Claims

[1] Device (120, 220) for detecting a mounting angle, which detects a mounting angle in which an in-vehicle camera (10) for detecting an image of the ground around a vehicle is mounted on the vehicle, wherein the device for detecting a mounting angle comprises: a unit (101) for obtaining a captured image, which obtains a captured image of the ground on which a target marker is marked with two mutually parallel straight lines of equal width; a unit (109) for converting to a bird's-eye view which receives a pre-defined value of the mounting angle of the vehicle's in-vehicle camera (10) relative to the vehicle and converts the captured image into a bird's-eye view image assuming that the vehicle's in-vehicle camera (10) is mounted according to the pre-defined value, wherein the bird's-eye view image is an image in which the ground appearing in the captured image appears as if it were seen from above; a mounting angle search unit (110) which, by changing the predefined value of the mounting angle used in the conversion to a bird's-eye view, searches for the mounting angle in which both the parallelism of the two straight parts of the target marker in the bird's-eye view image and the difference in width between the two straight parts of the target marker in the bird's-eye view image are within predefined tolerance ranges; and a mounting angle determination unit (113) which determines the mounting angle found by the search as the mounting angle of the vehicle-internal camera (10) relative to the vehicle. [2] Device (120, 220) for detecting a mounting angle according to claim 1, wherein the vehicle's internal camera (10) is mounted at the front or rear of the vehicle, The unit (101) receives numerous captured images to obtain a recorded image; these images are captured from different positions while the vehicle is moved in a straight line along one direction of the two straight parts of the target marker. The device (120, 220) for detecting a mounting angle further comprises the following: a unit (102) for detecting a straight center line, which detects the two straight parts of the target marker from the captured image and detects diagonals of two straight center lines (ac, bc) that are extensions of the two straight line parts, and intersection positions at which the two straight center lines (ac, bc) intersect a predetermined edge of the captured image in a state in which the two straight center lines (ac, bc) have been detected; and a parallel determination unit (105) that stores the slopes and intersection positions of the two straight center lines (ac, bc) captured from each of the numerous acquired images in the state in which the two straight center lines (ac, bc) have been identified, and that determines that the vehicle is parallel to the two straight parts of the target marking when deviations in the slopes and intersection positions of the respective two straight center lines (ac, bc) are within predefined tolerance ranges, and The mounting angle search unit searches for the mounting angle when it is determined that the vehicle is parallel to the two straight parts of the target marking. [3] Device (120, 220) for detecting a mounting angle according to claim 1, wherein the vehicle's internal camera (10) is mounted at the front or rear of the vehicle, The unit (101) receives numerous captured images to obtain a recorded image; these images are captured from different positions while the vehicle is moved in a straight line along one direction of the two straight parts of the target marker. continue with a unit (103) for detecting straight boundary lines, which detects the two straight parts of the target marker from the captured image and detects slopes of four straight boundary lines (ai, ao, bi, bo) which are extensions of inner and outer straight lines of the two straight parts, and boundary line intersection positions where the four straight boundary lines intersect a predetermined edge of the captured image in a state in which the four straight boundary lines (ai, ao, bi, bo) have been identified, wherein the two straight parts have predetermined widths, wherein a parallel determination unit (105) that stores the slopes and boundary line intersection positions of the four straight boundary lines (ai, ao, bi, bo) captured from each of the numerous acquired images in the state in which the four straight boundary lines (ai, ao, bi, bo) have been identified, and that determines that the vehicle is parallel to the two straight parts of the target marking when deviations in the slopes and intersection positions of the respective from the four straight boundary lines (ai, ao, bi, bo) are within predefined tolerance ranges, and The mounting angle search unit searches for the mounting angle when it is determined that the vehicle is parallel to the two straight parts of the target marking. [4] Device (120, 220) for detecting a mounting angle according to claim 3, further with a unit (201) for generating a representative captured image which, when it is determined that the vehicle is parallel to the two straight parts of the target marking, determines four representative straight boundary lines based on a result of extracting the four straight boundary lines (ai, ao, bi, bo) from each of the numerous captured images, and generates a representative captured image which reproduces the numerous captured images by representing the two straight parts of the target marking by the four representative straight boundary lines, wherein The bird's-eye view conversion unit creates the image from a bird's-eye view by converting the representative recorded image into a bird's-eye view. [5] Device (120, 220) for detecting a mounting angle according to one of claims 1 to 4, further comprising: a vehicle monitoring unit (106) that monitors a steering angle of the vehicle while the captured image is being obtained; and an alarm output unit (107) that issues an alarm if the vehicle is not moving straight ahead while the captured image is being received. [6] Device (100, 200) for calibrating a mounting angle, which calibrates a mounting angle in which an in-vehicle camera (10) for capturing an image of the ground around a vehicle is mounted on the vehicle, wherein the device for calibrating the mounting angle comprises: a unit (101) for obtaining a captured image, which obtains a captured image of the ground on which a target marker is marked with two straight parts parallel to each other and of equal width; a unit (109) for converting to a bird's-eye view, which receives a pre-defined value of the mounting angle of the in-vehicle camera (10) relative to the vehicle and converts the captured image into a bird's-eye view image, assuming that the in-vehicle camera (10) is mounted according to the pre-defined value, wherein the bird's-eye view image is an image in which the ground appearing in the captured image appears as if it were seen from above; a mounting angle search unit (110) which, by changing the predefined value of the mounting angle to be used when converting to a bird's-eye view, searches for the mounting angle at which both the parallelism of the two straight parts of the target marker in the bird's-eye view image and a difference in width between the two straight parts of the target marker in the bird's-eye view image are within predetermined tolerance ranges; and a mounting angle update unit (114) that updates the predefined value of the mounting angle based on the mounting angle found by the search and calibrates the mounting angle of the vehicle's in-vehicle camera (10) relative to the vehicle. [7] Device (100, 200) for calibrating a mounting angle according to claim 6, wherein the vehicle's internal camera (10) is mounted at the front or rear of the vehicle, The unit (101) receives numerous captured images to obtain a recorded image; these images are captured from different positions while the vehicle is moving straight ahead in one direction of the two straight parts of the target marker. wherein the device (120, 220) for detecting a mounting angle further comprises the following: a unit (102) for detecting a straight center line, which detects the two straight parts of the target marker from the captured image and detects slopes of two straight center lines (ac, bc) that are extensions of the two straight parts, and intersection positions at which the two straight center lines (ac, bc) intersect a predetermined edge of the captured image in a state in which the two straight center lines (ac, bc) have been identified; and a parallel determination unit (105) that stores the slopes and intersection positions of the two straight center lines (ac, bc) acquired from each of the numerous acquired images in the state in which the two straight center lines (ac, bc) were identified, and determines that the vehicle is parallel to the two straight parts of the target marking when it is detected that deviations in the slopes and intersection positions of the respective two straight center lines (ac, bc) are within predetermined tolerance ranges, and The mounting angle search unit searches for the mounting angle when it is determined that the vehicle is parallel to the two straight parts of the target marking. [8] Method for detecting a mounting angle wherein an in-vehicle camera (10) for detecting an image of the ground around a vehicle is mounted on the vehicle, wherein the method for detecting the mounting angle comprises: a step (S102) of obtaining a recorded image of the ground on which a target marker is marked with two parallel straight lines of equal width; a step (S115, S155) of obtaining a pre-defined value of the mounting angle of the vehicle's in-vehicle camera relative to the vehicle; a step (S116, S156) of converting the captured image into a bird's-eye view image when the pre-specified value is obtained, assuming that the vehicle's in-vehicle camera (10) is mounted on the vehicle according to the pre-specified value, wherein the bird's-eye view image is an image in which the ground in the captured image appears as if it were seen from above; a determination step (S117, S118, S157, S158) of detecting the parallelism of the two straight parts of the target marking in the bird's-eye view image and a difference in width between the two straight parts, and of determining whether the parallelism and the difference in width of the two straight parts detected are within predefined tolerance ranges; a step (S119, S159) of searching for a mounting angle in which, if either the parallelism or the difference in width of the two straight parts that have been detected is not within the predetermined tolerance ranges, the mounting angle is sought at which both the parallelism and the difference in width of the two straight parts are within the predetermined tolerance ranges by changing the predetermined value of the mounting angle of the vehicle's in-vehicle camera (10); and a step (S120, S160) to determine the mounting angle found by the search as the mounting angle of the vehicle's internal camera (10) relative to the vehicle. [9] Method for detecting a mounting angle according to claim 8, which is used for the vehicle-internal camera (10) for capturing an image of a ground located in front of or behind the vehicle, wherein In the step of capturing the recorded image, numerous recorded images are obtained, which are images taken from different positions while the vehicle is moving straight ahead along one direction of the two straight parts of the target marking, the procedure for determining the mounting angle further includes: a step (S103) of capturing a straight midline by - Capturing the two straight parts of the target marker from the captured image, - Capturing slopes of two straight center lines that are extensions of the two straight parts, and - Detecting intersection positions where the two straight center lines intersect a predetermined edge of the captured image in a state where the two straight center lines have been identified, as well as - Saving the slopes and the cutting positions in a predefined storage device; and a parallel determination step (S110, S150) which determines that the vehicle is parallel to the two straight parts of the target marking if deviations of the slopes and the intersection positions of the two straight center lines, which are captured from each of the numerous images taken, are within predefined tolerance ranges, and In the step of obtaining the predetermined value, the predetermined value of the mounting angle is obtained when it is determined that the vehicle is parallel to the two straight parts of the target marking.

Images