IMAGE PROCESSING SYSTEM AND IMAGE PROCESSING METHOD

Abstract

An image processing system comprises: several imaging devices; a first chart panel; a second chart panel; and an image processing unit, and the image processing unit comprises a position / angle estimation unit that estimates a position and angle of each of the several imaging devices, a collimated optical axis generation unit that collimates the optical axes of the several imaging devices by image processing, a parallelized image generation unit that produces an image obtained by parallelizing images acquired by several imaging devices, a perspective rectification processing unit that rectifies the perspectives of the images acquired by the several imaging devices so that they are directly opposite the second chart panel, and a distance difference correction unit.which corrects a measured distance from each of the multiple imaging devices to the second chart panel, a parallax calculation unit that calculates parallaxes of images acquired by the multiple imaging devices, and a relative angle correction value calculation unit that calculates a relative angle error correction value between the multiple imaging devices.

Description

Technical field

[0001] The present invention relates to an image processing system and method used for calibrating an onboard stereo camera. background

[0002] Normally, in-vehicle stereo cameras with a greater distance between the left and right cameras, i.e., a longer baseline length, are more advantageous for distance measurement. Furthermore, when an in-vehicle stereo camera is used as a sensor for autonomous driving, the optical axes of the two cameras must be aligned with high accuracy. While in a case where both cameras are housed in the same small enclosure, the rigidity of the enclosure allows the optical axes to be precisely aligned, the two cameras must be housed in separate enclosures to achieve a long baseline length, and physically aligning the optical axes is difficult.With regard to this problem, according to PTL 1, a method is performed to determine the misalignment of the optical axes of cameras from images captured by the cameras and by physically moving the cameras. Furthermore, according to PTL 2, a misalignment of the optical axes is detected by capturing images of the same pattern with cameras at intervals adapted to a baseline length. Citation list for patent literature PTL 1: JP 2018-031733 A PTL 2: WO 2019 / 087253 A Overview of the invention Technical problem

[0003] However, equipping an in-vehicle stereo camera with a drive mechanism that moves the cameras, as described in PTL 1, increases the price. Furthermore, the method described in PTL 2 requires providing two chart panels equidistant from each other, as the baseline length is, and cannot be used if the exact baseline length is unknown. Additionally, the two cameras must be essentially directly opposite the chart panels. As described above, there is a need for a technique that corrects optical axis misalignment for an in-vehicle stereo camera with an unknown baseline length and achieves accurate distance measurement.

[0004] The present invention was made with regard to the above problem, and one objective of the present invention is to provide means which enable a vehicle-internal stereo camera, the relative relationship between the cameras of which is unknown, to perform an accurate distance measurement with a low parallax error. Solution to the problem

[0005] To solve the above problem, an image processing system according to the present invention comprises: several imaging devices; a first diagram panel on which a repeating pattern has been drawn and which is arranged at a position separated from the several imaging devices by a first distance; a second diagram panel on which a repeating pattern and a non-repeating pattern have been drawn and which is arranged at a position separated from the several imaging devices by a second distance; and an image processing device, and the image processing device comprises a position / angle estimation unit that estimates a position and angle of each of the several imaging devices based on the imaging results of the repeating patterns imaged by the several imaging devices, and a unit for generating a collimated optical axis.which collimates the optical axes of the multiple imaging devices by image processing using an estimation result of the position / angle estimation unit; a parallelized image generation unit that produces an image obtained by parallelizing images acquired by the multiple imaging devices based on a processing result of the collimated optical axis generation unit; a perspective rectification processing unit that rectifies the perspectives of the images acquired by the multiple imaging devices using the images produced by the parallelized image generation unit so that they are directly opposite the second chart panel; a distance difference correction unit that corrects a measured distance from each of the multiple imaging devices to the second chart panel.by adjusting enlargements of the perspectively corrected images using the estimation result of the position / angle estimation unit, a parallax calculation unit that calculates parallaxes of the images acquired by the multiple imaging devices based on an imaging result of the non-repeating pattern in which a processing result of the distance difference correction unit has been reflected, and a relative angle correction value calculation unit that calculates a relative angle error correction value between the multiple imaging devices based on a calculation result of the parallax calculation unit and the second distance. Advantageous effects of the invention

[0006] By using the present invention, it is possible to provide means that enable a stereo camera, whose relative relationship between the cameras is unknown, to perform an accurate distance measurement with a low parallax error.

[0007] Further features related to the present invention will become apparent from the description and the accompanying drawings. In addition, other problems, configurations, and effects besides those mentioned above will be evident from the following embodiments. Brief description of the drawings [ Fig. 1] Fig. Figure 1 is a block diagram illustrating a functional configuration of an image processing system according to an embodiment of the present invention. [ Fig. 2] Fig. Figure 2 is a flowchart illustrating a processing operation carried out by an image processing system according to the embodiment of the present invention. [ Fig. 3] Fig. Figure 3 is a view illustrating an arrangement of a diagram board according to the embodiment of the present invention. [ Fig. 4] Fig. Figure 4 illustrates an example of a diagram arrangement used in the present invention. [ Fig. 5] Fig. Figure 5 illustrates another example of a diagram arrangement used in the present invention. Description of the embodiments

[0008] An embodiment of the present invention is described below with reference to the drawings. Fig. Figure 1 is a block diagram illustrating a functional configuration of an entire image processing system according to the embodiment of the present invention. The image processing system includes in-vehicle cameras 101 and 102, an image processing unit 100, and several diagram panels. More precisely, the several diagram panels refer to, as shown in Figure 1, the following: Fig. Figure 3 shows first diagram panels 302 and 303, which are arranged at positions separated by a first distance from camera units 304 and 305 and on which repeating patterns have been drawn, and a second diagram panel 301, which is arranged at a position separated from camera units 304 and 305 by a second distance greater than the first distance and on which a repeating pattern and a non-repeating pattern have been drawn. The image processing device 100, for example, includes a computer with a central processing unit (CPU) and memory, and the functional units included therein are as follows.

[0009] That is, the image processing device 100 includes distortion correction units 103 and 104, which correct the lens distortion of wide-angle cameras; repeat pattern detection units 105 and 106, which detect repeating patterns and are used to calibrate images from which the distortion has been corrected; camera position / angle estimation units 107 and 108 (corresponding to the position / angle estimation units in the claims), which estimate the angles and positions of the cameras from the repeating pattern; a relative parameter calculation unit 109, which calculates a relative position and a relative angle of the two cameras; a unit 110 for generating a collimated optical axis, which obtains collimated optical axes from the relative position and the relative angle and creates a rotation matrix for collimation; units 111 and 112 for generating parallelized images, which are applied to the images,from which the distortion has been removed, perform parallelization processing and generate parallelized images; processing units 113 and 114 for perspective rectification, which perform processing to correct the perspectives of the parallelized images so that they are directly opposite the chart boards; distance difference correction units 115 and 116, which adjust the distances from the two cameras to the chart boards; a parallax calculation unit 117, which calculates a parallax of a non-repeating pattern from the two perspective-corrected images; a relative angle correction value calculation unit 118, which calculates a relative angle error correction value between the imaging devices from the parallax; a correction value reflection unit 119, which corrects the relative angle using the correction value; and an output terminal 120.which outputs a corrected relative parameter as a calibration value.

[0010] Next, the operation of each unit will be described. Although the in-vehicle stereo camera contains two or more camera units, the present embodiment is described assuming two cameras. The two cameras are arranged in different positions, usually on the left and right sides. Furthermore, the units from the distortion correction unit 103 to the output terminal 120, which represent the functional units that receive input of imaging results from the cameras and output a computational result, can be arranged independently of each other as separate devices, or they can be arranged as a package in which some or all of the units from the distortion correction unit 103 to the output terminal 120 are housed in a single enclosure.This means that the image processing system according to the present embodiment forms a single unit, but can use a configuration obtained by combining several units. Furthermore, in some cases, an in-vehicle stereo camera is mounted in a vehicle by housing two cameras in separate enclosures to achieve a long baseline length, and is referred to in the present embodiment as a "separate-type camera".

[0011] As described later, the two in-vehicle cameras 101 and 102 capture images containing repeating and non-repeating patterns, and the resulting images are output as images containing distortion due to the wide-angle lenses. Since this distortion is unique to the camera units, the distortion correction units 103 and 104 convert the images into distortion-free images using a correction table or similar method.

[0012] Images from which the distortion has been removed are sent to the repeat pattern recognition units 105 and 106, and a known repeating pattern is determined from each image. Known markers are repeatedly arranged within this repeating pattern, and the position of each marker on the image is determined. Furthermore, as described later, the repeating patterns are arranged at multiple intervals, and at least one repeating pattern is drawn on the same plane as the non-repeating pattern.

[0013] Images from which repeating patterns, arranged at multiple intervals, have been determined are sent to the camera position / angle estimation units 107 and 108. The positions of the markers in an image space are known, so that if there are three or more markers, the camera position / angle estimation units 107 and 108 can calculate the three-dimensional positions and three angles of the cameras. Although a method can be used, for example, in which a design position and a design angle are specified as initial values, a projection position on an image plane is calculated using these parameters, a deviation from the actual marker detection position is assumed as an error function, the parameters are corrected so that the error function decreases, and the parameters are converged, the calculation method is not limited to this procedure.

[0014] The positions and angles of the two cameras, estimated by the camera position / angle estimation units 107 and 108, are sent to the relative parameter calculation unit 109. The relative parameter calculation unit 109 calculates the differences between the positions and angles of the two cameras and outputs these differences as relative parameters to unit 110 for generating a collimated optical axis.

[0015] The optical axes of the two cameras 101 and 102 are physically oriented in different directions, and it is necessary to align the directions of the optical axes to achieve correct parallax. Precise directional control would require physically changing the camera directions, which is not feasible for the in-vehicle stereo camera. Therefore, a method is used to generate images by aligning the directions of the left and right cameras through image processing, without altering their physical orientations. This processing is called collimation processing, and the direction of a virtual optical axis after collimation processing is generated by unit 110 to create a collimated optical axis.

[0016] For the direction of the collimated optical axis, a method can be used to align one of the left and right cameras with the other camera, using one of the left and right cameras as a reference camera, or a method can be used to determine a midpoint between the two optical axes. After collimation processing, a rotation matrix is ​​created from the angular deviation between the optical axis of the original camera and the virtual collimated optical axis. This rotation matrix is ​​sent to units 111 and 112 to generate parallelized images.Units 111 and 112, used for generating parallelized images, apply the rotation matrix sent by unit 110 (for generating a collimated optical axis) to the distortion-free images output by the distortion correction units 103 and 104, thus producing images aligned with the direction of the virtual collimated optical axis. These generated images are then sent to processing units 113 and 114 for perspective correction.

[0017] The perspective correction processing units 113 and 114 create the rotation matrices from the differences between the virtual collimated optical axes and the orientation of the diagram boards, perform an image conversion according to these rotation matrices, and thereby convert the images captured by the two cameras into images whose perspectives are corrected so that they are directly opposite the diagram boards and whose orientations are aligned with the left and right cameras. At the same time, the perspectives of the images captured by the two cameras are corrected so that they are directly opposite the diagram boards on which the non-repeating pattern and the repeating pattern are drawn on the same plane.In contrast to a configuration where the two cameras are housed in a single unit, the camera-separate type presents the particular challenge of physically ensuring a relative relationship between the two cameras. Consequently, misalignment in the relative relationship (such as a relative angle) between the two cameras can occur. The perspective correction processing units 113 and 114 perform the perspective correction processing, so that even in a state where the two cameras are not directly facing the chart boards, it is possible to position the two cameras directly facing the chart boards on which the non-repeating patterns have been drawn, and the parallax calculation unit 117 in the subsequent stage can accurately calculate any parallax.Consequently, the Relative Angle Correction Value Calculation Unit 118 can accurately calculate the Relative Angle Correction Value. Note that, although parallelization processing and perspective rectification processing are described here as distinct processes, both are rotation matrix processes. Therefore, it is also possible to create a rotation matrix obtained by simultaneously performing parallelization and perspective rectification processing and to perform the image conversion in one go.

[0018] Since the images undergoing perspective correction processing have different distances from the two cameras to the chart boards, it is necessary to enlarge or reduce the images. This processing is performed by the distance difference correction units 115 and 116. More precisely, the distance difference correction units 115 and 116 perform the same processing as the processing in which the distance between the two cameras and the chart boards is essentially adjusted, by matching the magnification of one of the perspective-corrected images to the magnification of the other image, thus obtaining images with the same magnification.

[0019] The image for which the distance difference correction processing is complete is sent to the parallax calculation unit 117. Parallax calculation unit 117 determines the parallax using the non-repeating pattern. Here, parallax is an index that indicates the difference in how an imaging target appears from the left and right cameras. Methods for obtaining the parallax include, for example, semi-global matching (SGM), sum-of-absolute-difference (SAD), and similar techniques. Since these calculation methods are already known, their detailed descriptions are omitted. Because the non-repeating pattern is a known pattern, the parallax is calculated by finding the known pattern in the images obtained from the left and right cameras through correlation processing or similar methods and determining the difference between the positions.Although a theoretical parallax value is generally determined according to the distances between the cameras and the charts with the non-repeating pattern, there is a deviation in the actual measured parallax value due to an angle estimation error. By converting this parallax deviation into an angle, it is possible to correct the relative angle between the two cameras. This calculation is performed by the relative angle correction value calculation unit 118.

[0020] This correction value is sent to the correction value reflection unit 119, and the relative parameter for which the correction value calculated by the relative angle correction value calculation unit 118 was reflected is output as the final output from the output terminal 120. Although the angle to be reflected as a correction value is usually a small angle, the reliability of the value calculated by the relative parameter calculation unit 109 is assumed to be low in cases where this correction value is large. Therefore, in cases where the correction value exceeds a predetermined threshold, it is also possible to output an error without outputting the corrected value. In this case, it is possible to initiate anomaly processing, where the error is used as a trigger for a process such as restarting image acquisition or repositioning the cameras.

[0021] Although it is possible to output the value corrected by the correction value reflection unit 119 unchanged from the output terminal 120, it is also possible to revalidate the corrected value using the non-repeating pattern. An operation at the time of revalidation is carried out according to a Fig. The processing sequence shown in Figure 2 is described. Note that this sequence also starts with two cameras, namely those aligned horizontally towards the diagram panels representing the imaging targets.

[0022] Initially, it is assumed that the images captured by both the left and right cameras are fed into the distortion correction unit 103 to generate a distortion-free image. Furthermore, step 201 is an initial estimation process of the camera positions and angles, utilizing the repeating patterns drawn on the spaced-apart chart panels, as described later. This step corresponds to the processing in the repeating pattern detection units 105 and 106, the camera position / angle estimation units 107 and 108, and the relative parameter calculation unit 109. After the processing in step 201 is completed, the process proceeds to step 202.

[0023] In step 202, the left and right camera images undergo parallelization processing based on a result obtained in step 201. This step corresponds to the processing by unit 110 to generate a collimated optical axis and by units 111 and 112 to generate parallelized images. In step 203, to which the process then proceeds, perspective correction processing is performed on the diagram boards on which both the repeating pattern and the non-repeating pattern to be described later have been drawn. This step corresponds to the processing by units 113 and 114 for perspective correction.

[0024] In step 204, the distances from the camera positions determined in step 201 to the diagram boards, on which both the repeating pattern and the non-repeating pattern (to be described later) were drawn, are calculated, and an image conversion is performed to eliminate any distance difference between the left and right cameras. This step corresponds to the processing of the distance difference correction units 115 and 116. In the subsequent processing in step 205, the parallax calculation unit 117, using the image for which the distance difference correction has been completed, performs a parallax calculation that utilizes the non-repeating pattern that underwent perspective correction in step 203.

[0025] Since the theoretical parallax value is determined according to the camera distances to the non-repeating patterns processed for perspective correction in step 203, the theoretical value is calculated from the camera positions obtained in step 201. Although this theoretical value and the actual parallax value obtained from the images will match if the camera positions and angles obtained in step 201 are correct, a difference corresponding to a deviation will occur if a deviation is present. This difference is the relative angle correction value, and only two of the camera positions and angles need to be corrected to achieve high accuracy. The calculation of this correction value is performed by the relative angle correction value calculation unit 118.

[0026] In step 206, this difference is determined, and if the difference is smaller than a predetermined, defined value, the camera positions and angles obtained in step 201 are considered correct. In the next step, 207, the correction value reflection unit 119 reflects the relative angle corresponding to the angular difference obtained in step 201, and the process is completed. This reflected value represents a final output.

[0027] In step 206, if the difference is equal to or greater than the defined value, the correction value reflection unit 119 counts the number of repetitions in step 208. If the number of repetitions is equal to or greater than the predetermined defined value, it is determined that any error that has occurred so far has become so large that it can no longer be corrected, and the processing is terminated assuming an error. If the number of repetitions is less than the defined value, the process proceeds to step 209, and the correction value reflection unit 119 reflects the difference in camera angles obtained in step 205. The processing in steps 202 to 205 is repeated for these new camera angles, and in step 206, a difference between the theoretical value and the parallax value obtained from the image is calculated.If the difference returns to or exceeds the defined value, the difference is reflected further, and the processing is repeated in steps 202 to 205. If, even after the processing has been repeated a certain number of times or more, the difference does not fall below the defined value, an error is detected in step 208, and the processing is terminated.

[0028] According to the procedure described above, after the initial estimation of the camera positions and angles using the repeating patterns arranged at multiple intervals, the perspectives of the captured images are rectified so that they directly correspond to the diagram boards on which both the repeating and non-repeating patterns are drawn. This rectification corrects the distance difference between the left and right cameras and any relative angle error between them. Consequently, an in-vehicle stereo camera system, where the relative relationship between the left and right cameras is unknown, can perform accurate parallax correction and calibration—for example, in a case where the two cameras are housed in separate enclosures and mounted on a vehicle, similar to a standalone camera system.

[0029] Fig. Figure 3 shows an example of a chart board arrangement used in the embodiment of the present invention. Fig. Figure 3 is a top-down view of the arrangement. In the present embodiment, as described above, a second diagram panel, on which a repeating pattern and a non-repeating pattern are arranged, is designated 301, first diagram panels, on which only repeating patterns are arranged, are designated 302 and 303, and camera units are designated 304 and 305.

[0030] Examples of the repeating pattern and the non-repeating pattern, arranged on the second chart panel 301, are shown in the Fig. 4 and Fig. 5 shown. First, in Fig. Four repeating patterns are designated 401 and 402, and one non-repeating pattern is designated 403. In Fig. Five non-repeating patterns are designated 501 and 502, and one repeating pattern is designated 503. Since the non-repeating pattern must be on the same level as one of the repeating patterns, this is one of several arranged diagrams that refer to Fig. As described in Figure 3, a diagram is shown in which the repeating pattern and the non-repeating pattern are drawn. The non-repeating pattern and the repeating pattern are drawn on the same plane as that of the second diagram panel 301, so that it is possible to perform the perspective correction processing on the non-repeating pattern at the same time as the perspective correction processing on the diagram panel on which the repeating patterns are drawn. Furthermore, it is possible to improve the accuracy of the parallax calculation performed by the parallax calculation unit 117 using the non-repeating pattern. Consequently, it is possible to accurately determine the relative angle error between the cameras.Furthermore, in the present embodiment, the second diagram panel 301, on which the non-repeating pattern is drawn, is positioned further away from the camera than the first diagram panels 302 and 303, on which only the repeating patterns are drawn. This is because the non-repeating pattern is used for camera parallax measurement, and the accuracy of the parallax measurement increases with the greater the distance between the cameras and the non-repeating pattern.

[0031] Note that the repeating pattern, although the repeating pattern in Fig. 4 is formed by aligning ring-shaped patterns in a grid pattern and in Fig. 5. A grid pattern of the type of a chessboard is not limited to these examples, as long as a pattern facilitates the recognition of points serving as features and causes minimal recognition errors in an image by regularly aligning the multiple points. Furthermore, although the non-repeating pattern is both in Fig. 4 as well as in Fig. 5. If a non-repeating pattern is used, any pattern is permissible as long as the same pattern is hardly generated in multiple sections. Although Fig. Figure 4 shows an example where the repeating patterns exist in two sections and the non-repeating pattern exists in one section, and Fig. Figure 5 presents an example where the repeating pattern exists in one section and the non-repeating patterns exist in two sections; the number of patterns is not limited thereto. Furthermore, diagram tables 302 and 303 in Fig. 3 only the repeating patterns, and the repeating patterns are the same as those in the Fig. 4 and Fig. 5 shown. Although the same recognition method can be used as long as the patterns of the repeating patterns are the same between all diagrams, and it is therefore desirable to make the patterns the same, different patterns can also be used.

[0032] Although in Fig. 3. Since all diagram panels are designed to stand vertically on the floor, so that the camera directions are essentially directly opposite the diagram panels, the first diagram panels (302 and 303 in) must be Fig.3) which only contain repeating patterns, are not directly facing the cameras and therefore do not need to be vertically oriented. For example, it is also possible to arrange the chart panels by attaching them to a floor or ceiling surface. If there is insufficient space laterally, it is also possible to attach the chart panels to a wall surface.

[0033] According to the embodiment of the present invention described above, the following functions and effects can be achieved. (1) An image processing system according to the present invention comprises: several imaging devices; a first diagram panel on which a first diagram, which is a repeating pattern, has been drawn and which is arranged at a position separated from the several imaging devices by a first distance; a second diagram panel on which a repeating pattern and a non-repeating pattern have been drawn and which is arranged at a position separated from the several imaging devices by a second distance; and an image processing device, and the image processing device comprises a position / angle estimation unit which estimates a position and an angle of each of the several imaging devices based on the imaging results of the repeating patterns imaged by the several imaging devices, and a unit for generating a collimated optical axis.which collimates the optical axes of the multiple imaging devices by image processing using an estimation result of the position / angle estimation unit; a parallelized image generation unit that produces an image obtained by parallelizing images acquired by the multiple imaging devices based on a processing result of the collimated optical axis generation unit; a perspective rectification processing unit that rectifies the perspectives of the images acquired by the multiple imaging devices using the images produced by the parallelized image generation unit so that they are directly opposite the second chart panel; a distance difference correction unit that corrects a measured distance from each of the multiple imaging devices to the second chart panel.by adjusting enlargements of the perspectively corrected images using the estimation result of the position / angle estimation unit, a parallax calculation unit that calculates parallaxes of the images acquired by the multiple imaging devices based on an imaging result of the non-repeating pattern in which a processing result of the distance difference correction unit has been reflected, and a relative angle correction value calculation unit that calculates a relative angle error correction value between the multiple imaging devices based on a calculation result of the parallax calculation unit and the second distance.

[0034] According to the above configuration, it is possible to provide means that allow a stereo camera, whose relative relationship between the cameras is unknown, to perform an accurate distance measurement with a low parallax error.

[0035] (2) The second distance is greater than the first distance. The second distance, at which the second diagram is arranged, is greater than the first distance, so that it is possible to measure the parallax between the imaging devices more accurately, and consequently, it is possible to correct a relative error between the imaging devices more accurately.

[0036] (3) The image processing device indicates an error if the relative angle error correction value calculated by the relative angle correction value calculation unit is greater than a predetermined threshold. Consequently, it is possible to make a decision, for example, to discard the correction value and start the image acquisition process again.

[0037] (4) If the relative angle error correction value calculated by the relative angle correction value calculation unit is greater than a predetermined threshold, the image processing unit reflects the relative angle error correction value in the multiple imaging units and then re-executes the image processing. Consequently, it is possible to repeat the fine-tuning of the imaging units and improve the accuracy of the correction.

[0038] (5) Furthermore, an image processing method according to the present invention comprises: estimating a position and angle of each of several imaging devices based on an imaging result of repeating patterns mapped by the several imaging devices and drawn on a first chart board and a second chart board located at positions separated from the several imaging devices by a first distance and a second distance; collimating the optical axes of the several imaging devices using a result of the estimation; rectifying the perspectives of images acquired by the several imaging devices so that they are directly opposite the second chart board using a result of the collimation;Correcting a measured distance from each of the multiple imaging devices to the second chart panel by adjusting the magnifications of the perspective-corrected images using an estimation result; calculating parallaxes of the images acquired by the multiple imaging devices based on an imaging result of a non-repeating pattern plotted on the second chart panel for which the measured distance was corrected; and calculating a relative angle error correction value between the multiple imaging devices based on the calculated parallax and the second distance. Consequently, the same effect as that in (1) can be expected. Furthermore, the method according to the present invention also has the same configuration and effect as those in (2) to (4).

[0039] Note that the present invention is not limited to the embodiment described above and can be modified in various ways. For example, the embodiment described above has been explained in detail to provide a comprehensive understanding of the present invention, and the present invention is not necessarily limited to aspects that include all the components described. Furthermore, it is possible to replace some of the components of a particular embodiment with components of another embodiment. It is also possible to add the components of the other embodiment to the components of the particular embodiment. In addition, some of the components of each embodiment can be removed, or the other components can be added or replaced by some of the components of each embodiment. Reference symbol list 101, 102 Camera (Imaging device) 107, 108 Camera position / angle estimation unit (position / angle estimation unit) 110 Unit for generating a collimated optical axis 111, 112 Unit for generating parallelized images 113, 114 Processing unit for perspective correction 115, 116 Distance difference correction unit 117 Parallax calculation unit 118 Relative angle correction value calculation unit 301 second chart table 302, 303 first diagram table QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2018-031733 A

[0002] WO 2019 / 087253 A

[0002]

Claims

[1] An image processing system has: several imaging facilities; a first diagram panel on which a repeating pattern has been drawn and which is arranged at a position separated from the multiple imaging devices by a first distance; a second diagram panel, on which a repeating pattern and a non-repeating pattern have been drawn, and which is positioned at a location separated from the multiple imaging devices by a second distance; and an image processing unit, the image processing unit includes the following: a position / angle estimation unit that estimates the position and angle of each of the multiple imaging devices based on the imaging results of the repeating patterns mapped by the multiple imaging devices, a unit for generating a collimated optical axis that collimates the optical axes of the multiple imaging devices by image processing using an estimation result from the position / angle estimation unit, a unit for generating parallelized images, which produces an image obtained by parallelizing images acquired by the multiple imaging devices based on a processing result of the unit for generating a collimated optical axis, a perspective correction processing unit that corrects the perspectives of the images captured by the multiple imaging devices using the images produced by the parallelized image generation unit, so that they are directly opposite the second diagram panel, a distance difference correction unit that corrects a measured distance from each of the multiple imaging devices to the second chart panel by adjusting magnifications of the perspectively corrected images using the estimation result of the position / angle estimation unit, a parallax calculation unit that calculates parallaxes of the images acquired by the multiple imaging devices based on an imaging result of the non-repeating pattern in which a processing result of the distance difference correction unit was reflected, and a relative angle correction value calculation unit that calculates a relative angle error correction value between the multiple imaging devices based on a calculation result of the parallax calculation unit and the second distance. [2] Image processing system according to claim 1, wherein the second distance is longer than the first distance. [3] Image processing system according to claim 1, wherein the image processing device indicates an error if the relative angle error correction value calculated by the relative angle correction value calculation unit is greater than a predetermined threshold. [4] Image processing system according to claim 1, wherein the image processing device, when the relative angle error correction value calculated by the relative angle correction value calculation unit is greater than a predetermined threshold, reflects the relative angle error correction value in the multiple imaging devices and then performs the image processing again. [5] Image processing method performed by an image processing system, wherein the image processing method comprises: Estimating the position and angle of each of several imaging devices based on an imaging result of repeating patterns mapped by the several imaging devices and drawn on a first chart panel and a second chart panel located at positions separated from the several imaging devices by a first distance and a second distance; Collimating the optical axes of multiple imaging devices using a result of the estimation; Correcting the perspectives of images taken by the multiple imaging devices so that they are directly opposite the second diagram panel, using a result of collimation; Correcting a measured distance from each of the multiple imaging devices to the second chart panel by adjusting the magnifications of the perspective-corrected images using a result of the estimation; Calculating parallaxes of the images acquired by the multiple imaging devices based on an imaging result of a non-repeating pattern plotted on the second chart panel for which the distance measurement has been corrected; and Calculating a relative angle error correction value between the multiple imaging devices based on the calculated parallax and the second distance. [6] Image processing method according to claim 5, wherein the second distance is longer than the first distance. [7] Image processing method according to claim 5, wherein an error is indicated if the calculated relative angle error correction value is greater than a predetermined threshold. [8] Image processing method according to claim 5, wherein if the calculated relative angle error correction value is greater than a predetermined threshold, the image processing is performed again after the relative angle error correction value has been reflected in the multiple imaging devices.

Images