Stereo camera device

Abstract

To provide a stereo camera device that stably maintains the attitude between cameras and a distance measuring sensor.SOLUTION: A stereo camera device has: a plurality of cameras; a distance measuring sensor that measures the distance to an object; a holding member that holds the distance measuring sensor and the plurality of cameras while aligning their optical axis directions; an electronic circuit; and a case that accommodates the plurality of cameras and the distance measuring sensor held by the holding member and the electronic circuit.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to a stereo camera device.

Background Art

[0002] In recent years, in addition to the automotive field, various safety devices have been installed in vehicles in the industrial machinery field such as forklifts and hydraulic shovels to avoid accidents caused by the carelessness of drivers and operators. As one of the safety measures, the installation of an external recognition sensor for detecting surrounding objects has been promoted. External recognition sensors include stereo cameras, millimeter-wave radars, and the like. By using such safety devices, when an object or an operator around the vehicle is detected, according to the detected surrounding situation, attention is called to the driver, the vehicle is driven and controlled, and an accident caused by a collision of the vehicle is avoided.

[0003] A stereo camera calculates the distance using parallax information from images obtained by two cameras. Therefore, although the detection distance is shorter than that of a monocular camera, objects can be detected accurately in the short-distance region. Distance measurement by a stereo camera is based on the principle that the optical axes of two cameras are parallel. When a stereo camera is mounted on a vehicle, it is difficult to maintain a precise parallel arrangement due to changes in the lens posture over time due to temperature or vibration, and mechanical tolerances of the housing.

[0004] A camera stay is proposed which is arranged in parallel at positions where the optical axes of a set of monocular cameras are on the same straight line and integrally connected, an image correction means for correcting a captured image from each monocular camera, and a distance calculation means for obtaining a parallax from the corrected image corrected by the image correction means and calculating a distance to a subject (see, for example, Patent Document 1). A method of calibrating the parameters of a stereo camera is proposed by installing a calibration target in front of the stereo camera and a laser rangefinder behind it, and using the distance L1 from the laser rangefinder to the reference plane of the camera stay and the distance L2 from the laser rangefinder to the calibration target (see, for example, Patent Document 2). A technique has been proposed in which the lens misalignment is estimated from the state of image distortion in an image captured by a stereo camera, and the image is electronically deformed by image correction processing to perform distortion correction and reproduce the original image (see, for example, Patent Document 3).

Summary of the Invention

Problems to be Solved by the Invention

[0005] As the camera has a wider angle, the image distortion that occurs becomes more complex and the correction amount also increases. It is difficult to achieve sufficient distortion correction only by geometric correction by polynomial approximation, and a mechanical structure for stably maintaining the posture between cameras is required. In an environment exposed to severe temperature changes, vibrations, shocks, etc., such as a construction site, the arrangement and posture between cameras change over time, and a deviation occurs between the distance measurement result and the parallax calculation. Non-linear deviations are difficult to predict in advance, and the distance measurement accuracy may deteriorate.

[0006] One aspect of the present invention aims to provide a stereo camera device that stably maintains the posture between a camera and a distance measurement sensor.

Means for Solving the Problems

[0007] In one aspect, a stereo camera device includes a plurality of cameras, a distance measuring sensor that measures the distance to an object, a holding member that holds the distance measuring sensor and the plurality of cameras with their optical axis directions aligned, an electronic circuit that calibrates image ranging by the plurality of cameras based on the measured distance of the distance measuring sensor, and a case as a protective member that houses the plurality of cameras, the distance measuring sensor, and the electronic circuit held by the holding member. The electronic circuit calibrates the difference in the optical axis direction between the optical centers of the plurality of cameras and the distance measuring origin of the distance measuring sensor in a state where the distance measuring sensor and the plurality of cameras are held by the holding member. Using the above, the image ranging by the plurality of cameras is adjusted according to the measurement distance of the ranging sensor The holding member is attached to the inner wall surface of the case.

Advantages of the Invention

[0008] A stereo camera device is realized that stably holds the posture between the camera and the distance measuring sensor.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2A

Figure 2B

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8A

Figure 8B

Embodiments for Carrying Out the Invention

[0010] FIG. 1 is an external view of the stereo camera device 100 according to the embodiment, and FIG. 2A is a hardware configuration diagram of the stereo camera device 100. As shown in FIGS. 1 and 2A, the stereo camera device 100 includes cameras 10A and 10B, a distance measurement sensor 20, an electronic circuit 50, and a housing case 101 that houses these optical devices and the electronic circuit 50. The housing case 101 is an example of a case. In this example, two cameras 10A and 10B are used, but distance measurement may be performed using three or more cameras.

[0011] In FIG. 1, the arrangement direction of the cameras 10A and 10B is the X direction, the optical axis direction is the Z direction, and the direction orthogonal to both the X direction and the Z direction is the Y direction. As will be described later, the cameras 10A and 10B and the distance measurement sensor 20 are held in a predetermined positional relationship inside the housing case 101 by a holding member 30 (see FIGS. 7, 8A, and 8B). The housing case 101 functions as a protective member for these optical devices and the electronic circuit 50. By arranging the cameras 10A and 10B and the distance measurement sensor 20 inside the housing case 101, robustness against colliding objects, dustproofness against dust and dirt, waterproofness against rain, etc. are ensured. The housing case 101 does not necessarily have complete airtightness as long as it can protect the internal optical devices, electronic circuits, etc. In one embodiment, the cameras 10A and 10B and the distance measurement sensor 20 are integrally held in a fixed positional relationship by the holding member 30 inside the housing case 101 to maintain positional accuracy.

[0012] The housing case 101 is, for example, a case made of metal or alloy formed by die-casting. In the example of FIG. 1, the housing case 101 is made of aluminum die-casting. The housing case 101 is formed of an upper case 15 and a lower case 40 that can be diagonally divided with respect to the bottom surface, and maintenance, repair, replacement, etc. of the components housed inside the housing case 101 are easy.

[0013] The housing case 101 has camera openings 16A and 16B that enable imaging by cameras 10A and 10B, and a distance measurement opening 17 for distance measurement by the distance measurement sensor 20. The housing case 101 has a cover glass 102 as a light transmissive member on its foremost surface. The cover glass 102 closes the camera openings 16A and 16B and the distance measurement opening 17, maintaining light transmissivity while keeping the housing case 101 airtight. The cover glass 102 is preferably a single sheet of plate glass, preventing optical errors due to differences in thickness and warping among the camera openings 16A, 16B, and the distance measurement opening 17.

[0014] The electronic circuit 50 performs image processing, distance calculation processing, etc. based on the image data (captured image) of the measurement object captured by the cameras 10A and 10B, and measures the distance to the measurement object.

[0015] On the other hand, the distance measurement sensor 20 measures the distance to the measurement object by receiving the reflected light of the light irradiated to the measurement object in order to calibrate (calibrate) image distance measurement. The distance measurement sensor 20 is, for example, a ToF sensor that measures the distance to the measurement object based on the time of flight (Time of Flight: TOF) of light from when the measurement object is irradiated with light (electromagnetic wave) until the reflected light from the measurement object is received.

[0016] FIG. 2B is a functional block diagram of the electronic circuit 50. The electronic circuit 50 may be a microprocessor composed of a large-scale integrated circuit (LSIC), or may be a logic device such as a field programmable gate array (FPGA). The electronic circuit 50 functions as a control unit 50A, and as its functions, has an image processing unit 51, a parallax calculation unit 52, a calibration calculation unit 53, a distance calculation unit 54, and a data input unit 55.

[0017] The data input unit 55 receives the captured images output from each of the cameras 10A and 10B and the distance information output from the distance measurement sensor 20. The captured images are input from the data input unit 55 to the image processing unit 51. The distance information is input from the data input unit 55 to the calibration calculation unit 53.

[0018] The image processing unit 51 performs appropriate image processing on the captured images to generate corrected images and supplies the corrected images to the disparity calculation unit 52. The disparity calculation unit 52 calculates the disparity p between the cameras 10A and 10B from the corrected images by a method such as the pattern matching method. The disparity p is input to the distance calculation unit 54 and the calibration calculation unit 53.

[0019] The calibration calculation unit 53 calculates the distance Z from the optical centers of the cameras 10A and 100B to the measurement object based on the distance information L to the measurement object obtained by the distance measurement sensor 20 and the known positional relationship between the distance measurement sensor 20 and the cameras 10A and 10B. The calibration calculation unit 53 uses the calculated distance Z and the disparity p input from the disparity calculation unit 52 to obtain the B*f value and the disparity offset Δp, and supplies the B*f value and Δp to the distance calculation unit 54. Here, as will be described later, the B*f value is the product of the actual distance between the optical axes (baseline length) B between the cameras 10A and 10B and the actual focal length f.

[0020] The distance calculation unit uses the disparity p input from the disparity calculation unit 52, the disparity offset Δp and the B*f value input from the calibration calculation unit 53 to calculate the distance Z to the measurement object as Z = B × f / (p + Δp) (1) and obtains it.

[0021] The calibration of the stereo camera device 100 may be performed in the product inspection before the factory shipment of the stereo camera device 100, or may be performed using a measurement object during use after the stereo camera device 100 is installed in a moving body or the like.

[0022] <Principle of distance measurement> Figure 3 is a diagram for explaining the principle of distance measurement of the stereo camera device 100. Cameras 10A and 10B are installed at a distance B apart. The distance B is the distance between the optical axes of cameras 10A and 10B, and is also called the baseline. Let the focal lengths of cameras 10A and 10B be f, and their respective optical centers be O A , and O B , and their respective imaging surfaces be s A , and s B . What is sought is the distance Z from the optical centers of cameras 10A and 10B to the measurement object A.

[0023] The image of the measurement object A located at a position Z away from the optical center OA of camera 10A in the optical axis direction (Z direction) is the intersection point P A between the straight line A - O A and the imaging surface s A . On the other hand, in camera 10B, the same measurement object A forms an image at the position P B on the imaging surface s B . Here, let the intersection point of the straight line PL passing through the optical center OB of camera 10B and parallel to the straight line A - O A and the imaging surface s B be P A '. The distance between P A ' and P B is the parallax p between cameras 10A and 10B.

[0024] The point P A ' represents the position of the image point P A with respect to the optical axis of camera 10A. The parallax p is the combined distance of P A ' and P B , and corresponds to the amount of positional deviation on the images obtained by photographing the same measurement object A with the two cameras 10A and 10B.

[0025] Since the triangles A - O A - O B and the triangle O B - P A '- P B are similar, Z = B × f / p (2) This is obtained. If the distance (baseline length) B between the optical axes of cameras 10A and 10B and the focal length f are known, the distance Z can be obtained from the parallax p. However, in reality, due to the influence of temperature changes, vibrations, etc., the baseline length B, the focal length f, and the parallax p change slightly due to the temporal variation of the lens optical axis, etc. Therefore, as described above, Z is obtained by Equation (1) using the parallax offset Δp. Z = B × f / (p + Δp) (1) By measuring the distance Z to the measurement object A from Equation (1) at multiple positions, the actual values of B and f can be obtained by a system of simultaneous equations.

[0026] <Calibration process> FIG. 4 is a flowchart of calibration, and FIG. 5 is a diagram showing the positional relationship of the devices during calibration. In the inspection before factory shipment, calibration may be performed using the calibration target T. During actual distance measurement, as described above, calibration may be performed using the measurement object. In FIG. 5, for the purpose of explaining the calibration process, the distance measurement sensor 20 is drawn behind (-Z side) the cameras 10A and 10B, but the distance measurement sensor 20 may be integrally held with the cameras 10A and 10B by the holding member 30.

[0027] First, as shown in FIG. 5, a calibration target T is installed in front of the measurement direction (Z direction) of the stereo camera device 100 (S1). Next, the stereo camera device 100 is installed at a predetermined position so that the light beam emitted from the distance measurement sensor 20 irradiates the feature point T1 of the calibration target T (S2).

[0028] The distance measurement sensor 20 includes a light source that irradiates a calibration target T with a light beam, and a photodetector that detects the reflected light from the calibration target T. The light source is, for example, a laser diode, and the photodetector is, for example, a photodiode. The optical axes of the light source and the photodetector are aligned in the same direction. The photodetector measures the time of flight ToF from when the light beam is emitted from the light source until the reflected light is received, and measures the distance L1 to the calibration target T based on the measured time (S3). The distance calculation unit 54 of the control unit 50A calculates the distance Z between the optical center O of the camera and the calibration target T from the measured distance L1 measured by the distance measurement sensor 20 and the known value ΔZ (S4). The known value ΔZ is the distance between the distance measurement origin of the distance measurement sensor 20 and the optical center O of the cameras 10A and 10B. The optical centers O of the cameras 10A and 10B attached to the same holding member 30 and the distance measurement origin of the distance measurement sensor 20 are in substantially the same plane, and ΔZ can be regarded as zero. When there is a difference in the Z direction between the optical centers O of the cameras 10A and 10B and the distance measurement sensor 20 in the state of being attached to the holding member 30, the distance Z is calculated using ΔZ. origin When there is a difference in the Z direction, the distance Z is calculated using ΔZ.

[0029] On the other hand, the cameras 10A and 10B image the feature point T1 of the calibration target T (S5). The image processing unit 51 processes the image of the imaged feature point T1 to generate corrected image data (corrected image) (S6). The disparity calculation unit 52 calculates the disparity p with respect to the calibration target T from the generated corrected image (S7). The positions of the calibration target T in the Z direction are changed, and steps S1 to S7 are repeated two or more times (YES in S8).

[0030] The calibration calculation unit 53 calculates the actual B*f value and the disparity offset Δp using two or more pairs of the distance Z and the disparity p. This B*f value and the disparity offset Δp are calibration data.

[0031] When the stereo camera device 100 is attached to a vehicle and used at a work site, the stereo camera device 100 detects obstacles and workers in the front, side, and rear. Based on the relative speed, relative position, etc. between the obstacle or worker and the vehicle, an alert may be issued or the automatic brake may be activated if there is a risk of collision. If the ranging accuracy is low, the automatic brake may frequently be activated, which may hinder the work. Therefore, it is desirable to activate the automatic brake only when the risk of contact with a person is high. For this purpose, a resolution capable of distinguishing between workers and materials, etc. is required. Also, it is necessary to maintain the ranging performance even in an outdoor environment with a large temperature difference, dust, and raindrops.

[0032] In the embodiment, the camera 10A and 10B and the ranging sensor 20 are held by a holding member 30 separate from the housing case 101 by the mechanical structure described below, so as to keep the positional accuracy between the camera 10A and 10B and the ranging sensor 20 as high as possible. Also, even when the positional relationship of the optical devices changes over time due to temperature differences, vibrations, etc., the measurement accuracy of the stereo camera device 100 is compensated by the above-described calibration function.

[0033] <Device Configuration> FIG. 6 is an exploded perspective view of the camera 10. Since the cameras 10A and 10B have the same configuration, they will be described as one camera 10. The camera includes a lens 21 and an image sensor 23 mounted on a printed circuit board 25. The lens 21 has a plurality of lens elements inside the lens cell 211. The image sensor 23 is, for example, a CMOS sensor. The lens 21 and the printed circuit board 25 on which the image sensor 23 is mounted are incorporated into an aluminum die-cast base member 22 to form a monocular camera 10. The base member 22 is formed with a cylindrical threaded hole 24 for receiving the lens 21. The threaded hole 24 is formed coaxially with the optical axis of the lens 21 inside a cylinder 223 protruding in the Z direction of the base member 22. The printed circuit board 25 is fixed to the mounting surfaces 221 and 222 of the base member 22 so that the image sensor 23 is coaxial with the optical axis of the lens 21.

[0034] The mounting surface 221 of the base member 22 surrounds the periphery of the image sensor 23, and mounting surfaces 222 are provided at a plurality of locations outside the mounting surface 221. As will be described later, the mounting surface 222 is used to temporarily fix the vicinity of the outer edge of the printed circuit board 25 at a plurality of locations, and finally fix the printed circuit board 25 with the mounting surface 221. By providing the mounting surface 222 at a position symmetric with respect to the central axis of the image sensor 23, it is possible to prevent the generation of a tensile stress biased on the printed circuit board 25 due to the curing shrinkage of the adhesive during temporary fixing, and suppress displacement.

[0035] FIG. 7 is an exploded perspective view of the stereo camera device 100, FIG. 8A is a vertical cross-sectional view taken along the YZ plane of FIG. 1, and FIG. 8B is a horizontal cross-sectional view taken along the XZ plane of FIG. 1. As shown in FIGS. 7 and 8B, the cameras 10A and 10B and the distance measuring sensor 20 are attached to the holding member 30. As described with reference to FIG. 6, the lens 21A of the camera 10A is fitted into the screw hole 24A of the base member 22A, and the optical axis of the lens 21A and the image sensor 23A mounted on the printed circuit board 25A are aligned. The cylinder 223 of the base member 22A is inserted into the fitting hole 13A of the holding member 30 and positioned.

[0036] Similarly, the lens 21B of the camera 10B is fitted into the screw hole 24B of the base member 22B, and the optical axis of the lens 21B and the image sensor 23B mounted on the printed circuit board 25B are aligned. The cylinder 223 (see FIG. 6) of the base member 22B is inserted into the fitting hole 13B of the holding member 30 and positioned.

[0037] By the fitting holes 13A and 13B, the cameras 10A and 10B are attached to the holding member 30 such that their respective optical axes are parallel. In each of the cameras 10A and 10B, the printed circuit boards 25A and 25B on which the image sensors 23A and 23B are mounted are aligned with the optical axes of the lenses 21A and 21B, and a tilt angle with the optical axis as the rotation axis is defined. Therefore, even after the cameras 10A and 10B are attached to the holding member 30 with their optical axes parallel to each other, the optical arrangement positioned by each camera 10 is maintained.

[0038] The distance measuring sensor 20 is mounted on the printed circuit board 31. The distance measuring sensor 20 is disposed substantially midway between the cameras 10A and 10B such that its light beam input / output window is exposed within the opening 14 of the holding member 30. The distance measuring sensor 20 is attached to the holding member 30 with the optical axes of the light source and the light detector aligned in the same direction. As a result, the cameras 10A and 10B and the distance measuring sensor 20 are held by the holding member 30 with their respective optical axes appropriately adjusted.

[0039] As shown in FIGS. 8A and 8B, the lenses 21A and 21B are each formed by a combination of a plurality of lens elements. The lenses 21A and 21B are screwed into the threaded holes 24A and 24B formed in the base members 22A and 22B, and are fixed so that the imaging surfaces of the image sensors 23A and 23B are substantially flush.

[0040] The lenses 21A and 21B are, for example, retrofocus type lenses formed by combining a plurality of lens elements, and each lens element is housed in a lens cell 211 with their axes aligned. The lens cell 211 is formed with a fitting portion 62 for positioning the optical axis, a threaded portion 63 for screwing the lens 21 in the optical axis direction with a predetermined axial force, and a locking portion 64. Generally, in a wide-angle lens such as a retrofocus lens, the focal length is short, and as shown in the figure, the principal point position 65 of the optical system is biased toward the image sensor 23 side. Even in such a retrofocus type arrangement, it is desirable that the principal point position 65 does not fluctuate due to displacement or posture change of each lens element inside the threaded hole 24.

[0041] In the embodiment, by providing the fitting portion 62 at the end of the lens cell 211 closest to the image sensor 23, the lens 21 is fixed at least in the vicinity of the principal point position 65 of the optical system, and the parallelism of each lens element with respect to the optical axis is maintained.

[0042] The screwing portion 63 that screws into the screw hole 24 is provided on the subject side (the side farther from the image sensor 23) than the fitting portion 62, and the locking portion 64 is further provided on the subject side. When the lens 21 is screwed into the screw hole 24, the locking portion 64 abuts against the base member 22 with a predetermined axial force, and the lens 21 can be positioned with respect to the image sensor 23. Even in the case of expansion and contraction due to temperature change, creep due to aging deterioration, etc., an axial force that can withstand vibration and impact is maintained.

[0043] Since the lenses 21A and 21B are respectively constrained by the fitting portions 62A and 62B, when the temperature rises, for example, they move in a direction away from the image sensors 23A and 23B due to thermal expansion. On the other hand, the principal point position 65 moves so as to approach the image sensor 23 relative to the screwing portion 63. Thereby, the variation in the distance between the principal point position 65 and the image sensor 23 can be reduced, and the change in imaging characteristics due to temperature change can be suppressed.

[0044] The printed circuit boards 25A and 25B on which the image sensors 23A and 23B are mounted are alignment-adjusted so that the imaging characteristics become appropriate with respect to the lenses 21A and 21B fixed to the base members 22A and 22B, respectively. In the alignment adjustment, alignment adjustment is performed for six elements: the position in the optical axis (Z) direction, the coordinates in the XY plane orthogonal to the optical axis, the tilt component α around the x-axis, the tilt component β around the y-axis, and the tilt component γ around the z-axis. Thereafter, while maintaining the aligned position and posture, UV light is irradiated to the ultraviolet (UV) curable adhesive loaded in the gap between the mounting surfaces 222 (see FIG. 6) formed on the base members 22A and 22B and the printed circuit boards 25A and 25B to cure it.

[0045] By this adhesive curing, the printed circuit boards 25A and 25B that have been alignment-adjusted are temporarily fixed, and the position and posture of the imaging surfaces of the image sensors 23A and 23B with respect to the lenses 21A and 21B are maintained.

[0046] On one hand, a thermosetting adhesive is pre-applied to the mounting surfaces 221 (see FIG. 6) of the base members 22A and 22B. The cameras 10A and 10B after alignment adjustment are placed into a heating furnace to cure the thermosetting adhesive. Thereby, even when left in a high-temperature environment assuming hot weather, the adhesive strength between the base members 22A, 22B and the printed circuit boards 25A, 25B is maintained.

[0047] In the embodiment, as the UV curable adhesive, an adhesive mainly composed of an acrylic resin or a urethane resin, which is relatively resistant to impact and tension and has high flexibility, is used. As the thermosetting adhesive, an adhesive mainly composed of an epoxy resin with high heat resistance, hardness, and toughness is used. By providing two types of mounting surfaces 221 and 222 on the base member 22 to separate the application positions and arranging the highly flexible UV curable adhesive on the outside, peeling due to shear stress caused by the difference in the coefficient of thermal expansion between the base member 22 and the printed circuit board 25 is prevented.

[0048] For the thermosetting adhesive, it is preferable to select an adhesive with a higher intermolecular cohesive force, that is, a Young's modulus (elastic modulus) equal to or higher than that of the UV curable adhesive. The UV curable adhesive and the thermosetting adhesive may have the same main agent. For example, by using an epoxy resin as the main agent and selecting a curing accelerator to be blended, either the UV curable type or the thermosetting type may be properly used according to the design requirements of the process.

[0049] Generally, since the UV curable adhesive cures by short-time UV irradiation, it is effective for ensuring the tact time of the assembly process, but there may be cases where the heat resistance is insufficient. Although the thermosetting adhesive has high heat resistance, it takes a long time (more than 1 hour) to completely cure, so it is difficult to incorporate it into the alignment adjustment process.

[0050] Therefore, in the embodiment, after the alignment adjustment is completed, first, temporary fixing is performed with a UV-curable adhesive. Then, the process design is carried out such that a plurality of adjusted semi-finished products are collectively put into a heating furnace to perform a batch process of curing the thermosetting adhesive. Thereby, even if the mounting surface 222 of the base member 22 is a portion where the energy of the UV irradiation light hardly reaches, the final adhesive force is ensured.

[0051] Also, as shown in FIG. 6, the mounting surface 221 of the base member 22 surrounds the periphery of the image sensor 23, and by adhering and sealing the printed circuit board 25 and the mounting surface 221 with a thermosetting adhesive, the intrusion of dust and water droplets into the camera 10 is prevented.

[0052] In the embodiment, an example of using both a UV-curable adhesive and a thermosetting adhesive is shown for shortening the tact time, but it is not limited to this method. For example, a method may be used in which hot air is applied to the thermosetting adhesive for a short time to temporarily cure it to such an extent that the alignment adjustment does not shift, and then it is collectively put into a heating furnace to be completely cured. Even the latter method has little influence on the tact time, and it is not always necessary to use a UV-curable adhesive.

[0053] In order to ensure the mechanical strength and reliability of the joining, if a process capable of temporarily curing in a short time can be added prior to complete curing, it is not limited to a UV-curable type or a thermosetting type, and any combination of adhesives may be used.

[0054] The cameras 10A and 10B are individually alignment-adjusted, and after the thermosetting adhesive is completely cured, a monocular inspection is performed and they are attached to the holding member 30. The cylinders 223 of the base members 22A and 22B formed coaxially with the optical axes of the lenses 21A and 21B are inserted into the fitting holes 13A and 13B of the holding member 30, and by abutting the outer surfaces of the base members 22A and 22B against the locking portions 29 (see FIG. 7) in the rotational direction, the position and orientation around the optical axis are determined.

[0055] In the XY plane orthogonal to the optical axes of the lenses 21A and 21B, the front surfaces of the base members 22A and 22B are abutted against the contact surface 30a of the holding member 30, and the cameras 10A and 10B are fixed to the holding member 30 by screw fastening. This contact surface 30a functions as a positioning means common to the cameras 10A and 10B. Also during the above-described alignment adjustment, the arrangement of the printed circuit boards 25 with respect to the base member 22 can be adjusted with reference to the contact surface 30a.

[0056] In the embodiment, the cameras 10A and 10B are configured to be fixed to the holding member 30 after being individually assembled to the base members 22A and 22B in advance. However, without using the base members 22A and 22B, screw holes 24, mounting surfaces 221, and 222 may be provided in the holding member 30, and the lenses 21A and 21B and the printed circuit boards 25A and 25B may be directly assembled to the holding member 30.

[0057] On the front surface of the upper case 15 of the housing case 101, a camera opening 16 and a distance measuring opening 17 are formed, and on the back surface of the lower case 40, an opening 43 for a cable wiring 400 (see FIG. 8A) serving as an external interface is formed. The upper case 15 and the lower case 40 have a joining surface (or dividing surface) 42 that is inclined obliquely with respect to the XZ plane (see FIG. 1) so that functions are shared on the front side and the back side of the housing case 101. The upper case 15 and the lower case 40 are joined flush at the joining surface 42.

[0058] The upper case 15 and the lower case 40 each have the shape of a triangular prism. The upper case 15 constitutes the front surface and the upper surface of the housing case 101, and the lower case 40 constitutes the rear surface and the bottom surface of the housing case 101. The side surface of the upper case 15 is a triangle with the hypotenuse facing downward, and the side surface of the lower case 40 is a triangle with the hypotenuse facing upward. By joining the upper case 15 and the lower case 40 with a joining surface 42 that is inclined obliquely with respect to a plane horizontal to the upper surface of the upper case 15 or the bottom surface of the lower case 40, a rectangular housing case 101 is assembled. By making each of the upper case 15 and the lower case 40 have the shape of a triangular prism obtained by obliquely dividing a rectangular parallelepiped case, the number of dividing surfaces can be reduced, and the waterproof effect can be enhanced. Also, the manufacturing and assembly of the housing case 101 are easy.

[0059] By sandwiching a sheet-like sealing member 44 made of silicone rubber between the upper case 15 and the lower case 40 and connecting them with the joining surface 42, the intrusion of dust and water droplets into the stereo camera device 100 is prevented. In the embodiment, in order to suppress the heat conduction between the upper case 15 and the lower case 40, a configuration is adopted in which the upper case 15 and the lower case 40 do not come into direct contact using the sheet-like sealing member 44, but it is not limited to this example. A general sealing method such as forming a groove in one of the upper case 15 and the lower case 40 and fitting a packing may be used.

[0060] The cameras 10A, 10B and the distance measuring sensor 20 are arranged on the upper case 15. As an example, a holding member 30 to which the cameras 10A, 10B and the distance measuring sensor 20 are assembled abuts on the inner wall of the front surface of the upper case 15 in the XY plane orthogonal to the optical axis and is screwed. The attachment and detachment of the holding member 30 to the upper case 15 are enabled while maintaining the optical positional relationship between the cameras 10A and 10B and the distance measuring sensor 20. Thereby, when accommodating the cameras 10A, 10B, and the distance measuring sensor 20 in the housing case 101, it is not necessary to individually readjust the arrangement of each optical device, and the distance measuring performance can be guaranteed.

[0061] Even if an external impact is applied to the housing case 101, by providing the holding member 30 that is separate from the upper case 15, it is possible to suppress changes in the relative positional relationship, posture, etc. among the cameras 10A, 10B, and the distance measurement sensor 20. The camera openings 16A, 16B, and the distance measurement opening 17 formed in the upper case 15 may be formed in a diffused shape (zigzag) or matte black painted in order to prevent internal surface reflection, avoid strong sunlight, and suppress the occurrence of detection failures due to external disturbance light.

[0062] A cover glass 102 that covers the camera openings 16A, 16B and the distance measurement opening 17 is attached to the front surface of the upper case 15. The outer periphery of the cover glass 102 may be sealed with a sealing material. As described above, the cover glass 18 is composed of a single plate glass. Even when the cameras 10A, 10B, and the distance measurement sensor 20 are attached and removed in an integrated state, differences in distortion (aberration) generated by each camera 10 and deviations in the focus position due to variations in the thickness, warpage, etc. of the cover glass 18 are suppressed.

[0063] The printed circuit board 41 to which the electronic circuit 50 is attached is disposed in the lower case 40. As an example, the printed circuit board 41 on which the electronic circuit 50 is mounted is disposed on the inner bottom surface of the lower case 40. By removing the lower case 40 from the upper case 15, connection to the cable wiring 400, maintenance, etc. are performed.

[0064] The printed circuit boards 25A, 25B, and the printed circuit board 31 held by the upper case 15 and the printed circuit board 41 held by the lower case 40 are wire-connected by a flexible printed circuit (FPC) cable 48 when the upper case 15 and the lower case 40 are combined. The FPC cable 48 is folded and housed inside the housing case 101.

[0065] The cable wiring 400 serving as an external interface includes a relay connector 45, a waterproof packing 46, and a lock nut 47. The threaded portion of the relay connector 45 is inserted from the outside through the opening 43 formed in the back surface of the lower case 40 with the waterproof packing 46 sandwiched therebetween, and the lock nut 47 is tightened from the inside, thereby attaching the cable wiring 400. With this configuration of the cable wiring 400, intrusion of water, oil, dust, etc. into the stereo camera device 100 is prevented.

[0066] In order to improve the detection resolution, it is necessary to increase the number of pixels of the imaging device. As the number of pixels of the imaging device increases, the amount of electronic information processing inevitably increases, and the heat generation by the electronic circuit 50 also increases.

[0067] In the embodiment, a heat dissipation path is provided in the lower case 40, and the heat generated in the printed circuit board 41 on which the electronic circuit 50 formed of an LSIC, an FPGA, etc. is mounted is released to the outside from the lower case 40. Further, as described above, the printed circuit board 25 on which the image sensor 23 is mounted is joined to the base member 22 with a heat-curing type or UV-curing type adhesive with poor heat transfer, so the heat generated in the printed circuit board 25 is mainly transferred to the printed circuit board 41 via the FPC cable 48. For example, the ground patterns of the FPC cable 48 and the printed circuit board 41 may be laminated by solid wiring to efficiently transfer heat to the lower case 40.

[0068] The electronic circuit 50 mounted on the printed circuit board 41 may be brought into close contact with the lower case 40 with heat dissipation grease. The heat dissipation grease is a paste-like resin in which a powder with good thermal conductivity such as alumina is blended in silicone oil. By applying the heat dissipation grease to the pedestal surface formed in the lower case 40 and assembling the printed circuit board 41 to the lower case 40, the electronic circuit 50 is brought into close contact with the lower case 40.

[0069] The area of the pedestal of the lower case 40 to which the heat-dissipating grease is applied is determined based on the heat generation amount (power consumption) of heat sources such as LSI and FPGA, and the thermal resistance of the heat-dissipating grease. A sheet-like silicone pad or the like may be used instead of the heat-dissipating grease. In the embodiment, when the printed circuit board 41 is assembled, a paste-like heat transfer material that is in an uncured state and spreads and adheres flexibly according to the gap between the printed circuit board 41 and the pedestal surface is used, so that an overload due to pressing is not applied to the printed circuit board 41.

[0070] On the outer surface of the lower case 40, heat dissipation fins 401 for expanding the surface area are formed. The heat dissipation fins 401 and the outer surface of the lower case 40 may be further improved in heat dissipation performance by performing surface treatment such as black painting. A heat circuit is formed so that the heat transferred to the lower case 40 is efficiently radiated to the outside air and does not accumulate heat by natural convection.

[0071] As described above, by integrating the optical system including the camera in the upper case 15 and arranging the electronic circuit 50 in the lower case 40, the imaging function and the arithmetic processing function are shared. By sandwiching the seal member 44 having a higher thermal resistance than either the upper case 15 or the lower case 40 between the upper case 15 and the lower case 40, the heat generated in the printed circuit board 41 on which the electronic circuit 50 is mounted is dissipated from the lower case 40. The seal member 44 with low thermal resistance serves as a buffer material, and can suppress the heat transfer from the lower case 40 to the upper case 15. With this configuration, the influence of heat on the base members 22A and 22B that hold the lenses 21A and 21B is minimized, and the positions and postures of the lenses 21A and 21B are stably held over a long period.

[0072] The stereo camera device 100 can be directly mounted on a frame that constitutes a head guard of, for example, a forklift or a hydraulic shovel. In the embodiment, a mounting portion 49 for a vehicle is provided on the upper case 15. When the stereo camera device 100 is mounted on a vehicle, the ranging performance by the cameras 10A and 10B can be maintained by optically adjusting the visual field.

[0073] When the stereo camera device 100 is mounted on a forklift or a hydraulic excavator, it is necessary to estimate not only the contact distance between the vehicle and the object, but also the contact distance between the edge of the load and the object. In a system that equips the stereo camera device 100 in the front and the rear respectively to enable 360-degree detection corresponding to the rotation operation, it is necessary to ensure the continuity of the field of view. Therefore, when mounting the stereo camera device 100 on a vehicle, it is desirable that the object recognition function can be adjusted according to the on-site situation. When component replacement is required due to updates of the object recognition function, malfunctions of the electronic circuit 50, aging deterioration of the seal member 44, etc., the lower case 40 can be removed while the upper case 15 holding the cameras 10A and 10B is fixed to the vehicle, and maintenance work can be performed. The stereo camera device 100 has a configuration suitable for vehicle mounting and can maintain the ranging performance. The stereo camera device 100 is suitable for mounting not only on vehicles but also on moving bodies such as flying objects like drones and ships.

[0074] Since the cameras 10A and 10B and the ranging sensor 20 are integrally held by a holding member 30 different from the housing case 101, the positional relationship between the cameras 10A and 10B and the ranging sensor 20 is accurately maintained. If optical devices such as the cameras 10A, 10B, and the ranging sensor 20 are directly attached to the housing case 101, the influence of vibration and heat on the optical system becomes large. However, with the configuration of the embodiment, the influence of the environment on the imaging system is minimized and the ranging accuracy is maintained. Since the holding member 30 is fixed to the upper case 15, even when the lower case 40 is removed, the positional relationship between the imaging system including the cameras 10A, 10B and the ranging sensor 20 and the vehicle is maintained. Note that if the ranging sensor 20 and the cameras 10A and 10B are maintained in a predetermined positional relationship, they do not necessarily have to be integrally held by the holding member 30. For example, the ranging sensor 20 and the cameras 10A and 10B can be attached to separate holding members, and the holding members can be combined or connected so that the positional relationship between the ranging sensor 20 and the cameras 10A and 10B is maintained.

[0075] In the stereo camera device 100, deterioration of the distance measurement performance is suppressed even in an outdoor environment. When the stereo camera device 100 is attached to a moving body such as a vehicle or a drone, the distance is calculated from the parallax regardless of the moving speed or direction of the moving body, so that object detection and distance estimation can be accurately performed in the short-distance region. Further, using a retrofocus lens, an operator and a material can be discriminated in a wide-angle near field of a viewing angle of 120° or more.

Explanation of Signs

[0076] 10A, 10B cameras 13 fitting holes (positioning means) 15 upper case (first case member) 16A, 16B camera openings 17 distance measurement opening 20 distance measurement sensor 22 base member 25, 25A, 25B, 31, 41 printed circuit boards 30 holding member 30a contact surface 40 lower case (second case member) 42 bonding surface 43 opening (for external interface) 44 sealing member 49 mounting portion 50 electronic circuit 50A control unit 51 image processing unit 52 parallax calculation unit 53 calibration calculation unit 54 distance calculation unit 55 data input unit 100 stereo camera device 101 housing case (case) 102 cover glass 400 cable wiring (external interface) 401 heat dissipation fins

Prior Art Documents

Patent Documents

[0077] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-189324 [Patent Document 2] Japanese Patent Application Laid-Open No. 2012-167944 [Patent Document 3] Japanese Patent Application Laid-Open No. 11-325890

Claims

1. A plurality of cameras, a distance measuring sensor for measuring the distance to an object, a holding member for aligning and holding the distance measuring sensor and the optical axis directions of the plurality of cameras, an electronic circuit for calibrating the image distance measurement by the plurality of cameras based on the measured distance of the distance measuring sensor, a case as a protective member for housing the plurality of cameras, the distance measuring sensor, and the electronic circuit held by the holding member, and having in a state where the distance measuring sensor and the plurality of cameras are held by the holding member, the electronic circuit uses the difference in the optical axis direction of the plurality of cameras between the optical centers of the plurality of cameras and the distance measuring origin of the distance measuring sensor to calibrate the image distance measurement by the plurality of cameras based on the measured distance of the distance measuring sensor, the holding member is attached to the inner wall surface of the case, a stereo camera device.

2. The holding member has a first contact surface that abuts the plurality of cameras and the distance measuring sensor within a plane orthogonal to the optical axes of the plurality of cameras and the distance measuring sensor in a state where the distance measuring sensor and the plurality of cameras are held, and a second contact surface that is the surface on the back side of the first contact surface and is attached to the inner wall surface of the case. The stereo camera device according to Claim 1.

3. The holding member is detachable from the case that houses the electronic circuit in a state where the distance measuring sensor and the plurality of cameras are held. The stereo camera device according to Claim 1 or 2.

4. The case is formed of a first case and a second case that is detachable from the first case, and the first case has a mounting portion to a moving body. The stereo camera device according to any one of Claims 1 to 3.

5. The holding member is detachably attached to the first case. The stereo camera device according to Claim 4.

6. The second case is detachable from the first case in a state where the first case is attached to the moving body. The stereo camera device according to Claim 4 or 5.

7. In each of the plurality of cameras, the optical axis of the optical component included in the camera and the tilt angle with the optical axis as the rotation axis are adjusted. The stereo camera device according to any one of Claims 1 to 6.

8. The distance measuring sensor has a light source and a light detector, and the optical axes of the light source and the light detector are aligned and held by the holding member. The stereo camera device according to any one of Claims 1 to 7.

9. The optical centers of the plurality of cameras and the distance measurement origin of the distance measurement sensor are located in a plane orthogonal to the optical axis direction of the plurality of cameras. The stereo camera device according to any one of claims 1 to 8.

10. The distance measurement sensor is integrally held with the plurality of cameras by the holding member. The electronic circuit calculates the distance to the object based on the image data captured by the plurality of cameras, and corrects the calculated distance based on the distance information obtained by the distance measurement sensor. The stereo camera device according to any one of claims 1 to 9.

Images