Image acquisition device, robot and robot image acquisition method

Abstract

The present invention discloses an image acquisition device, a robot and a robot image acquisition method, comprising a main body, the main body comprising a base, a first camera assembly and a second camera assembly, each camera assembly group comprising a first camera and a second camera, the two cameras of each camera assembly group realizing the function of a binocular camera; the angle between the central axis of the first camera of the first camera assembly and the central axis of the first camera of the second camera assembly is a set angle, and the angle between the central axis of the first camera of the first camera assembly and the central axis of the first camera of the second camera assembly is equal to the angle between the central axis of the second camera of the first camera assembly and the central axis of the second camera of the second camera assembly. A binocular application with a larger angle is realized by setting multiple cameras in a specific structure, thereby acquiring more environmental data, thereby achieving a better obstacle avoidance effect.

Description

Technical Field

[0001] The invention relates to the technical field of intelligent robots, and in particular to an image acquisition device, a robot and a robot image acquisition method. Background Art

[0002] At present, there is a strong demand for positioning technology and obstacle avoidance technology for indoor robots. Positioning is used to plan the path, and obstacle avoidance is used to avoid obstacles that hinder the operation of the robot, making the robot move more intelligently. When performing positioning and obstacle avoidance, the robot hopes to be able to perceive a larger range to achieve a better obstacle avoidance effect. The existing binocular camera has a small perception range and cannot obtain more environmental data, and the obstacle avoidance effect is poor. Summary of the invention

[0003] In order to solve the above technical defects, the technical solution of the present invention discloses an image acquisition device, a robot and a robot image acquisition method. The technical solution of the present application realizes a binocular application with a larger angle by using multiple cameras arranged in a specific structure, thereby acquiring more environmental data, thereby achieving a better obstacle avoidance effect. The specific technical solution is as follows:

[0004] An image acquisition device includes a main body, wherein the main body includes a base, a first camera assembly and a second camera assembly, each group of camera assemblies includes a first camera and a second camera, and the first camera and the second camera of each group of camera assemblies are combined to realize the function of a binocular camera; the second camera of the first camera assembly is arranged between the first camera and the second camera of the second camera assembly, and the first camera of the second camera assembly is arranged between the first camera and the second camera of the first camera assembly; the angle between the central axis of the first camera of the first camera assembly and the central axis of the first camera of the second camera assembly is a set angle, and the angle between the central axis of the first camera of the first camera assembly and the central axis of the first camera of the second camera assembly is equal to the angle between the central axis of the second camera of the first camera assembly and the central axis of the second camera of the second camera assembly.

[0005] Furthermore, the four cameras of the two groups of camera assemblies are arranged on the same horizontal line, and symmetrical triangular protrusions are respectively provided on both sides of a plane of the base. The first camera of the first camera assembly and the first camera of the second camera assembly are respectively arranged on the inclined surface of the triangular protrusion on one side of the base, and the second camera of the first camera assembly and the second camera of the second camera assembly are respectively arranged on the inclined surface of the triangular protrusion on the other side of the base.

[0006] Furthermore, the central axis of the first camera and the central axis of the second camera of each group of camera assemblies are parallel, the distance between the first camera and the second camera of the first camera assembly in the central axis direction is equal to the distance between the first camera and the second camera of the second camera assembly in the central axis direction, and the vertical distance between the central axis of the first camera of the first camera assembly and the central axis of the second camera is equal to the vertical distance between the central axis of the first camera of the second camera assembly and the central axis.

[0007] Furthermore, the set angle is any angle between 0° and 90°, and the set angle makes the viewing angle of the first camera of the first camera assembly intersect with the viewing angle of the first camera of the second camera assembly.

[0008] An image acquisition device includes a main body, the main body includes a base, a first camera assembly, a second camera assembly and a third camera assembly, each group of camera assemblies includes a first camera and a second camera, and the first camera and the second camera of each group of camera assemblies are combined to realize the function of a binocular camera; the first camera of the second camera assembly and the first camera of the third camera assembly are arranged between the first camera and the second camera of the first camera assembly; the angle between the central axis of the first camera of the first camera assembly and the central axis of the first camera of the second camera assembly is a set angle, and the angle between the central axis of the first camera of the first camera assembly and the central axis of the first camera of the second camera assembly is equal to the angle between the central axis of the first camera of the second camera assembly and the central axis of the first camera of the third camera assembly.

[0009] Furthermore, the six cameras of the three camera assemblies are arranged on the same horizontal line, and symmetrical isosceles trapezoidal protrusions are respectively provided on both sides of a plane of the base, the first camera of the first camera assembly and the first camera of the third camera assembly are respectively arranged on the two side surfaces of the isosceles trapezoidal protrusion on one side of the base, the first camera of the second camera assembly is arranged on the top of the isosceles trapezoidal protrusion, the second camera of the first camera assembly and the second camera of the third camera assembly are respectively arranged on the two side surfaces of the isosceles trapezoidal protrusion on the other side of the base, and the second camera of the second camera assembly is arranged on the top of the isosceles trapezoidal protrusion.

[0010] Furthermore, the set angle is any angle between 0° and 90°, and the set angle makes the viewing angle of the first camera of the first camera assembly intersect with the viewing angle of the first camera of the second camera assembly and the viewing angle of the first camera of the second camera assembly intersect with the viewing angle of the first camera of the third camera assembly.

[0011] A robot is provided with the above-mentioned image acquisition device.

[0012] A robot image data acquisition method, the robot adopts the above-mentioned image acquisition device to execute the robot image data acquisition method, the method comprises the following steps: S1: the robot acquires environmental images through a first camera component and a second camera component respectively, each camera component realizes the function of a binocular camera to acquire depth information from the environmental image; S2: the robot fuses the depth information acquired by the first camera component and the second camera component.

[0013] Furthermore, in step S2, the robot fuses the depth information obtained by the first camera assembly and the second camera assembly, including the following steps: the robot obtains the extrinsic parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly and the extrinsic parameter matrix between the first camera of the second camera assembly and the robot; according to the extrinsic parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly, the point cloud data obtained by the first camera of the first camera assembly is converted into the camera coordinate system of the first camera of the second camera assembly; the robot converts all point cloud data in the camera coordinate system of the first camera of the second camera assembly into the world coordinate system of the robot.

[0014] Furthermore, the robot obtains the extrinsic parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly and the extrinsic parameter matrix between the first camera of the second camera assembly and the robot, including the following steps: the robot obtains the extrinsic parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly and the extrinsic parameter matrix between the first camera of the second camera assembly and the robot through a checkerboard calibration method.

[0015] Furthermore, the depth information acquired by the first camera assembly is point cloud data in a camera coordinate system of a first camera of the first camera assembly, and the depth information acquired by the second camera assembly is point cloud data in a camera coordinate system of a first camera of the second camera assembly.

[0016] A robot image data acquisition method, the robot adopts the above-mentioned image acquisition device to execute the robot image data acquisition method, the method comprising the following steps: S1: the robot acquires environmental images through a first camera assembly, a second camera assembly and a third camera assembly respectively, and each group of camera assemblies realizes the function of a binocular camera to acquire depth information from the environmental image; S2: the robot acquires an external parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly, an external parameter matrix between the first camera of the second camera assembly and the first camera of the third camera assembly, and an external parameter matrix between the first camera of the second camera assembly and the robot; S3: according to the external parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly and the external parameter matrix between the first camera of the second camera assembly and the first camera of the third camera assembly, the point cloud data acquired by the first camera of the first camera assembly and the first camera of the third camera assembly are converted into the camera coordinate system of the first camera of the second camera assembly; S4: the robot converts all point cloud data in the camera coordinate system of the first camera of the second camera assembly into the world coordinate system of the robot.

[0017] Compared with the existing technology, the beneficial effects of the present invention are: the technical solution of the present application realizes binocular application at a larger angle by setting up multiple cameras in a specific way, thereby obtaining more environmental data and achieving a better obstacle avoidance effect; by converting the depth information obtained by multiple groups of camera components into the coordinates of one of the cameras, and then converting it to the robot's own coordinate system, the data calculation speed and data accuracy are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 FIG. 1 is a schematic diagram of the structure of an image acquisition device according to an embodiment of the present invention. Figure 1 ;

[0019] Figure 2 yes Figure 1 A front view of

[0020] Figure 3 FIG. 1 is a schematic diagram of the structure of an image acquisition device according to an embodiment of the present invention. Figure 2 ;

[0021] Figure 4 FIG. 1 is a schematic diagram of a robot image acquisition method according to an embodiment of the present invention. Figure 1 ;

[0022] Figure 5 FIG. 1 is a schematic diagram of a robot image acquisition method according to an embodiment of the present invention. Figure 2 . DETAILED DESCRIPTION

[0023] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions.

[0024] In the description of the present invention, it should be noted that directional words, such as the terms "center", "lateral", "longitudinal", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc., indicating directions and positional relationships are based on the directions or positional relationships shown in the accompanying drawings, which are only for the convenience of narrating the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and cannot be understood as limiting the specific scope of protection of the present invention.

[0025] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features. Therefore, the definition of "first" and "second" features can explicitly or implicitly include one or more of the features. In the description of the present invention, "at least" means one or more, unless otherwise clearly and specifically defined.

[0026] In the present invention, unless otherwise specified and limited, the terms "assemble", "connect", and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can also be a mechanical connection; it can be directly connected, or it can be connected through an intermediate medium, or the two elements can be internally connected. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0027] In the invention, unless otherwise specified and limited, a first feature being “above” or “below” a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being “above”, “below”, and “above” a second feature includes that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being “above”, “below”, and “below” a second feature includes that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0028] The following is a further description of the specific embodiments of the present invention in conjunction with the drawings of the specification, so that the technical solution and its beneficial effects of the present invention are clearer and more explicit. The following description of the embodiments with reference to the drawings is exemplary and intended to explain the present invention, but cannot be understood as limiting the present invention.

[0029] The binocular camera uses the principle of bionics to obtain synchronous exposure images through the calibrated dual cameras, and then calculates the third-dimensional depth information of the obtained 2D image pixels. The binocular camera ranging process: The binocular camera needs to be calibrated to obtain the internal and external parameters and homography matrix of the two cameras; the original image is corrected according to the calibration results, and the two corrected images are located in the same plane and parallel to each other; the two corrected images are matched in pixels; the depth of each pixel is calculated according to the matching results to obtain a depth map.

[0030] like Figure 1 and Figure 2 As shown, an image acquisition device includes a main body, the main body includes a base 101, a first camera assembly and a second camera assembly, and each camera assembly includes a first camera and a second camera. The two cameras of each camera assembly realize the function of a binocular camera; the second camera of the first camera assembly is arranged between the first camera and the second camera of the second camera assembly, and the first camera of the second camera assembly is arranged between the first camera and the second camera of the first camera assembly, that is, arranged in the order of the first camera of the first camera assembly, the first camera of the second camera assembly, the second camera of the first camera assembly, and the second camera of the second camera assembly. The first camera A-1 of the first camera assembly is symmetrically arranged with the first camera B-1 of the second camera assembly, the second camera A-2 of the first camera assembly is symmetrically arranged with the second camera B-2 of the second camera assembly, the first camera A-1 of the first camera assembly is symmetrically arranged with the second camera B-2 of the second camera assembly, and the second camera A-2 of the first camera assembly is symmetrically arranged with the first camera B-1 of the second camera assembly. That is, the first camera assembly includes a first camera A-1 and a second camera A-2, which are combined to realize the function of a binocular camera, and the second camera assembly includes a first camera B-1 and a second camera B-2, which are combined to realize the function of a binocular camera. The viewing angle ranges of the two cameras in the same group are the same, and the viewing angle ranges between cameras in different groups can be the same or different. If the viewing angle ranges between cameras in different groups are the same, it is convenient to convert the data obtained by the first camera of the first camera assembly into the coordinate system of the first camera of the second camera assembly. If the viewing angle ranges between cameras in different groups are different, data within different viewing angle ranges can be obtained, which has a better data acquisition effect.

[0031] As one of the embodiments, symmetrical triangular protrusions are respectively provided on both sides of one plane of the base 101, and the triangular protrusion has two inclined surfaces. The first camera A-1 of the first camera assembly and the first camera B-1 of the second camera assembly are respectively arranged on the inclined surfaces of the triangular protrusion on one side of the base, and the second camera A-2 of the first camera assembly and the second camera B-2 of the second camera assembly are respectively arranged on the inclined surfaces of the triangular protrusion on the other side of the base. The angle between the central axis of the first camera A-1 of the first camera assembly and the central axis of the first camera B-1 of the second camera assembly is equal to the angle between the central axis of the second camera A-2 of the first camera assembly and the central axis of the second camera B-2 of the second camera assembly, and this angle is a set angle. The set angle is any angle between 0° and 90°, and the set angle makes the viewing angle of the first camera of the first camera assembly intersect with the viewing angle of the first camera of the second camera assembly, and the viewing angle of the first camera of the second camera assembly intersects with the viewing angle of the first camera of the third camera assembly. That is, the set angle can be 0°, 15°, 30°, 45°, 60°, 75° or 90°, etc. When setting the set value, the viewing angle of the first camera of the first camera assembly can be made to intersect with the viewing angle of the first camera of the second camera assembly. The viewing angle of the first camera of the first camera assembly and the viewing angle of the first camera of the second camera assembly may not intersect, but when the viewing angle of the first camera of the first camera assembly and the viewing angle of the first camera of the second camera assembly intersect, the detection effect of the image acquisition device is better. The function of the triangular protrusion is to place the cameras of the first camera assembly and the second camera assembly, and it will not block the viewing angle of the camera, so the triangular protrusion can also be replaced with an isosceles trapezoid, a hemisphere or a rectangle.

[0032] As one embodiment, the central axis of the first camera and the central axis of the second camera of each camera assembly are parallel, the distance between the first camera A-1 and the second camera A-2 of the first camera assembly in the central axis direction is equal to the distance between the first camera B-1 and the second camera of the second camera assembly in the central axis direction, the distance between the first camera A-1 and the second camera A-2 of the first camera assembly in the central axis direction is H_d, the vertical distance between the central axis of the first camera A-1 and the central axis of the second camera A-2 of the first camera assembly is equal to the vertical distance between the central axis of the first camera B-1 and the central axis of the second camera B-2 of the second camera assembly, and the distance between the central axis of the first camera A-1 and the central axis of the second camera A-2 of the first camera assembly is S_d. The specific values ​​of H_d and S_d are not limited and can be adjusted accordingly according to actual conditions. The values ​​of H_d and S_d are mainly used to obtain the extrinsic parameter matrix between the first camera A-1 of the first camera assembly and the first camera B-1 of the second camera assembly and the extrinsic parameter matrix between the first camera B-1 of the second camera assembly and the robot, so that the robot can convert the point cloud data acquired by the first camera A-1 of the first camera assembly into the camera coordinate system of the first camera B-1 of the second camera assembly according to the extrinsic parameter matrix between the first camera A-1 of the first camera assembly and the first camera B-1 of the second camera assembly, and the robot can convert all point cloud data in the camera coordinate system of the first camera B-1 of the second camera assembly into the world coordinate system of the robot.

[0033] As one embodiment, the four cameras of the two camera assemblies are arranged on the same horizontal line. The base 101 includes a processing unit, which is used to receive and process the depth image information obtained by the two camera assemblies. The base also includes a PCB board, and the processing unit is arranged on the PCB board. The PCB board is fixedly connected to the base to form a module, which improves the integrity of the product and facilitates assembly and after-sales maintenance. The base 101 includes at least one of a serial interface, a parallel interface, an infrared interface, a USB interface or an IEEE1394 (FireWire) interface. The shape of the base is not limited, and can be an ellipse or a rectangle, etc.

[0034] As one embodiment, the image acquisition device described in the technical solution of the present application can be set up with 4 low-cost small-angle cameras, which can realize the application of large-angle binocular cameras, reduce production costs, and facilitate installation on robots; and large-angle cameras are relatively expensive, and have relatively large distortion and high computing power requirements. Small-angle cameras have small distortion, and when matching binocular cameras, the search range can be limited to a relatively narrow area, and the computing power requirements are relatively low.

[0035] A robot is provided with the above-mentioned image acquisition device.

[0036] like Figure 4 As shown, a robot image acquisition method is provided, wherein the robot adopts the above-mentioned image acquisition device to execute the robot image acquisition method, and the method comprises the following steps: S1: the robot acquires an environmental image through a first camera component and a second camera component respectively, and each group of camera components realizes the function of a binocular camera to acquire depth information from the environmental image; S2: the robot fuses the depth information acquired by the first camera component and the second camera component.

[0037] As one of the embodiments, in step S2, the robot fuses the depth information obtained by the first camera assembly and the second camera assembly, including the following steps: the robot obtains the external parameter matrix between the first camera A-1 of the first camera assembly and the first camera B-1 of the second camera assembly and the external parameter matrix between the first camera B-1 of the second camera assembly and the robot; according to the external parameter matrix between the first camera A-1 of the first camera assembly and the first camera B-1 of the second camera assembly, the point cloud data obtained by the first camera A-1 of the first camera assembly is converted to the camera coordinate system of the first camera B-1 of the second camera assembly, and the robot converts all the point cloud data in the camera coordinate system of the first camera B-1 of the second camera assembly to the world coordinate system of the robot. When performing the coordinate system conversion, the coordinates of the point cloud data in the current coordinate system are multiplied by the corresponding external parameter matrix to obtain the coordinates of the point cloud data after the coordinate system is transformed. The meaning of each element of the internal parameter matrix: the physical size dx and dy of a pixel, the focal length f, the distortion factor r of the physical coordinates of the image, and the vertical and horizontal offsets u and v (in pixels) of the image origin relative to the optical center imaging point. Extrinsic matrix: the rotation R and translation T matrix of the world coordinate system to the camera coordinate system. World coordinate system: the coordinate system of the three-dimensional world defined by the user, introduced to describe the position of the target in the real world, with the unit of m. Camera coordinate system: the coordinate system established on the camera, defined to describe the position of the object from the perspective of the camera, as the intermediate link between the world coordinate system and the image / pixel coordinate system, with the unit of m.

[0038] As one embodiment, the robot obtains the external parameter matrix between the first camera A-1 of the first camera assembly and the first camera B-1 of the second camera assembly and the external parameter matrix between the first camera B-1 of the second camera assembly and the robot, including the following steps: the robot obtains the external parameter matrix between the first camera A-1 of the first camera assembly and the first camera B-1 of the second camera assembly and the external parameter matrix between the first camera B-1 of the second camera assembly and the robot through a chessboard calibration method. The chessboard calibration method is a commonly used method for calibrating the internal and external parameters of the camera, which will not be repeated here (inventor: Zhang Zhengyou, paper title: "Flexible Camera Calibration By Viewing aPlane From Unknown Orientations", source: International Top Conference ICCV, publication time: 1999).

[0039] As one of the embodiments, the depth information obtained by the first camera component is point cloud data in the camera coordinate system of the first camera A-1 of the first camera component, and the depth information obtained by the second camera component is point cloud data in the camera coordinate system of the first camera B-1 of the second camera component.

[0040] like Figure 3As shown, an image acquisition device includes a main body, the main body includes a base, a first camera assembly, a second camera assembly and a third camera assembly, each group of camera assemblies includes a first camera and a second camera, the first camera and the second camera of each group of camera assemblies are combined to realize the function of a binocular camera, that is, the first camera assembly includes a first camera A-1 and a second camera A-2, which are combined to realize the function of a binocular camera, the second camera assembly includes a first camera B-1 and a second camera B-2, which are combined to realize the function of a binocular camera, and the third camera assembly includes a first camera C-1 and a second camera C-2, which are combined to realize the function of a binocular camera. The first camera B-1 of the second camera assembly and the first camera C-1 of the third camera assembly are arranged between the first camera A-1 and the second camera A-2 of the first camera assembly; the second camera A-2 of the first camera assembly and the second camera B-2 of the second camera assembly are arranged between the first camera C-1 and the second camera C-2 of the third camera assembly, that is, the first camera A-1 of the first camera assembly, the first camera B-1 of the second camera assembly, the first camera C-1 of the third camera assembly, the second camera A-2 of the first camera assembly, the second camera B-2 of the second camera assembly and the second camera C-3 of the third camera assembly are arranged in this order, the first cameras of the three camera assemblies are arranged on one side of the base, and the second cameras of the three camera assemblies are arranged on the other side of the base with the same structure. The angle a between the central axis of the first camera A-1 of the first camera assembly and the central axis of the first camera B-1 of the second camera assembly is a set angle, and the angle a between the central axis of the first camera A-1 of the first camera assembly and the central axis of the first camera B-1 of the second camera assembly is equal to the angle b between the central axis of the first camera B-1 of the second camera assembly and the central axis of the first camera C-1 of the third camera assembly. Figure 3 The image acquisition device can be Figure 2 Add a set of camera components to the image acquisition device to achieve, Figure 3 The image acquisition device has Figure 2 The image acquisition device has the same function and the same partial structure, but this time Figure 3 The viewing angle ratio of the image acquisition device Figure 2 The image acquisition device has a larger viewing angle.

[0041] As one of the embodiments, the six cameras of the three camera assemblies are arranged on the same horizontal line, and symmetrical isosceles trapezoidal protrusions are respectively provided on both sides of a plane of the base, the first camera A-1 of the first camera assembly and the first camera C-1 of the third camera assembly are respectively arranged on the two side surfaces of the isosceles trapezoidal protrusion on one side of the base, the first camera B-1 of the second camera assembly is arranged on the top of the isosceles trapezoidal protrusion, the second camera A-2 of the first camera assembly and the second camera C-2 of the third camera assembly are respectively arranged on the two side surfaces of the isosceles trapezoidal protrusion on the other side of the base, and the second camera B-2 of the second camera assembly is arranged on the top of the isosceles trapezoidal protrusion. The isosceles trapezoidal protrusion is used to place the cameras of the first camera assembly, the second camera assembly and the third camera assembly, and will not block the viewing angle of the camera, so the isosceles trapezoidal protrusion can also be replaced with a hemispherical or rectangular shape.

[0042] As one embodiment, the central axis of the first camera and the central axis of the second camera of each camera assembly are parallel, the distance between the first camera A-1 and the second camera A-2 of the first camera assembly in the central axis direction is equal to the distance between the first camera C-1 and the second camera C-2 of the third camera assembly in the central axis direction, and the vertical distance between the central axis of the first camera A-1 and the central axis of the second camera A-2 of the first camera assembly is equal to the vertical distance between the central axis of the first camera C-1 and the central axis of the second camera C-2 of the third camera assembly. The distance between the first camera B-1 and the second camera B-2 of the second camera assembly on the central axis is zero, and the vertical distance between the central axis of the first camera B-1 and the central axis of the second camera B-2 of the second camera assembly may be equal to or unequal to the vertical distance between the central axis of the first camera A-1 and the central axis of the second camera A-2 of the first camera assembly. The specific values ​​are not limited and can be adjusted accordingly according to actual conditions.

[0043] As one embodiment, the set angle is any angle between 0° and 90°, and the set angle makes the viewing angle of the first camera A-1 of the first camera assembly intersect with the viewing angle of the first camera B-1 of the second camera assembly, and the viewing angle of the first camera B-1 of the second camera assembly intersect with the viewing angle of the first camera C-1 of the third camera assembly. The viewing angle ranges of two cameras in the same group are the same, and the viewing angle ranges of cameras in different groups can be the same or different. If the viewing angle ranges of cameras in different groups are the same, it is convenient to convert the data acquired by the first camera A-1 of the first camera assembly and the first camera C-1 of the third camera assembly into the coordinate system of the first camera B-1 of the second camera assembly. If the viewing angle ranges of cameras in different groups are different, data within different viewing angle ranges can be acquired, which has a better data acquisition effect.

[0044] As one embodiment, the base 101 includes a processing unit, which is used to receive and process the depth image information obtained by the two camera assemblies. The base also includes a PCB board, and the processing unit is arranged on the PCB board. The PCB board is fixedly connected to the base to form a module, which improves the integrity of the product and facilitates assembly and after-sales maintenance. The base 101 includes at least one of a serial interface, a parallel interface, an infrared interface, a USB interface or an IEEE1394 (FireWire) interface. The shape of the base is not limited, and can be an ellipse or a rectangle, etc.

[0045] like Figure 5As shown, a robot image data acquisition method, the robot adopts the above-mentioned image acquisition device to execute the robot image data acquisition method, the method comprising the following steps: S1: the robot acquires the environment image through the first camera component, the second camera component and the third camera component respectively, and each group of camera components realizes the function of a binocular camera to acquire depth information from the environment image; S2: the robot acquires the extrinsic parameter matrix between the first camera of the first camera component and the first camera of the second camera component, the extrinsic parameter matrix between the first camera of the second camera component and the first camera of the third camera component, and the extrinsic parameter matrix between the first camera of the second camera component and the robot; S3: according to the extrinsic parameter matrix between the first camera of the first camera component and the first camera of the second camera component and the extrinsic parameter matrix between the first camera of the second camera component and the first camera of the third camera component, the point cloud data acquired by the first camera of the first camera component and the first camera of the third camera component are converted into the camera coordinate system of the first camera of the second camera component; S4: the robot converts all the point cloud data in the camera coordinate system of the first camera of the second camera component into the world coordinate system of the robot. When performing coordinate system conversion, multiply the coordinates of the point cloud data in the current coordinate system by the corresponding external parameter matrix to obtain the coordinates of the point cloud data after the coordinate system is transformed. The meaning of each element of the internal parameter matrix: the physical size dx and dy of a pixel, the focal length f, the distortion factor r of the image physical coordinates, the vertical and horizontal offsets u and v (in pixels) of the image origin relative to the optical center imaging point. External parameter matrix: the rotation R and translation T matrix of the world coordinate system to the camera coordinate system. World coordinate system: a user-defined coordinate system of the three-dimensional world, introduced to describe the position of the target object in the real world, with units of m. Camera coordinate system: a coordinate system established on the camera, defined to describe the position of the object from the perspective of the camera, as an intermediate link between the world coordinate system and the image / pixel coordinate system, with units of m. When the central axis of the first camera B-1 and the second camera B-2 of the second camera assembly are parallel to the front of the robot, all point cloud data in the camera coordinate system of the first camera B-1 of the second camera assembly can be converted to the world coordinate system of the robot by translation according to the distance between the first camera B-1 of the second camera assembly and the center of the robot; or the camera coordinate system of the first camera B-1 of the second camera assembly can be used as the world coordinate system of the robot.

[0046] As one embodiment, the robot obtains the external parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly, the external parameter matrix between the first camera of the second camera assembly and the first camera of the third camera assembly, and the external parameter matrix between the first camera of the second camera assembly and the robot, including the following steps: the robot obtains the external parameter matrix between the first camera A-1 of the first camera assembly and the first camera B-1 of the second camera assembly and the external parameter matrix between the first camera B-1 of the second camera assembly and the robot by a chessboard calibration method. The chessboard calibration method is a commonly used method for calibrating the internal and external parameters of the camera, which will not be repeated here (inventor: Zhang Zhengyou, paper title: "Flexible Camera Calibration By Viewing a Plane From Unknown Orientations", source: International Top Conference ICCV, published in 1999).

[0047] As one of the embodiments, the depth information obtained by the first camera component is point cloud data in the camera coordinate system of the first camera A-1 of the first camera component, the depth information obtained by the second camera component is point cloud data in the camera coordinate system of the first camera B-1 of the second camera component, and the depth information obtained by the third camera component is point cloud data in the camera coordinate system of the first camera C-1 of the third camera component.

[0048] Compared with the existing technology, the beneficial effects of the present invention are: the technical solution of the present application realizes binocular application at a larger angle by setting up multiple cameras in a specific way, thereby obtaining more environmental data and achieving a better obstacle avoidance effect; by converting the depth information obtained by multiple groups of camera components into the coordinates of one of the cameras, and then converting it to the robot's own coordinate system, the data calculation speed and data accuracy are improved.

[0049] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. An image acquisition device, comprising a main body, characterized in that: The main body includes a base, a first camera assembly and a second camera assembly. Each camera assembly includes a first camera and a second camera, and the first camera and the second camera of each camera assembly are combined to realize the function of a binocular camera; The second camera of the first camera assembly is arranged between the first camera and the second camera of the second camera assembly, and the first camera of the second camera assembly is arranged between the first camera and the second camera of the first camera assembly; The angle between the central axis of the first camera of the first camera assembly and the central axis of the first camera of the second camera assembly is a set angle, and the angle between the central axis of the first camera of the first camera assembly and the central axis of the first camera of the second camera assembly is equal to the angle between the central axis of the second camera of the first camera assembly and the central axis of the second camera of the second camera assembly; The set angle is any angle between 0° and 90°, and the set angle makes the viewing angle of the first camera of the first camera assembly intersect with the viewing angle of the first camera of the second camera assembly.

2. The image acquisition device according to claim 1, characterized in that: The four cameras of the two groups of camera assemblies are arranged on the same horizontal line, and symmetrical triangular protrusions are respectively provided on both sides of a plane of the base. The first camera of the first camera assembly and the first camera of the second camera assembly are respectively arranged on the inclined surface of the triangular protrusion on one side of the base, and the second camera of the first camera assembly and the second camera of the second camera assembly are respectively arranged on the inclined surface of the triangular protrusion on the other side of the base.

3. The image acquisition device according to claim 2, characterized in that: The central axis of the first camera and the central axis of the second camera of each camera assembly are parallel, the distance between the first camera and the second camera of the first camera assembly in the central axis direction is equal to the distance between the first camera and the second camera of the second camera assembly in the central axis direction, and the vertical distance between the central axis of the first camera and the central axis of the second camera of the first camera assembly is equal to the vertical distance between the central axis of the first camera and the central axis of the second camera of the second camera assembly.

4. An image acquisition device, comprising a main body, characterized in that: The main body includes a base, a first camera assembly, a second camera assembly and a third camera assembly, each camera assembly includes a first camera and a second camera, and the first camera and the second camera of each camera assembly are combined to realize the function of a binocular camera; The first camera of the second camera assembly and the first camera of the third camera assembly are arranged between the first camera and the second camera of the first camera assembly; The angle between the central axis of the first camera of the first camera assembly and the central axis of the first camera of the second camera assembly is a set angle, and the angle between the central axis of the first camera of the first camera assembly and the central axis of the first camera of the second camera assembly is equal to the angle between the central axis of the first camera of the second camera assembly and the central axis of the first camera of the third camera assembly; The set angle is any angle between 0° and 90°, and the set angle makes the viewing angle of the first camera of the first camera assembly intersect with the viewing angle of the first camera of the second camera assembly and the viewing angle of the first camera of the second camera assembly intersect with the viewing angle of the first camera of the third camera assembly.

5. The image acquisition device according to claim 4, characterized in that: The six cameras of the three camera assemblies are arranged on the same horizontal line, and symmetrical isosceles trapezoidal protrusions are respectively provided on both sides of a plane of the base. The first camera of the first camera assembly and the first camera of the third camera assembly are respectively arranged on the two side surfaces of the isosceles trapezoidal protrusion on one side of the base, and the first camera of the second camera assembly is arranged on the top of the isosceles trapezoidal protrusion; the second camera of the first camera assembly and the second camera of the third camera assembly are respectively arranged on the two side surfaces of the isosceles trapezoidal protrusion on the other side of the base, and the second camera of the second camera assembly is arranged on the top of the isosceles trapezoidal protrusion.

6. A robot, characterized in that: The robot is provided with the image acquisition device according to any one of claims 1 to 5.

7. A robot image acquisition method, characterized in that: The robot uses the image acquisition device according to any one of claims 1 to 3 to perform a robot image acquisition method, the method comprising the following steps: S1: The robot obtains an environmental image through a first camera component and a second camera component respectively, and each camera component implements the function of a binocular camera to obtain depth information from the environmental image; S2: The robot integrates the depth information obtained by the first camera component and the second camera component.

8. The robot image acquisition method according to claim 7, characterized in that: In step S2, the robot fuses the depth information obtained by the first camera component and the second camera component, including the following steps: The robot obtains an extrinsic parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly and an extrinsic parameter matrix between the first camera of the second camera assembly and the robot; According to the external parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly, the point cloud data acquired by the first camera of the first camera assembly is converted into the camera coordinate system of the first camera of the second camera assembly; The robot converts all point cloud data in the camera coordinate system of the first camera of the second camera assembly into the world coordinate system of the robot.

9. The robot image acquisition method according to claim 8, characterized in that: The robot obtains an external parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly, and an external parameter matrix between the first camera of the second camera assembly and the robot, including the following steps: The robot obtains the external parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly and the external parameter matrix between the first camera of the second camera assembly and the robot through a chessboard calibration method.

10. The robot image acquisition method according to claim 8, characterized in that: The depth information acquired by the first camera assembly is point cloud data in the camera coordinate system of the first camera of the first camera assembly, and the depth information acquired by the second camera assembly is point cloud data in the camera coordinate system of the first camera of the second camera assembly.

11. A robot image acquisition method, characterized in that: The robot uses the image acquisition device according to any one of claims 4 to 5 to perform a robot image acquisition method, the method comprising the following steps: S1: The robot obtains environmental images through the first camera component, the second camera component and the third camera component respectively, and each camera component realizes the function of a binocular camera to obtain depth information from the environmental image; S2: The robot obtains an extrinsic matrix between the first camera of the first camera assembly and the first camera of the second camera assembly, an extrinsic matrix between the first camera of the second camera assembly and the first camera of the third camera assembly, and an extrinsic matrix between the first camera of the second camera assembly and the robot; S3: converting the point cloud data acquired by the first camera of the first camera assembly and the first camera of the third camera assembly into the camera coordinate system of the first camera of the second camera assembly according to the extrinsic parameter matrix between the first camera of the first camera assembly and the first camera of the second camera assembly and the extrinsic parameter matrix between the first camera of the second camera assembly and the first camera of the third camera assembly; S4: The robot converts all point cloud data in the camera coordinate system of the first camera of the second camera assembly into the world coordinate system of the robot.

Images