Camera external parameter calibration method, device and equipment and readable storage medium

Abstract

The invention provides a camera external parameter calibration method, device and equipment and a readable storage medium, and the method comprises the steps: controlling at least one movable object to move based on a preset position set, so as to enable a key point of each movable object to be located at a respective designated position; capturing an environment image corresponding to a first object through a camera on the first object; determining a real position of a calibration point based on a specified position of a key point of the second object and a relative position between the key point and the calibration point of the second object; and determining the external parameters of the camera according to the image position of the calibration point in the environment image and the real position. The flexibility of external parameter determination can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of camera calibration technology, and in particular, to a camera extrinsic parameter calibration method, device, electronic device and computer-readable storage medium. Background Art

[0002] With the development of technology, cameras are increasingly used in computer vision tasks such as recognition and detection. The camera captures the external environment image, then recognizes the environment image, and performs various applications based on the recognition results, such as obstacle detection, distance measurement, etc. In order to accurately process the image, it is necessary to determine the external parameters of the camera.

[0003] Currently, calibration points are pre-set at specific locations in a specific space such as a workshop to construct a calibration field, and then the calibration points in the calibration field are captured by a camera, and then the extrinsic parameters are calculated based on the calibration points.

[0004] However, in the current scheme, the calibration process is based on a specific space and a specific position. The entire external parameter calibration process is heavily dependent on the specific space and specific position, which has great limitations. Summary of the invention

[0005] In view of the above problems, embodiments of the present application are proposed to provide a camera extrinsic parameter calibration method, device, electronic device and computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, an embodiment of the present application discloses a camera extrinsic calibration method, the method comprising:

[0007] Based on a preset position set, controlling at least one movable object to move so that a key point of each movable object is at a respective designated position; the preset position set includes a designated position corresponding to each movable object; and the movable object is provided with a calibration point and a camera;

[0008] Capturing an environment image corresponding to the first object by a camera on the first object; the first object is any movable object; the environment image includes a second object other than the first object;

[0009] Determining the true position of the calibrated point based on the specified position of the key point of the second object and the relative position between the key point and the calibrated point of the second object;

[0010] The external parameters of the camera are determined according to the image position of the calibration point in the environment image and the real position.

[0011] In a second aspect, an embodiment of the present application discloses a camera extrinsic calibration device, comprising:

[0012] An object movement module, used to control the movement of at least one movable object based on a preset position set so that the key points of each movable object are at their respective designated positions; the preset position set includes the designated positions corresponding to each movable object; and the movable object is provided with a calibration point and a camera;

[0013] An image capturing module, configured to capture an environment image corresponding to a first object through a camera on the first object; the first object is any movable object; and the environment image includes a second object other than the first object;

[0014] a calibration position module, configured to determine the true position of the calibration point based on the specified position of the key point of the second object and the relative position between the key point and the calibration point of the second object;

[0015] The external parameter calculation module is used to determine the external parameters of the camera according to the image position of the calibration point in the environment image and the real position.

[0016] In a third aspect, an embodiment of the present application further discloses an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the camera extrinsic parameter calibration method described in the first aspect are implemented.

[0017] In a fourth aspect, an embodiment of the present application further discloses a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the camera extrinsic parameter calibration method as described in the first aspect are implemented.

[0018] In the embodiment of the present application, at least one movable object can be controlled to move based on a preset position set so that the key points of each movable object are at their respective designated positions; calibration points are set on the movable objects; the environment image corresponding to the first object is captured by a camera; the real position of the calibration point is determined based on the designated position of the key point of the second object and the relative position between the key point and the calibration point; the external parameters of the camera are determined according to the image position and the real position of the calibration point in the environment image. Based on the movement control of the movable object, a calibration field with flexible size and position can be constructed, and then the external parameters of the camera on the movable object used to constitute the calibration field can be calibrated, which can be applied to different scenes, has strong terrain adaptability, and is highly convenient, thereby improving the flexibility of the camera external parameter calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1It is a step diagram of a camera extrinsic parameter calibration method provided in an embodiment of the present application;

[0020] Figure 2 is a step diagram of another camera extrinsic parameter calibration method provided in an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of the structure of a movable object provided in an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a preset pattern on the top of a shell provided in an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of a calibration figure and calibration points provided in an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of the movement of a movable object provided in an embodiment of the present application;

[0025] Figure 7 is a top view schematic diagram of the movement of a movable object provided in an embodiment of the present application;

[0026] Figure 8 is a flow chart of external parameter calibration provided in an embodiment of the present application;

[0027] Fig. 9 is a block diagram of a camera extrinsic calibration device provided in an embodiment of the present application;

[0028] Fig.10 It is a block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0030] With the development of technology, cameras are increasingly used in computer vision tasks such as recognition and detection. The camera captures the external environment image, then recognizes the environment image, and performs various applications based on the recognition results, such as obstacle detection, distance measurement, etc. In order to accurately process the image, it is necessary to determine the external parameters of the camera.

[0031] As an important part of daily life, the vehicle experience is becoming more and more intelligent. Camera-based computer vision is also being applied. The vehicle's electronic rearview mirror system (CMS, Camera Monitor System), which is a combination product based on cameras and displays, is used to enhance the driver's visual perception of the vehicle's surroundings and rear, and improve driving safety and comfort. The electronic rearview mirror system collects images through external cameras, and displays them on the display screen inside the car after data processing. At the same time, it can integrate blind spot warnings, obstacle prompts and other functions to enhance the driving experience.

[0032] CMS also needs to be calibrated with external parameters so that it can be applied to intelligent applications based on long-distance accurate distance measurement, such as obstacle detection and lane change reminders, that is, CMS intelligent recognition applications.

[0033] Currently, external parameter calibration is performed in a fixed space based on fixed calibration points. For example, for vehicle CMS, factory calibration is performed, which usually uses workshop calibration.

[0034] Furthermore, using the traditional factory calibration method, the long-distance calibration points are blurred in the image, resulting in inaccurate recognition, making it difficult to ensure accuracy.

[0035] Figure 1 A step diagram of a camera extrinsic calibration method provided in an embodiment of the present application is shown, and the method includes:

[0036] Step 101, based on a preset position set, control at least one movable object to move so that the key points of each movable object are at their respective designated positions; the preset position set includes the designated positions corresponding to each movable object; and the movable object is provided with calibration points and cameras.

[0037] In the embodiment of the present application, the movable object may be a vehicle such as an automated guided vehicle (AGV). AGV is an intelligent vehicle used for automated transportation and handling of goods in an industrial environment. AGV can automatically travel along a preset path to complete the transportation task of materials, and is widely used in manufacturing, logistics warehousing, medical and other industries. The key technologies of AGV include navigation and guidance technology. The navigation methods of AGV are diverse, including magnetic nail navigation, magnetic strip navigation, laser navigation, electromagnetic navigation, ranging navigation, contour navigation, hybrid navigation and optical navigation. Among them, laser navigation has become an advanced navigation method preferred by many AGV manufacturers due to its precise positioning and flexible and changeable paths. AGC may have an omnidirectional wheel chassis, which is composed of a motor, wheels and corresponding electromechanical components. The omnidirectional wheel chassis can move according to control instructions, and the movement mode supports omnidirectional movement. Among them, the omnidirectional wheel chassis logo can perform forward linear motion, lateral linear motion and spin motion.

[0038] The number of movable objects can be multiple, for example, 4, 6, etc. The preset position set includes a designated position corresponding to each movable object. Based on the preset position set, at least one movable object is controlled to move, with the purpose of making a key point in the movable object at a designated position. The key point of the movable object can be the geometric center of the object, the center of the front axle of the vehicle, or the center of the rear axle of the vehicle, which is not specifically limited here.

[0039] The number of calibration points and cameras for each movable object may be multiple, for example, there may be 4 cameras and 12 calibration points. The multiple calibration points and the multiple cameras may be respectively set on different sides of the movable object.

[0040] When each movable object is an AGV, the AGV may include an omni-directional wheel chassis, an electronic hydraulic suspension system, a laser ranging module, calibration points, a camera, and a housing on the top of which a calibration pattern may be installed, etc. The number of calibration points and cameras is not limited. The omni-directional wheel chassis may be used to achieve flexible movement of the AGV, the electronic hydraulic suspension system may be used to adjust the posture of the vehicle, the laser ranging may be used to measure the position of the vehicle and other objects, and the camera may capture environmental images.

[0041] The electronic hydraulic suspension system refers to an electronic control system that can accurately adjust the flow and pressure of the hydraulic oil through the electronic control unit to achieve active control of the AGV suspension system. Through the four hydraulic components and separate control of the system, the upper body of the AGV can be adjusted in pitch, yaw, and height during travel.

[0042] The laser ranging module contains four laser ranging units, which can determine the distance of the target object by emitting a laser beam and measuring the time or phase change of its reflection back. The four laser ranging units contained in the laser ranging module can be respectively set on different sides of the movable object.

[0043] Step 102, capturing an environment image corresponding to the first object through a camera on the first object; the first object is any movable object; the environment image includes a second object other than the first object;

[0044] In the embodiment of the present application, each vehicle may have one or more cameras, and the camera on each vehicle may obtain an environmental image from the perspective of the current vehicle, i.e., the first object. The environmental image may include other vehicles other than the current vehicle, i.e., the second object, and the number of the second object may be at least one. Since the first object is any movable object, multiple vehicles may simultaneously obtain environmental images at the same time, and each vehicle exists in the environmental images captured by the cameras on other vehicles.

[0045] Step 103, determining the real position of the calibration point based on the designated position of the key point of the second object and the relative position between the key point and the calibration point of the second object;

[0046] In an embodiment of the present application, when there is a second object in the environmental image, since each movable object has a corresponding designated position, and the designated position is the key point of the movable object. Therefore, the designated position where the key point of the second object is located can be obtained. In addition, since the key point position of the movable object is fixed relative to the movable object itself, and the calibration point position of the movable object is also fixed relative to the movable object itself, that is, the relative position between the key point and the calibration point of the second object is fixed, then when the key point position is determined, the real position of the calibration point can be clarified, and the real position is the actual position of the calibration point in the physical space, that is, the position of the calibration point in the world coordinate system.

[0047] Step 104, determining the external parameters of the camera according to the image position of the calibration point in the environment image and the real position.

[0048] In the embodiment of the present application, the image position refers to the pixel position of the calibration point in the environment image relative to the entire image. For example, the pixel position of the lower left corner of the image can be used as the origin of the image coordinate system, and then the image coordinate system is constructed, and the image position of the calibration point in the image coordinate system is determined. It is understandable that the environment image can be dedistorted first, and then the image position of the calibration point can be obtained, so that the image position is more accurate.

[0049] The camera's external parameters are calculated based on the image position and the real position, and can be obtained by mathematical methods such as least squares method, singular value decomposition (SVD), etc. The external parameters can be a rotation matrix and a translation vector.

[0050] To calculate the rotation matrix, you can convert the pixel coordinates in the image into normalized plane coordinates by selecting a set of calibration points (usually at least three non-collinear points are required), and use methods such as SVD decomposition to calculate the rotation matrix.

[0051] The translation vector can be calculated by using the selected calibration points and their world coordinates and normalized plane coordinates. The translation vector can be solved by linear or nonlinear least squares method.

[0052] In summary, in the embodiments of the present application, at least one movable object can be controlled to move based on a preset position set so that the key points of each movable object are at their respective designated positions; calibration points are set on the movable objects; the environment image corresponding to the first object is captured by a camera; the real position of the calibration point is determined based on the designated position of the key point of the second object and the relative position between the key point and the calibration point; the external parameters of the camera are determined according to the image position and the real position of the calibration point in the environment image. Based on the movement control of the movable object, a calibration field with flexible size and position can be constructed, and then the external parameters of the camera on the movable object used to constitute the calibration field can be calibrated, which can be applied to different scenes, has strong terrain adaptability and high convenience, thereby improving the flexibility, applicability and convenience of the camera external parameter calibration.

[0053] refer to Figure 2 , which shows a flowchart of the steps of a camera extrinsic parameter calibration method provided in an embodiment of the present application, the method comprising:

[0054] Step 201, based on a preset position set, controlling at least one movable object to move so that a key point of each movable object is at a respective designated position; the preset position set includes a designated position corresponding to each movable object; and the movable object is provided with a calibration point and a camera;

[0055] Step 202: capturing an environment image corresponding to the first object through a camera on the first object; the first object is any movable object; and the environment image includes a second object other than the first object;

[0056] Step 203, determining the real position of the calibration point based on the designated position of the key point of the second object and the relative position between the key point and the calibration point of the second object;

[0057] Step 204: determining the external parameters of the camera according to the image position of the calibration point in the environment image and the real position.

[0058] The above steps 201-204 can refer to the above Figure 1 The contents of the embodiments will not be described in detail here.

[0059] Optionally, after the step of controlling the key points on at least one movable object to move to respective designated positions based on the preset position set, the method further comprises:

[0060] For each movable object, determining at least one point around the movable object;

[0061] The movable object is controlled to adjust its posture so that at least one point of the movable object is located in the same horizontal plane.

[0062] In the embodiment of the present application, for each movable object, at least one point is determined around each movable object. When the movable object is a vehicle, a point can be determined according to the positions of the four wheels of the vehicle, for example, the left side of the left front wheel, the right side of the right front wheel, the left side of the left rear wheel, and the left side of the right rear wheel, and the distance between each point and the wheel can be the same.

[0063] Then the vehicle can be controlled to adjust its posture, for example, by adjusting the state of the vehicle's hydraulic system, so that at least one point is located at the same horizontal plane, thereby making the vehicle reach a stable balanced posture. In addition, it is understandable that due to terrain reasons, the hydraulic stars of multiple vehicles can be controlled to rise or fall, so that all points of all vehicles are located at the same horizontal plane, so that a stable and highly usable calibration field can be further constructed.

[0064] For example, there can be 4 points, the AGV corresponding to the left front wheel fl AGV corresponding to the right front wheel fr , AGV corresponding to the left rear wheel bl1 AGV corresponding to the right rear wheel br1 , the four points are all located in the same plane. If the vehicle's forward direction is the X positive axis, the left side of the vehicle is the Y positive axis, and the upward direction is the Z positive axis, the movable object's own calibration coordinate system O is constructed. cali , then the four points are all located in the same Z plane.

[0065] In the implementation of the embodiment of the present application, for each movable object, at least one point is determined around the movable object; the movable object is controlled to adjust its posture so that at least one point of the movable object is located on the same horizontal plane. When constructing a calibration field based on the movable object, it is not necessary to rely too much on good terrain conditions. Through its own posture adjustment, a stable and balanced posture can be achieved, thereby constructing a stable and highly usable calibration field.

[0066] Optionally, the movable object has identification information; the preset position set includes the identification information and its corresponding designated position;

[0067] The step of controlling the movement of at least one movable object based on the preset position set so that the key points of each movable object are at their respective designated positions comprises:

[0068] According to the target identification information of the movable object to be moved, determining a target designated position corresponding to the target identification information in a preset position set;

[0069] The movable object is controlled to move so that the key point of the movable object is located at the target specified position.

[0070] In the embodiment of the present application, each movable object has identification information, and the preset position set includes the identification information and its corresponding designated position, the identification information can be a number, and the designated position is a position in the world coordinate system. When controlling at least one movable object, only one can be controlled at the same time, or multiple objects can be controlled at the same time.

[0071] According to the target identification information of the currently controlled movable object, the corresponding target designated position is determined, and then according to the target designated position, the current movable object is controlled to move so that the key point of the current movable object reaches the target designated position. When controlling other movable objects to move, the process is similar and will not be repeated here.

[0072] Optionally, the movable object is provided with visual identification information; the preset position set includes the identification information and its corresponding designated position;

[0073] After the step of capturing the environment image corresponding to the first object by a camera on the first object, the method further includes:

[0074] Identify the environment image to obtain target identification information of the second object in the environment image;

[0075] In the preset position set, a target designated position corresponding to the target identification information is obtained to determine the designated position where the key point of the second object is located.

[0076] In an embodiment of the present application, the identification information of the movable object can be displayed on the movable object in a visual manner, for example, it can be a pattern with color, a light with a shape, etc. The pattern and the light with a shape can display identification information such as number and code.

[0077] After the environment image corresponding to the first object is captured by the camera on the first object, image recognition can be performed on the environment image to obtain the target identification information in the environment image. Through the target identification information, it can be determined which specific second object is. For example, in addition to the four movable objects, their respective identification information is A, B, C and D. When the movable object with identification information A is the first object, the movable objects with identification information B, C and D are the second objects. When the identification information is identified as B in the environment image, the specific second object is determined.

[0078] In the preset position set including identification information and its corresponding designated position, the target designated position corresponding to the target identification information is determined, and then the designated position where the key point of the second object in the environment image is located is determined.

[0079] In the implementation of the embodiment of the present application, the target identification information of the second object in the environment image is obtained by identifying the environment image, and the target designated position corresponding to the target identification information is obtained in the preset position set to determine the designated position where the key point of the second object is located. The target identification information of the current second object can be directly obtained by the image recognition method, and then the position of the key point of the current second object can be quickly found to determine the position, and the key point position can be obtained in a convenient and fast manner, thereby improving the efficiency of determining the true position of the calibration point.

[0080] Optionally, a calibration figure is provided on the movable object, and the calibration point is located at the geometric center of the calibration figure;

[0081] Before the step of determining the external parameters of the camera according to the image position of the calibration point in the environment image and the real position, the method further includes:

[0082] In the environment image, determining the calibration pattern;

[0083] Based on at least one graphic key point on the calibration graphic, the calibration point indicated by the calibration graphic is determined; the distance between the calibration point and any graphic key point is the same.

[0084] In the embodiment of the present application, the calibration point of the movable object is located at the geometric center of the calibration figure, and the calibration figure can be a regular figure, such as a circle, an equilateral triangle, a square, etc. It can be understood that the calibration point is a point, and the range of the point is very small, so in the environmental image, the display range of the calibration point may be very small or not clear at all. In addition, if the distance between the first object and the second object is far, it will also lead to the problem of not being able to obtain a clear calibration point.

[0085] Therefore, the calibration figure can be determined first in the environmental image. Since the calibration figure obviously has a larger display range than the calibration point, the calibration figure usually has an obvious display range in the captured environmental image, so the calibration figure can be easily obtained.

[0086] Then, at least one key point of the graph can be determined on the calibration graph. For example, the calibration graph can be a circle, and the key point of the graph can be any point on the circle; the calibration graph can be a square, and the key point of the graph can be any corner point; it can also be a triangle, a rhombus, etc.

[0087] The calibration point indicated by the calibration graphic is determined according to the key points of the graphic. The distance between the calibration point and any key point of the graphic is the same. For example, if the calibration graphic is a circle, the center of the circle can be directly determined as the calibration point.

[0088] Figure 3 is a schematic diagram of the structure of a movable object provided in an embodiment of the present application;

[0089] Figure 3 The system includes a housing 301, a camera 302, a laser distance measuring device 303, a calibration pattern 304, a hydraulic system 305, and wheels 306. A preset pattern such as a black and white chessboard can be set on the top of the housing 301, and key points in the black and white chessboard can also be used as calibration points. The camera 302 is used to capture environmental graphics. The laser distance measuring device 303 is used to measure the distance between the current movable object and other objects. The calibration pattern 304 includes calibration points. The hydraulic system 305 is used to adjust the posture. The wheels 306 are used to move.

[0090] Figure 4 is a schematic diagram of a preset pattern on the top of a shell provided in an embodiment of the present application; Figure 4 The default pattern in is a black and white checkerboard.

[0091] Figure 5 is a schematic diagram of a calibration figure and calibration points provided in an embodiment of the present application; Figure 5 The calibration figure 50 is included, and the geometric center of the calibration figure 50 is the calibration point 501.

[0092] The embodiments of the present application are implemented by determining a calibration figure in an environment image, and determining a calibration point indicated by the calibration figure based on at least one key graphic point on the calibration figure. This can avoid the problem of not being able to obtain a clear calibration point due to a too small display range of the calibration point, and thus avoid the problem of not being able to obtain a clear calibration point. By determining the calibration point through the calibration figure, the accuracy and efficiency of determining the calibration point can be improved.

[0093] Optionally, the display brightness of the calibration graphic is adjustable;

[0094] The step of determining the calibration pattern in the environment image comprises:

[0095] In the environment image, determining the image range of the calibration figure;

[0096] When the edge definition of the image range is less than a preset threshold, increasing the display brightness of the calibration graphic;

[0097] Continue to calculate the edge clarity of the image range until the edge clarity is greater than or equal to the preset threshold value to obtain the final calibration figure.

[0098] In the embodiment of the present application, as mentioned above, the calibration graphic can be a pattern with color, for example, a graphic mark is added to the movable object by coating. The calibration graphic can also be realized by light, for example, a circular light display area.

[0099] Then the process of determining the calibration figure in the environment image can be, in the environment image, determining the image range of the calibration figure, and calculating the edge clarity of the image range. The edge clarity can be measured by calculating the gradient of the image through the gradient amplitude. The gradient amplitude can reflect the strength of the edge. The larger the gradient amplitude, the clearer the edge. The standard deviation of the pixel value in the edge area can also be calculated through the standard deviation. The larger the standard deviation, the larger the change in pixel value and the clearer the edge. The clarity evaluation function can also be used, such as the Laplace energy function used to evaluate the image clarity.

[0100] After calculating the edge clarity, if the edge clarity of the image range is less than the preset threshold, increase the display brightness of the calibration graphic until the edge clarity of the image range in the environmental image is greater than or equal to the preset threshold to obtain a sufficiently clear calibration graphic. The clearer the calibration graphic in the environmental image, the more accurate the calibration points determined based on the calibration graphic.

[0101] In the implementation of the present application, when the edge clarity of the image range is less than a preset threshold, the display brightness of the calibration pattern is increased, and the edge clarity of the image range is continuously calculated until the edge clarity is greater than or equal to the preset threshold, so as to obtain the final calibration pattern. A calibration pattern with higher clarity can be determined, thereby making the accuracy of the calibration points determined based on the calibration pattern higher, and finally improving the accuracy of the camera extrinsic parameter calculation.

[0102] In CMS intelligent recognition applications, it is necessary to identify and measure the distance of objects behind the vehicle, so external parameters with a wide coverage range are required. This calibration method can only provide external parameter calibration points within a small range, which is difficult to meet the external parameter calibration needs of CMS. According to the original factory calibration plan, building a larger calibration space of the same type has the problems of high construction cost and large fixed space occupation.

[0103] Optionally, after the step of capturing the environment image corresponding to the first object by a camera on the first object, the method further includes:

[0104] Step A1, controlling the first object and / or the second object to move at least once, so that the first object is away from the second object;

[0105] Step A2: After each movement is completed, continue to capture the environment image of the first object through the camera on the first object.

[0106] In the embodiment of the present application, the key points of each movable object are at their respective designated positions, which can be understood as the key points reaching the initial positions, at which the first capture of the environmental image can be performed.

[0107] After the capture is completed, the movable object can be further moved, which can be the first object moving or the second object moving, or the first object and the second object moving at the same time, the purpose is to make the first object away from the second object. Since the calibration field is jointly constructed by the first object and the second object (that is, all movable objects), after the first object moves away from the second object, the calibration field expands, thereby constructing a calibration field of a larger range.

[0108] The number of movements can be multiple times, and after each movement is completed, the environment image of the first object can be captured based on the camera on the first object. It is understandable that after multiple movements, multiple environment images can be obtained, and the second object in each environment image can be different. For example, the calibration field is expanded, and the camera can capture more movable objects. In addition, it is understandable that before and after the movement, the calibration point of the same second object, the real coordinates in the world coordinate system and the image coordinates in the environment image will change.

[0109] In addition, since the actual position of the calibration point needs to be based on the specified position of the key point of the second object and the relative position between the key point and the calibration point, the current position of the key point can be obtained by simply updating the position of the key point according to the specific moving direction and distance, and the relative position is unchanged, which will not be repeated here.

[0110] The camera extrinsic parameters are calculated based on a set of positions of the calibration points (i.e., image positions and real positions). After at least one movement of the first object and / or the second object, more calibration points in a larger calibration field can be obtained. These calibration points also have a set of image positions and real positions. Then, the extrinsic parameters can be jointly calculated based on the positions of multiple calibration points, so that the calculated camera extrinsic parameters can be applicable to larger scenes.

[0111] Figure 6 Schematic diagram of the movable object provided in the embodiment of the present application moving; Figure 6 As shown, a wheel position is used as a reference for the moving process. When the wheel is at position 601, the key point is at the target specified position (which can be understood as the initial position). Then the movable object moves. After the first movement, the wheel is at position 602. After the second movement, the wheel is at position 603. After the third movement, the wheel is at position 604. It can be understood that Figure 6 The moving process is only shown for example, and the specific number of moves and the distance of each move can be adjusted.

[0112] Figure 7 It is a top view schematic diagram of the movement of a movable object provided in an embodiment of the present application; vehicle 70 can move to the positions indicated by position 701, position 702, position 703, and position 704 respectively.

[0113] The embodiment of the present application is implemented to control the first object and / or the second object to move at least once so that the first object is away from the second object; after each movement is completed, the environment image of the first object continues to be captured by the camera on the first object. A larger calibration field can be constructed and more calibration points can be obtained. External parameter calculation based on a larger calibration field and a larger number of calibration points can make the calculated camera external parameters applicable to larger scenes, thereby improving the availability of camera external parameters.

[0114] Figure 8 is a flow chart of external parameter calibration provided in an embodiment of the present application;

[0115] Step 801, coordinates of each AGV, i.e., determining the position of the key points of each movable object;

[0116] Step 802, determining the corresponding movable object according to the AGV number in the environment image;

[0117] Step 803, determining the real coordinates of each calibration point;

[0118] Step 804, the real coordinates of each calibration point form a set;

[0119] Step 805, collecting environment images;

[0120] Step 806, performing dedistortion processing on the environment image;

[0121] Step 807, identifying the corner points of the black and white chessboard and the center point of the calibration figure;

[0122] Step 808, rearrange the point sequence, that is, make the real coordinates of the multiple calibration points correspond to the graphic coordinates one by one;

[0123] Step 809, calculating external parameters based on the real coordinates and graphic coordinates of multiple calibration points.

[0124] In summary, in the embodiments of the present application, at least one movable object can be controlled to move based on a preset position set so that the key points of each movable object are at their respective designated positions; calibration points are set on the movable objects; the environment image corresponding to the first object is captured by a camera; the real position of the calibration point is determined based on the designated position of the key point of the second object and the relative position between the key point and the calibration point; the external parameters of the camera are determined according to the image position and the real position of the calibration point in the environment image. Based on the movement control of the movable object, a calibration field with flexible size and position can be constructed, and then the external parameters of the camera on the movable object used to constitute the calibration field can be calibrated. It can be applied to different scenes, has strong terrain adaptability, and is highly convenient, thereby improving the flexibility of the camera external parameter calibration.

[0125] refer to Fig. 9 , which shows a camera extrinsic parameter calibration device 90 provided in an embodiment of the present application, the device comprising:

[0126] The object movement module 901 is used to control the movement of at least one movable object based on a preset position set so that the key points of each movable object are at their respective designated positions; the preset position set includes the designated positions corresponding to each movable object; and the movable object is provided with a calibration point and a camera;

[0127] An image capturing module 902 is configured to capture an environment image corresponding to a first object through a camera on the first object; the first object is any movable object; and the environment image includes a second object other than the first object;

[0128] A calibration position module 903, configured to determine the real position of the calibration point based on the specified position of the key point of the second object and the relative position between the key point and the calibration point of the second object;

[0129] The external parameter calculation module 904 is used to determine the external parameters of the camera according to the image position of the calibration point in the environment image and the real position.

[0130] Optionally, the device further comprises:

[0131] A point determination module, configured to determine, for each movable object, at least one point around the movable object;

[0132] The point leveling module is used to control the movable object to adjust its posture so that at least one point of the movable object is located in the same horizontal plane.

[0133] Optionally, the movable object has identification information; the preset position set includes the identification information and its corresponding designated position;

[0134] The object moving module comprises:

[0135] A target object position submodule, configured to determine a target designated position corresponding to the target identification information in a preset position set according to the target identification information of the movable object to be moved;

[0136] The target object control submodule is used to control the movement of the movable object so that the key point of the movable object is at the target specified position.

[0137] Optionally, the movable object is provided with visual identification information; the preset position set includes the identification information and its corresponding designated position;

[0138] The device also includes:

[0139] An object identification module, used to identify the environment image and obtain target identification information of the second object in the environment image;

[0140] The identification position module is used to obtain the target designated position corresponding to the target identification information in the preset position set to determine the designated position where the key point of the second object is located.

[0141] Optionally, a calibration figure is provided on the movable object, and the calibration point is located at the geometric center of the calibration figure;

[0142] The device also includes:

[0143] A graphic determination module, used to determine the calibration graphic in the environment image;

[0144] The graphic interior point module is used to determine the calibration point indicated by the calibration graphic based on at least one graphic key point on the calibration graphic; the distance between the calibration point and any graphic key point is the same.

[0145] Optionally, the display brightness of the calibration graphic is adjustable;

[0146] The graphic determination module comprises:

[0147] A graphics range submodule, used to determine the image range of the calibration graphics in the environment image;

[0148] An edge clarity submodule, used to increase the display brightness of the calibration graphic when the edge clarity of the image range is less than a preset threshold;

[0149] The final graphics submodule is used to continue calculating the edge clarity of the image range until the edge clarity is greater than or equal to the preset threshold value, so as to obtain the final calibration graphics.

[0150] Optionally, the device further comprises:

[0151] Controlling the first object and / or the second object to move at least once so that the first object moves away from the second object;

[0152] After each movement is completed, the environment image of the first object continues to be captured by the camera on the first object.

[0153] In summary, in the embodiments of the present application, at least one movable object can be controlled to move based on a preset position set so that the key points of each movable object are at their respective designated positions; calibration points are set on the movable objects; the environment image corresponding to the first object is captured by a camera; the real position of the calibration point is determined based on the designated position of the key point of the second object and the relative position between the key point and the calibration point; the external parameters of the camera are determined according to the image position and the real position of the calibration point in the environment image. Based on the movement control of the movable object, a calibration field with flexible size and position can be constructed, and then the external parameters of the camera on the movable object used to constitute the calibration field can be calibrated. It can be applied to different scenes, has strong terrain adaptability, and is highly convenient, thereby improving the flexibility of the camera external parameter calibration.

[0154] The present application also provides an electronic device, such as Fig.10As shown, it includes a processor 1001 , a communication interface 1002 , a memory 1003 and a communication bus 1004 , wherein the processor 1001 , the communication interface 1002 , and the memory 1003 communicate with each other via the communication bus 1004 .

[0155] The memory 1003 is used to store computer programs.

[0156] When the processor 1001 is used to execute the program stored in the memory 1003, the following steps are implemented: based on a preset position set, controlling at least one movable object to move so that the key point of each movable object is at a respective designated position; the preset position set includes the designated position corresponding to each movable object; the movable object is provided with a calibration point and a camera;

[0157] Capturing an environment image corresponding to the first object by a camera on the first object; the first object is any movable object; the environment image includes a second object other than the first object;

[0158] Determining the true position of the calibrated point based on the specified position of the key point of the second object and the relative position between the key point and the calibrated point of the second object;

[0159] The external parameters of the camera are determined according to the image position of the calibration point in the environment image and the real position.

[0160] Among them, the processor 1001 can also implement other steps in the above-mentioned mainboard controller function verification method, which will not be repeated here.

[0161] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0162] The communication interface is used for communication between the above electronic device and other devices.

[0163] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0164] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0165] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the mainboard controller function verification method described in the above embodiment.

[0166] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the mainboard controller function verification method described in the above embodiment.

[0167] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0168] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0169] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. For the embodiments of the device, electronic device, computer-readable storage medium and computer program products containing instructions, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0170] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A camera extrinsic calibration method, characterized in that: The method comprises: Based on a preset position set, controlling at least one movable object to move so that a key point of each movable object is at a respective designated position; the preset position set includes a designated position corresponding to each movable object; and the movable object is provided with a calibration point and a camera; Capturing an environment image corresponding to the first object by a camera on the first object; the first object is any movable object; the environment image includes a second object other than the first object; Determining the true position of the calibrated point based on the specified position of the key point of the second object and the relative position between the key point and the calibrated point of the second object; The external parameters of the camera are determined according to the image position of the calibration point in the environment image and the real position.

2. The method according to claim 1, characterized in that The method further comprises: Controlling the first object and / or the second object to move at least once so that the first object moves away from the second object; After each movement is completed, the environment image of the first object continues to be captured by the camera on the first object.

3. The method according to claim 1, characterized in that The movable object is provided with visual identification information; the preset position set includes the identification information and its corresponding designated position; The method further comprises: Identify the environment image to obtain target identification information of the second object in the environment image; In the preset position set, a target designated position corresponding to the target identification information is obtained to determine the designated position where the key point of the second object is located.

4. The method according to claim 1, characterized in that: A calibration figure is provided on the movable object, and the calibration point is located at the geometric center of the calibration figure; The method further comprises: In the environment image, determining the calibration pattern; Based on at least one graphic key point on the calibration graphic, the calibration point indicated by the calibration graphic is determined; the distance between the calibration point and any graphic key point is the same.

5. The method according to claim 4, characterized in that The display brightness of the calibration graph is adjustable; The step of determining the calibration pattern in the environment image comprises: In the environment image, determining the image range of the calibration graphic; When the edge definition of the image range is less than a preset threshold, increasing the display brightness of the calibration graphic; Continue to calculate the edge clarity of the image range until the edge clarity is greater than or equal to the preset threshold value to obtain the final calibration figure.

6. The method according to claim 1, characterized in that: The movable object has identification information; the preset position set includes the identification information and its corresponding designated position; The step of controlling the movement of at least one movable object based on the preset position set so that the key points of each movable object are at their respective designated positions comprises: According to the target identification information of the movable object to be moved, determining a target designated position corresponding to the target identification information in a preset position set; The movable object is controlled to move so that the key point of the movable object is located at the target specified position.

7. The method according to claim 1, characterized in that: The method further comprises: For each movable object, determining at least one point around the movable object; The movable object is controlled to adjust its posture so that at least one point of the movable object is located in the same horizontal plane.

8. A camera external parameter calibration device, characterized in that: The device comprises: An object movement module, used to control the movement of at least one movable object based on a preset position set so that the key points of each movable object are at their respective designated positions; the preset position set includes the designated positions corresponding to each movable object; and the movable object is provided with a calibration point and a camera; An image capturing module, configured to capture an environment image corresponding to a first object through a camera on the first object; the first object is any movable object; and the environment image includes a second object other than the first object; a calibration position module, configured to determine the true position of the calibration point based on the specified position of the key point of the second object and the relative position between the key point and the calibration point of the second object; The external parameter calculation module is used to determine the external parameters of the camera according to the image position of the calibration point in the environment image and the real position.

9. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus; wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the steps in the camera extrinsic parameter calibration method as described in any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the camera extrinsic parameter calibration method as described in any one of claims 1 to 7 are implemented.

Images