External parameter calibration method, device and system for image acquisition device

Abstract

The present disclosure provides an external parameter calibration method, device and system for an image acquisition device, and relates to the field of computer technology. Specific implementation scheme: Send a first acquisition instruction to a controllable car, so that the controllable car moves to a calibration point in a target site, and uses its own position acquisition device to acquire the world coordinates of the calibration point; when the controllable car moves to the calibration point, send a second acquisition instruction to an image acquisition device set in the target site, so that the image acquisition device acquires a target image including a marker set on the controllable car; obtain the external parameters of the calibrated image acquisition device according to the world coordinates of the calibration point and the image coordinates of the marker determined based on the target image. According to the scheme of the present disclosure, the external parameter calibration of the image acquisition device is performed by using the cooperation of the controllable car and the image acquisition device, which can not only solve the personal safety problem of manually performing the external parameter calibration of the image acquisition device, but also improve the calibration accuracy and avoid manual calibration errors.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an external parameter calibration method, device and system for an image acquisition device. Background Art

[0002] With the development of autonomous driving technology and vehicle-road collaboration technology, the detection speed and accuracy of road traffic conditions by roadside equipment have become increasingly important. Among them, the image acquisition device (for example, camera, camera, etc.) used by the roadside equipment can effectively detect the category, position, size, etc. of road users from rich image information through target detection and tracking algorithms, and is an indispensable device in road testing. The prerequisite for using the image acquisition device to perform image detection tasks is to correctly calibrate the image acquisition device and obtain accurate internal and external parameters of the image acquisition device to ensure the accuracy of image recognition. Summary of the invention

[0003] The present disclosure provides an external parameter calibration method, device and system for an image acquisition device to solve or alleviate one or more technical problems in the prior art.

[0004] In a first aspect, the present disclosure provides an external parameter calibration method for an image acquisition device, comprising:

[0005] Sending a first acquisition instruction to the controllable vehicle so that the controllable vehicle moves to a calibration point in the target site and uses its own position acquisition device to acquire the world coordinates of the calibration point;

[0006] When the controllable vehicle moves to the calibration point, a second acquisition instruction is sent to an image acquisition device disposed at the target site, so that the image acquisition device acquires a target image including a marker disposed on the controllable vehicle; and

[0007] The external parameters of the calibration image acquisition device are obtained according to the world coordinates of the calibration points and the image coordinates of the marker determined based on the target image.

[0008] In a second aspect, the present disclosure provides an external parameter calibration device for an image acquisition device, comprising:

[0009] A first sending module is used to send a first acquisition instruction to the controllable vehicle, so that the controllable vehicle moves to a calibration point in the target site, and uses its own position acquisition device to acquire the world coordinates of the calibration point;

[0010] A second sending module is used to send a second acquisition instruction to an image acquisition device set at a target site when the controllable vehicle moves to the calibration point, so that the image acquisition device acquires a target image including a marker set on the controllable vehicle; and

[0011] The calibration module is used to obtain the external parameters of the calibration image acquisition device according to the world coordinates of the calibration point and the image coordinates of the marker determined based on the target image.

[0012] In a third aspect, the present disclosure provides an external parameter calibration system for an image acquisition device, comprising:

[0013] A controllable car includes a car body, a position acquisition device and a marker, wherein the position acquisition device and the marker are arranged on the car body, and the controllable car is used to collect the world coordinates of the calibration points in the target site;

[0014] An image acquisition device is disposed at the target site, and is used to acquire a target image of the controllable vehicle at a calibration point; and

[0015] A control unit is connected to the controllable vehicle and the image acquisition device, and is used to execute the method of any embodiment of the present disclosure.

[0016] In a fourth aspect, the present disclosure provides an electronic device, including:

[0017] At least one processor. And

[0018] A memory in communication with the at least one processor.

[0019] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any embodiment of the present disclosure.

[0020] In a fifth aspect, the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute a method according to any embodiment of the present disclosure.

[0021] According to the scheme disclosed in the present invention, the external parameter calibration of the image acquisition device is performed by using the cooperation between the controllable vehicle and the image acquisition device, which can not only solve the personal safety problem of manually performing the external parameter calibration of the image acquisition device, but also improve the calibration accuracy and avoid manual calibration errors.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments provided according to the present disclosure and should not be regarded as limiting the scope of the present disclosure.

[0024] Figure 1 It is a flowchart of an external parameter calibration method of an image acquisition device according to an embodiment of the present disclosure.

[0025] Figure 2 It is a schematic diagram of the application of the external parameter calibration method of the image acquisition device according to the embodiment of the present disclosure.

[0026] Figure 3 is a schematic diagram of an external parameter calibration system of an image acquisition device according to an embodiment of the present disclosure.

[0027] Figure 4 is a schematic diagram of an external parameter calibration device of an image acquisition device according to an embodiment of the present disclosure.

[0028] Figure 5 It is a block diagram of an electronic device used to implement the external parameter calibration method of the image acquisition device of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The present disclosure will be further described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0030] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0031] like Figure 1 As shown, the embodiment of the present disclosure provides an external parameter calibration method for an image acquisition device, comprising:

[0032] Step S101: Send a first acquisition instruction to the controllable vehicle to enable the controllable vehicle to move to a calibration point in the target site and use its own position acquisition device to acquire the world coordinates of the calibration point.

[0033] Step S102: When the controllable vehicle moves to the calibration point, a second acquisition instruction is sent to an image acquisition device disposed at the target site, so that the image acquisition device acquires a target image including a marker disposed on the controllable vehicle.

[0034] Step S103: Obtaining external parameters of the calibration image acquisition device according to the world coordinates of the calibration points and the image coordinates of the markers determined based on the target image.

[0035] According to the embodiments of the present disclosure, it should be noted that:

[0036] The controllable car can be understood as a small remote-controlled car that can be remotely controlled, or it can be understood as a robot with automatic navigation function that can automatically cruise according to mass, and is not specifically limited here. A position acquisition device and a marker are integrated on the controllable car. Among them, the position acquisition device can be understood as a device in the prior art that can collect the world coordinates of a specific point on the ground. For example, the position acquisition device can be an RTK (Real-time kinematic) carrier phase differential device integrated on the controllable car. The marker can be understood as a brightly colored or reflective object set on the body of the car or on the position acquisition device. The marker is used for image acquisition by the image acquisition device.

[0037] The target site can be understood as a road that can accommodate vehicles and pedestrians.

[0038] The image acquisition device can be understood as a device that has undergone internal parameter calibration and needs to undergo external parameter calibration. The image acquisition device can be a camera, etc., which is not specifically limited here. When the controllable car reaches each calibration point to collect world coordinates, the image acquisition device collects the target image of the controllable car located at the calibration point. According to the target image, image recognition technology can be used to automatically identify the image coordinates of the pixel points corresponding to the markers on the controllable car. The image coordinates are the coordinates in the coordinate system of the image acquisition device. It should be noted that since the controllable car is stationary at the calibration point, obtaining the image coordinates of the markers on the controllable car is equivalent to obtaining the image coordinates of the calibration point.

[0039] Calibration points can be understood as points that are set in the target site in reality or virtually. The number of calibration points, setting locations, etc. can be adjusted as needed, and are not specifically limited here. The calibration points can be preset in the target site, or they can be specially set for the image acquisition device to calibrate the external parameters. When there are multiple calibration points, the controllable car needs to move to each calibration point and obtain the world coordinates of the calibration point, and the image acquisition device needs to collect the target image of the controllable car at each calibration point. When there are multiple calibration points, multiple calibration points can be arranged on the road or road surface in a left-right symmetrical manner.

[0040] The world coordinates of the collected calibration points can be understood as the GPS (Global Positioning System) coordinates or longitude and latitude of the collected calibration points.

[0041] The external parameters of the image acquisition device can be solved by the Zhang Zhengyou calibration method through an iterative method.

[0042] According to the technology of the disclosed embodiment, the controllable car is used to move to the calibration point to collect world coordinates, which can completely replace the method of manually collecting calibration points on the road to calibrate the external parameters of the image acquisition device. This can completely solve the personal safety problem of many vehicles when manually calibrating the external parameters of the image acquisition device on an open road. In addition, by using the image acquisition device to collect images of the markers located at the calibration points, the image coordinates of the calibration points can be quickly and accurately obtained using image recognition technology, which not only has high calibration accuracy, but also solves the error problem of manual calibration.

[0043] In one example, in order to further improve the accuracy of extrinsic parameter calibration, before calibrating the external parameters of the image acquisition device based on the world coordinates of the calibration points and the image coordinates of the marker, the image coordinates of the marker can be converted into the image coordinates of the calibration points according to the relative distance between the marker and the calibration points, and then the external parameters of the calibrated image acquisition device can be calculated based on the image coordinates of the calibration points and the world coordinates of the calibration points.

[0044] In one example, if Figure 2 As shown, Figure 2 The target site shown is provided with a controllable car A, an image acquisition device B for calibrating external parameters, and four calibration points D1, D2, D3, and D4. When calibrating the external parameters of the image acquisition device B, first, a first acquisition instruction is sent to the controllable car A, so that the controllable car A moves to the calibration points D1, D2, D3, and D4 in the target site in sequence, and uses its own position acquisition device to acquire the world coordinates of the calibration points D1, D2, D3, and D4. Then, when the controllable car A moves to the calibration point D1, a second acquisition instruction is sent to the image acquisition device B set in the target site, so that the image acquisition device B acquires a target image including the controllable car located at the calibration point D1. Similarly, when the controllable car A moves to the calibration points D2, D3, and D4, the target images of the controllable car located at the calibration points D2, D3, and D4 are acquired respectively. Among them, markers (i.e. Figure 2 Finally, according to the world coordinates of the four calibration points D1, D2, D3, and D4 and the four image coordinates of the marker determined based on the four target images, the external parameters of the calibration image acquisition device are obtained using the Zhang Zhengyou calibration method.

[0045] In one embodiment, the external parameter calibration method of the image acquisition device of the embodiment of the present disclosure includes steps S101 to S103, wherein step S101: sending a first acquisition instruction to the controllable vehicle to move the controllable vehicle to a calibration point in the target site, and using its own position acquisition device to acquire the world coordinates of the calibration point, including:

[0046] According to the position information of multiple calibration points in the target site, the navigation sub-instruction in the first acquisition instruction is sent to the controllable car, so that the controllable car can be moved to the multiple calibration points in sequence by remote control or automatic navigation.

[0047] When the controllable car moves to each calibration point, a calibration point acquisition instruction in the first acquisition instruction is sent to the controllable car, so that the controllable car acquires the world coordinates of each calibration point.

[0048] According to the embodiments of the present disclosure, it should be noted that:

[0049] The specific walking strategy of the controllable car to collect multiple calibration points is not specifically limited here. The controllable car can collect the world coordinates of multiple calibration points sequentially along a certain direction, or randomly collect the world coordinates of multiple calibration points, as long as the world coordinate collection of all calibration points can be completed.

[0050] In one embodiment, the external parameter calibration method of the image acquisition device of the embodiment of the present disclosure includes steps S101 to S103, wherein step S102: when the controllable vehicle moves to the calibration point, sending a second acquisition instruction to the image acquisition device set at the target site, so that the image acquisition device acquires a target image including a marker set on the controllable vehicle, includes:

[0051] When the controllable car moves to each calibration point, a second acquisition instruction is sent to the image acquisition device set in the target site, so that the image acquisition device acquires the target image of the controllable car located at each calibration point, wherein the target image includes the marker set on the controllable car.

[0052] According to the embodiments of the present disclosure, it should be noted that:

[0053] The timing at which the image acquisition device acquires the target image of the controllable car located at the mark point is not specifically limited here. For example, the target image of the controllable car can be acquired while the controllable car acquires the world coordinates of the calibration point. The target image of the controllable car can also be acquired when the controllable car moves to the calibration point but the world coordinates of the calibration point have not been acquired. The target image of the controllable car can also be acquired when the controllable car moves to the calibration point and the world coordinates of the calibration point have been acquired.

[0054] In one embodiment, the external parameter calibration method of the image acquisition device of the embodiment of the present disclosure includes steps S101 to S103, wherein step S103: obtaining the external parameters of the calibrated image acquisition device according to the world coordinates of the calibration point and the image coordinates of the marker determined based on the target image, includes:

[0055] Step S1031: Determine a world coordinate matrix according to the world coordinates of multiple calibration points.

[0056] Step S1032: determining an image coordinate matrix according to the image coordinates of the markers in a plurality of target images, wherein the plurality of target images are acquired based on a controllable vehicle located at a plurality of calibration points.

[0057] Step S1033: Obtaining external parameters for calibrating the image acquisition device according to the world coordinate matrix and the image coordinate matrix.

[0058] According to the embodiments of the present disclosure, it should be noted that:

[0059] Multiple target images are collected based on the controllable car located at multiple calibration points, which can be understood as: Figure 2 As shown, when the controllable car A moves to the calibration point D1, a second acquisition instruction is sent to the image acquisition device B set at the target site, so that the image acquisition device B acquires the target image including the controllable car located at the calibration point D1. Similarly, when the controllable car A moves to the calibration points D2, D3, and D4, the target images of the controllable car located at the calibration points D2, D3, and D4 are acquired respectively.

[0060] In one example, the specific steps of obtaining the external parameters for calibrating the image acquisition device include:

[0061] The world coordinate matrix and the image coordinate matrix can be brought into Zhang Zhengyou's calibration method, and the following nonlinear equation can be solved by iteration. Finally, the transformation relationship between the three-dimensional and two-dimensional planes can be obtained, that is, the rotation and translation matrix R|t from the world coordinate system to the camera coordinate system, which is the external parameter of the camera.

[0062] The nonlinear equation is:

[0063] sX=K[R|t]X ′

[0064] Where s is the scale factor, X is the image coordinate matrix, K is the intrinsic parameter matrix of the image acquisition device, R is the rotation matrix of the image acquisition device, t is the translation vector of the image acquisition device, and X ′ is the world coordinate matrix.

[0065] In one implementation, the external parameter calibration method of the image acquisition device of the embodiment of the present disclosure includes steps S101 to S103, and steps S1031 to S1033, wherein step S1031: determining a world coordinate matrix according to the world coordinates of a plurality of calibration points includes:

[0066] The world coordinates of the multiple calibration points are first sorted.

[0067] According to the result of the first sorting, the first mean coordinates of the world coordinates of the two calibration points located at the head end and the second mean coordinates of the world coordinates of the two calibration points located at the tail end are determined.

[0068] The midline vector is obtained according to the first mean coordinate and the second mean coordinate.

[0069] A plurality of first vectors are obtained according to the world coordinates of the plurality of calibration points and the first mean coordinates.

[0070] The world coordinates of the multiple calibration points are sorted in a second order according to the absolute values ​​of the angles between the multiple first vectors and the midline vector.

[0071] According to the result of the second sorting, a world coordinate matrix is ​​determined based on the world coordinates of the plurality of calibration points.

[0072] According to the embodiments of the present disclosure, it should be noted that:

[0073] The first sorting of the world coordinates of the multiple calibration points can be understood as sorting the multiple calibration points in a preset direction according to the latitude and longitude of the multiple calibration points.

[0074] The first mean coordinates may be understood as coordinates obtained by taking the average of the world coordinates of the first positive calibration point and the second positive calibration point located at the beginning of the first sort.

[0075] The second mean coordinates may be understood as coordinates obtained by taking the average of the world coordinates of the last calibration point and the second last calibration point at the end of the first sort.

[0076] According to the world coordinates of the multiple calibration points and the first mean coordinates, multiple first vectors are obtained. It can be understood that the world coordinates of each calibration point are calculated with the first mean coordinates to obtain a first vector, and this is repeated multiple times to obtain multiple first vectors.

[0077] The second sorting refers to sorting multiple calibration points on the left and right sides from far to near along the road direction.

[0078] According to the technology of the embodiments of the present disclosure, it can be ensured that even if the controllable car collects the world coordinates of multiple calibration points in random order, it is possible to reorder the multiple calibration points according to the road direction, so as to generate a world coordinate matrix based on the multiple calibration points sorted along the road direction, and then cooperate with the image coordinate matrix generated based on the image coordinates of multiple landmarks sorted along the road direction to realize external parameter calculation.

[0079] In one implementation, the external parameter calibration method of the image acquisition device of the embodiment of the present disclosure includes steps S101 to S103, and steps S1031 to S1033, wherein step S1032: determining an image coordinate matrix according to the image coordinates of the markers in the multiple target images, includes:

[0080] The image coordinates of the markers in the plurality of target images are thirdly sorted.

[0081] According to the result of the third sorting, the third mean coordinates of the image coordinates of the two markers located at the head end and the fourth mean coordinates of the image coordinates of the two markers located at the tail end are determined.

[0082] A third vector is obtained according to the third mean coordinate and the fourth mean coordinate.

[0083] A plurality of fourth vectors are obtained according to the image coordinates of the plurality of markers and the third mean coordinates.

[0084] The image coordinates of the multiple markers are sorted in a fourth manner according to the absolute values ​​of the angles between the multiple fourth vectors and the third vector.

[0085] According to the result of the fourth sorting, an image coordinate matrix is ​​determined based on the image coordinates of the plurality of markers.

[0086] According to the embodiments of the present disclosure, it should be noted that:

[0087] The third sorting of the image coordinates of the markers in the multiple target images may be understood as sorting the image coordinates of the multiple markers in a preset direction according to the image coordinates of the multiple markers.

[0088] The third mean coordinates may be understood as coordinates obtained by taking the average of the image coordinates of the first positive marker located at the head of the third sort and the image coordinates of the second positive marker.

[0089] The fourth mean coordinates may be understood as coordinates obtained by taking the average of the image coordinates of the penultimate marker at the end of the third sort and the image coordinates of the penultimate calibration point.

[0090] According to the image coordinates of multiple markers and the third mean coordinates, multiple fourth vectors are obtained. It can be understood that the image coordinates of each marker are calculated with the third mean coordinates to obtain a fourth vector, and this is repeated multiple times to obtain multiple fourth vectors.

[0091] The fourth sorting refers to sorting the image coordinates of the landmarks corresponding to the multiple calibration points on the left and right sides from far to near along the road direction.

[0092] According to the technology of the embodiments of the present disclosure, it can be ensured that even if the controllable car collects the world coordinates of multiple calibration points in random order, it is possible to reorder the image coordinates of the corresponding multiple markers according to the road direction, so as to generate an image coordinate matrix based on the image coordinates of the multiple markers sorted along the road direction, and then cooperate with the world coordinate matrix generated based on the multiple calibration points sorted along the road direction to realize external parameter calculation.

[0093] In one implementation, the external parameter calibration method of the image acquisition device of the embodiment of the present disclosure includes steps S101 to S103, and further includes:

[0094] Step S104: determining the target field of view of the image acquisition device according to the location of the image acquisition device in the target site, the environmental information around the image acquisition device, and the resolution parameter of the image acquisition device.

[0095] Step S105: setting a calibration point on a target area in a target field corresponding to the target visual field.

[0096] Among them, the image captured by the image acquisition device in the target field of view has the highest clarity, and the target area corresponding to the target field of view can be reached by the controllable vehicle.

[0097] According to the embodiments of the present disclosure, it should be noted that:

[0098] Step S104 can be understood as: according to the setting position of the image acquisition device and its own resolution parameters, multiple candidate fields of view that can capture high-definition images are determined. Then, according to the target site areas corresponding to the multiple candidate fields of view and the environmental information around the image acquisition device, it is determined which candidate field of view corresponds to the target site area that is easy for the controllable car to drive to, so that the candidate field of view is determined as the target field of view range of the image acquisition device.

[0099] According to the technology of the embodiment of the present disclosure, the definition of the target image of the controllable car located at the calibration point acquired by the image acquisition device can be made higher, thereby improving the accuracy of identifying the image coordinates of the marker from the target image. In addition, according to the technology of the embodiment of the present disclosure, exclusive calibration points can be set according to the position of the image acquisition device for calibrating the external parameters and the performance of the controllable car, which can not only facilitate the acquisition of the data required for the external parameter calibration, but also improve the accuracy of the external parameter calibration.

[0100] like Figure 3 As shown, the embodiment of the present disclosure provides an external parameter calibration system for an image acquisition device, comprising:

[0101] The controllable vehicle 310 includes a vehicle body, a position acquisition device 311 and a marker 312. The position acquisition device 311 and the marker 312 are arranged on the vehicle body. The controllable vehicle is used to collect the world coordinates of the calibration points in the target site.

[0102] The image acquisition device 320 is arranged at the target site, and is used to acquire the target image of the controllable vehicle at the calibration point.

[0103] The control unit 330 is connected to the controllable vehicle and the image acquisition device, and is used to execute the method of any embodiment of the present disclosure.

[0104] According to the embodiments of the present disclosure, it should be noted that:

[0105] The controllable vehicle 310 and the image acquisition device 320 in the embodiment of the present disclosure may be understood as the controllable vehicle and the image acquisition device mentioned in the method of any embodiment of the present disclosure.

[0106] According to the technology of the disclosed embodiment, the controllable car is used to move to the calibration point to collect world coordinates, which can completely replace the method of manually collecting calibration points on the road to calibrate the external parameters of the image acquisition device. This can completely solve the personal safety problem of many vehicles when manually calibrating the external parameters of the image acquisition device on an open road. In addition, by using the image acquisition device to collect images of the markers located at the calibration points, the image coordinates of the calibration points can be quickly and accurately obtained using image recognition technology, which not only has high calibration accuracy, but also solves the error problem of manual calibration.

[0107] like Figure 4 As shown, the embodiment of the present disclosure provides an external parameter calibration device for an image acquisition device, comprising:

[0108] The first sending module 410 is used to send a first acquisition instruction to the controllable vehicle, so that the controllable vehicle moves to the calibration point in the target site and uses its own position acquisition device to acquire the world coordinates of the calibration point.

[0109] The second sending module 420 is used to send a second acquisition instruction to the image acquisition device set at the target site when the controllable vehicle moves to the calibration point, so that the image acquisition device acquires the target image including the marker set on the controllable vehicle.

[0110] The calibration module 430 is used to obtain external parameters of the calibration image acquisition device according to the world coordinates of the calibration points and the image coordinates of the marker determined based on the target image.

[0111] In one implementation, the first sending module 410 is used to:

[0112] According to the position information of multiple calibration points in the target site, the navigation sub-instruction in the first acquisition instruction is sent to the controllable car, so that the controllable car can be moved to the multiple calibration points in sequence by remote control or automatic navigation.

[0113] When the controllable car moves to each calibration point, a calibration point acquisition instruction in the first acquisition instruction is sent to the controllable car, so that the controllable car acquires the world coordinates of each calibration point.

[0114] In one implementation, the second sending module 420 is used to:

[0115] When the controllable car moves to each calibration point, a second acquisition instruction is sent to the image acquisition device set in the target site, so that the image acquisition device acquires the target image of the controllable car located at each calibration point, wherein the target image includes the marker set on the controllable car.

[0116] In one embodiment, the calibration module 430 includes:

[0117] The first determination submodule is used to determine a world coordinate matrix according to the world coordinates of a plurality of calibration points.

[0118] The second determination submodule is used to determine the image coordinate matrix according to the image coordinates of the markers in the multiple target images, wherein the multiple target images are collected based on the controllable vehicle located at multiple calibration points.

[0119] The calibration submodule is used to obtain external parameters for calibrating the image acquisition device according to the world coordinate matrix and the image coordinate matrix.

[0120] In one implementation, the first determining submodule is used to:

[0121] The world coordinates of the multiple calibration points are first sorted.

[0122] According to the result of the first sorting, the first mean coordinates of the world coordinates of the two calibration points located at the head end and the second mean coordinates of the world coordinates of the two calibration points located at the tail end are determined.

[0123] The midline vector is obtained according to the first mean coordinate and the second mean coordinate.

[0124] A plurality of first vectors are obtained according to the world coordinates of the plurality of calibration points and the first mean coordinates.

[0125] The world coordinates of the multiple calibration points are sorted in a second order according to the absolute values ​​of the angles between the multiple first vectors and the midline vector.

[0126] According to the result of the second sorting, a world coordinate matrix is ​​determined based on the world coordinates of the plurality of calibration points.

[0127] In one implementation, the second determining submodule is used to:

[0128] The image coordinates of the markers in the plurality of target images are thirdly sorted.

[0129] According to the result of the third sorting, the third mean coordinates of the image coordinates of the two markers located at the head end and the fourth mean coordinates of the image coordinates of the two markers located at the tail end are determined.

[0130] A third vector is obtained according to the third mean coordinate and the fourth mean coordinate.

[0131] A plurality of fourth vectors are obtained according to the image coordinates of the plurality of markers and the third mean coordinates.

[0132] The image coordinates of the multiple markers are sorted in a fourth manner according to the absolute values ​​of the angles between the multiple fourth vectors and the third vector.

[0133] According to the result of the fourth sorting, an image coordinate matrix is ​​determined based on the image coordinates of the plurality of markers.

[0134] In one embodiment, the extrinsic calibration device of the image acquisition device further includes:

[0135] The determination module is used to determine the target field of view of the image acquisition device according to the setting position of the image acquisition device in the target site, the environmental information around the image acquisition device and the resolution parameter of the image acquisition device.

[0136] The setting module is used to set calibration points on a target area in a target field corresponding to a target field of view.

[0137] Among them, the image captured by the image acquisition device in the target field of view has the highest clarity, and the target area corresponding to the target field of view can be reached by the controllable vehicle.

[0138] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0139] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0140] Figure 5 FIG. 1 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Figure 5As shown, the electronic device includes: a memory 510 and a processor 520, and the memory 510 stores a computer program that can be run on the processor 520. The number of memories 510 and processors 520 can be one or more. The memory 510 can store one or more computer programs. When the one or more computer programs are executed by the electronic device, the electronic device executes the method provided by the above method embodiment. The electronic device may also include: a communication interface 530, which is used to communicate with external devices and perform data exchange transmission.

[0141] If the memory 510, the processor 520 and the communication interface 530 are implemented independently, the memory 510, the processor 520 and the communication interface 530 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0142] Optionally, in a specific implementation, if the memory 510, the processor 520 and the communication interface 530 are integrated on a chip, the memory 510, the processor 520 and the communication interface 530 can communicate with each other through an internal interface.

[0143] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0144] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0145] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computer, the process or function according to the embodiment of the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (for example: coaxial cable, optical fiber, data subscriber line (Digital Subscriber Line, DSL)) or wireless (for example: infrared, Bluetooth, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by the computer, or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc. It is worth noting that the computer-readable storage medium mentioned in the present disclosure may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0146] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0147] In the description of the embodiments of the present disclosure, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0148] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means or, for example, A / B can mean A or B. "And / or" in this article is only a way to describe the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0149] In the description of the embodiments of the present disclosure, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

[0150] The above description is only an exemplary embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for calibrating an external parameter of an image acquisition device, comprising: Sending a first acquisition instruction to the controllable vehicle to enable the controllable vehicle to move to a calibration point in the target site and use its own position acquisition device to acquire the world coordinates of the calibration point; When the controllable vehicle moves to the calibration point, a second acquisition instruction is sent to an image acquisition device disposed at the target site, so that the image acquisition device acquires a target image including a marker disposed on the controllable vehicle; In the case where there are multiple calibration points, performing a first sorting on the world coordinates of the multiple calibration points; Determine, according to the result of the first sorting, a first mean coordinate of the world coordinates of the two calibration points at the head end and a second mean coordinate of the world coordinates of the two calibration points at the tail end; Obtaining a midline vector according to the first mean coordinate and the second mean coordinate; Obtaining a plurality of first vectors according to the world coordinates of the plurality of calibration points and the first mean coordinates; performing a second sorting of the world coordinates of the plurality of calibration points according to the absolute values ​​of the angles between the plurality of first vectors and the midline vector; Determine a world coordinate matrix based on the world coordinates of the plurality of calibration points according to the result of the second sorting; Determining an image coordinate matrix according to the image coordinates of the marker in a plurality of target images, wherein the plurality of target images are acquired based on the controllable vehicle located at the plurality of calibration points; as well as According to the world coordinate matrix and the image coordinate matrix, external parameters for calibrating the image acquisition device are obtained.

2. The method according to claim 1, wherein: Sending a first acquisition instruction to the controllable vehicle so that the controllable vehicle moves to a calibration point in a target site and uses its own position acquisition device to acquire the world coordinates of the calibration point, including: According to the position information of the multiple calibration points in the target site, a navigation sub-command in the first acquisition command is sent to the controllable car, so that the controllable car can be moved to the multiple calibration points in sequence by remote control or automatic navigation; When the controllable car moves to each calibration point, a calibration point acquisition instruction in the first acquisition instruction is sent to the controllable car, so that the controllable car acquires the world coordinates of each calibration point.

3. The method according to claim 1, wherein: When the controllable vehicle moves to the calibration point, a second acquisition instruction is sent to an image acquisition device disposed at the target site so that the image acquisition device acquires a target image including a marker disposed on the controllable vehicle, including: When the controllable car moves to each calibration point, a second acquisition instruction is sent to the image acquisition device set in the target site, so that the image acquisition device acquires the target image of the controllable car located at each calibration point, wherein the target image includes the marker set on the controllable car.

4. The method according to claim 1, wherein: Determining an image coordinate matrix according to the image coordinates of the marker in the plurality of target images includes: performing a third sorting on the image coordinates of the markers in the plurality of target images; Determine, according to the result of the third sorting, a third mean coordinate of the image coordinates of the two markers located at the head end and a fourth mean coordinate of the image coordinates of the two markers located at the tail end; Obtaining a third vector according to the third mean coordinate and the fourth mean coordinate; Obtaining a plurality of fourth vectors according to the image coordinates of the plurality of markers and the third mean coordinates; performing a fourth sorting of the image coordinates of the plurality of markers according to the absolute values ​​of the angles between the plurality of fourth vectors and the third vector; According to the result of the fourth sorting, an image coordinate matrix is ​​determined based on the image coordinates of the plurality of markers.

5. The method according to any one of claims 1 to 4, before sending a first acquisition instruction to the controllable vehicle to move the controllable vehicle to a calibration point in the target site and using its own position acquisition device to acquire the world coordinates of the calibration point, further comprises: Determining a target field of view of the image acquisition device according to a setting position of the image acquisition device in the target site, environmental information around the image acquisition device, and a resolution parameter of the image acquisition device; Setting calibration points on a target area in a target field corresponding to the target visual field; Among them, the clarity of the image collected by the image acquisition device in the target field of view is the highest, and the controllable vehicle can reach the target area corresponding to the target field of view.

6. An external parameter calibration system for an image acquisition device, comprising: A controllable vehicle comprises a vehicle body, a position acquisition device and a marker, wherein the position acquisition device and the marker are arranged on the vehicle body, and the controllable vehicle is used to acquire the world coordinates of a calibration point in a target site; An image acquisition device is arranged at the target site, and is used to acquire a target image of the controllable vehicle located at the calibration point; as well as A control unit, connected to the controllable vehicle and the image acquisition device, for executing the method according to any one of claims 1 to 5.

7. An external parameter calibration device for an image acquisition device, comprising: A first sending module is used to send a first acquisition instruction to the controllable vehicle, so that the controllable vehicle moves to a calibration point in a target site, and uses its own position acquisition device to acquire the world coordinates of the calibration point; A second sending module is used to send a second acquisition instruction to an image acquisition device set at the target site when the controllable vehicle moves to the calibration point, so that the image acquisition device acquires a target image including a marker set on the controllable vehicle; as well as A calibration module, used to obtain external parameters for calibrating the image acquisition device according to the world coordinates of the calibration point and the image coordinates of the marker determined based on the target image; Wherein, the calibration module comprises: A first determination submodule is used to perform a first sorting of the world coordinates of the multiple calibration points; determine, according to the result of the first sorting, a first mean coordinate of the world coordinates of the two calibration points located at the head end and a second mean coordinate of the world coordinates of the two calibration points located at the tail end; obtain a midline vector according to the first mean coordinate and the second mean coordinate; obtain a plurality of first vectors according to the world coordinates of the multiple calibration points and the first mean coordinate; perform a second sorting of the world coordinates of the multiple calibration points according to the absolute values ​​of the angles between the multiple first vectors and the midline vector; determine a world coordinate matrix based on the world coordinates of the multiple calibration points according to the result of the second sorting; A second determination submodule is used to determine an image coordinate matrix according to the image coordinates of the marker in a plurality of target images, wherein the plurality of target images are acquired based on the controllable vehicle located at the plurality of calibration points; The calibration submodule is used to obtain external parameters for calibrating the image acquisition device according to the world coordinate matrix and the image coordinate matrix.

8. The device according to claim 7, wherein: The second determining submodule is used for: performing a third sorting on the image coordinates of the markers in the plurality of target images; Determine, according to the result of the third sorting, a third mean coordinate of the image coordinates of the two markers located at the head end and a fourth mean coordinate of the image coordinates of the two markers located at the tail end; Obtaining a third vector according to the third mean coordinate and the fourth mean coordinate; Obtaining a plurality of fourth vectors according to the image coordinates of the plurality of markers and the third mean coordinates; performing a fourth sorting of the image coordinates of the plurality of markers according to the absolute values ​​of the angles between the plurality of fourth vectors and the third vector; According to the result of the fourth sorting, an image coordinate matrix is ​​determined based on the image coordinates of the plurality of markers.

9. The device according to claim 7 or 8, further comprising: A determination module, used to determine the target field of view of the image acquisition device according to the setting position of the image acquisition device in the target site, the environmental information around the image acquisition device and the resolution parameter of the image acquisition device; A setting module, used to set a calibration point on a target area in a target field corresponding to the target visual field; Among them, the clarity of the image collected by the image acquisition device in the target field of view is the highest, and the controllable vehicle can reach the target area corresponding to the target field of view.

10. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

Images