Event camera extrinsic parameter calibration method and device, electronic equipment and storage medium

Abstract

The application provides an event camera extrinsic parameter calibration method and device, electronic equipment and a storage medium, which comprehensively considers the kinematic characteristics (linear correlation of two sets of linear velocities) of the system and the limited accuracy (existence of time offset t d Based on the correlation of the two sets of linear velocity estimates from the event camera and the sensor on the carrier in kinematics, a first-order kinematic model is established based on the CCA scheme to estimate the external parameters of the event camera.

Description

Technical Field

[0001] This invention belongs to the field of event camera technology, and particularly relates to an event camera extrinsic parameter calibration method, device, electronic device and storage medium. Background Technology

[0002] For physical platforms equipped with cameras, such as robots and autonomous vehicles, determining the installation position and orientation of each camera relative to the platform is crucial; this problem is known as the camera extrinsic parameter calibration problem. Current extrinsic calibration schemes designed for traditional cameras primarily employ hand-eye calibration, which involves reconstructing two trajectories using visual information provided by the camera and other sensors on the platform (such as wheel odometers), and then estimating extrinsic parameters based on the alignment of these two trajectories. However, this method requires ensuring the reliability of the trajectories reconstructed from the visual information.

[0003] Unlike traditional frame cameras, event cameras, as a novel biologically inspired sensor, asynchronously measure pixel-level brightness changes that meet trigger conditions and output a series of events, where each event is a time interval t corresponding to a brightness change. k Location (x) k y k ) and symbol p k The encoding of ∈{-1, +1} is denoted as This special working principle results in better performance in terms of temporal resolution and dynamic range.

[0004] However, due to the different output formats, processing techniques for traditional cameras cannot be directly applied to event cameras. Therefore, mature methods that can effectively eliminate accumulated errors, such as bundle adjustment (BA), are still unavailable in visual odometry based on event cameras. This makes it impossible to guarantee the reliability of the trajectory recovered from visual information during the calibration of event camera extrinsic parameters. Therefore, it is necessary to find new and reliable methods for calibrating event camera extrinsic parameters. Summary of the Invention

[0005] Based on this, and in response to the aforementioned technical problems, an event camera extrinsic parameter calibration method, apparatus, electronic device, and storage medium are provided.

[0006] The technical solution adopted in this invention is as follows:

[0007] As a first aspect of the present invention, an event camera extrinsic parameter calibration method is provided, comprising:

[0008] During the linear motion of the carrier, the translational linear velocity v of the event camera on the carrier is determined. eDetermine the translational linear velocity v of the designated sensor on the carrier. o Two sets of linear velocities were obtained in sync. and Where N represents the number of samples, t i Indicates the sampling time;

[0009] The two sets of linear velocities at time offset t are calculated using formula (1). d The cross-covariance and autocovariance are as follows:

[0010]

[0011] in, and They represent v respectively e and v o The average value, and They represent v respectively e and v o The average value after time compensation;

[0012] Based on the first-order kinematic model of the two sets of linear velocities v o =R oe v e Let I be the identity matrix. 3×3 =[s1 s2 s3], outer rotation matrix Based on formula (2), the trace correlation between the two sets of linear velocities is expressed by formula (3):

[0013]

[0014]

[0015] Where Tr(·) represents the trace of the matrix;

[0016] The time offset t is calculated using formula (4). d The optimal solution:

[0017]

[0018] According to the time offset t d The optimal solution and formula (5) are used to calculate the external rotation matrix:

[0019]

[0020] U and V are obtained by performing SVD decomposition on the following matrices:

[0021] As a second aspect of the present invention, an event camera extrinsic parameter calibration device is provided, comprising:

[0022] The velocity sampling module is used to determine the translational linear velocity v of the event camera on the carrier during the carrier's linear motion. e Determine the translational linear velocity v of the designated sensor on the carrier. o Two sets of linear velocities were obtained in sync. and Where N represents the number of samples, t i Indicates the sampling time;

[0023] The variance calculation module is used to calculate the time offset t of two sets of linear velocities using formula (1). d The cross-covariance and autocovariance are as follows:

[0024]

[0025]

[0026]

[0027]

[0028] in, and They represent v respectively e and v o The average value, and They represent v respectively e and v o The average value after time compensation;

[0029] The trace correlation definition module is used to define the first-order kinematic model v based on the two sets of linear velocities. o =R oe v e Let I be the identity matrix. 3×3 =[s1 s2 s3], outer rotation matrix Based on formula (2), the trace correlation between the two sets of linear velocities is expressed by formula (3):

[0030]

[0031] Where Tr(·) represents the trace of the matrix;

[0032] The time offset calculation module is used to calculate the time offset t using formula (4). d The optimal solution:

[0033]

[0034] The extrinsic parameter calculation module is used to calculate the time offset t. dThe optimal solution and formula (5) are used to calculate the external rotation matrix:

[0035]

[0036] U and V are obtained by performing SVD decomposition on the following matrices:

[0037] As a third aspect of the present invention, an electronic device is provided, including a storage module, the storage module including instructions loaded and executed by a processor, the instructions, when executed, causing the processor to perform an event camera extrinsic calibration method according to the first aspect described above.

[0038] As a fourth aspect of the present invention, a computer-readable storage medium is provided that stores one or more programs, which, when executed by a processor, implement the event camera extrinsic calibration method of the first aspect described above.

[0039] This invention comprehensively considers the kinematic characteristics of the system (two sets of linear velocities are linearly correlated) and the finite accuracy of visual odometry based on event cameras (there is a time offset t). d By utilizing the kinematic correlation between two sets of linear velocity estimates from sensors on the event camera and the carrier, respectively, a first-order kinematic model was established based on the CCA scheme to estimate the external parameters of the event camera. Attached Figure Description

[0040] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments:

[0041] Figure 1 A flowchart of an event camera extrinsic parameter calibration method provided in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of an event camera extrinsic parameter calibration device provided in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0044] The embodiments of the present invention will be described below with reference to the accompanying drawings. It should be noted that the embodiments described in this specification are not exhaustive and do not represent the only embodiments of the present invention. The corresponding embodiments below are only for clearly illustrating the inventive content of this patent and are not intended to limit its implementation. For those skilled in the art, different variations and modifications can be made based on the embodiments described. Any variations or modifications that fall within the technical concept and inventive content of this invention and are obvious are also within the protection scope of this invention.

[0045] like Figure 1 As shown in the embodiment of this application, an event camera extrinsic parameter calibration method is provided, including:

[0046] S101. During the linear motion of the carrier, determine the translational linear velocity v of the event camera on the carrier. e Determine the translational linear velocity v of the specified sensor on the carrier. o Two sets of linear velocities were obtained in sync. and Where N represents the number of samples, t i Indicates the sampling time.

[0047] In this embodiment, the translational linear velocity v is determined based on the feature point method. e The specific process is as follows:

[0048] (11) Acquire event data from the event camera and convert the event data into an image-like representation:

[0049] The event data is represented as TS and TOS images. A logical AND operation is performed on the pixel values ​​of corresponding points in the TS and TOS images to form the event image.

[0050] Considering the advantages and disadvantages of various representation methods in pattern recognition (TS is superior) and the fact that the linear velocity estimation based on the feature point method in this application scenario needs to ensure that the event image conforms to speed-invariant representations (TOS can ensure that the 2D spatial gradient of the moving edge remains constant when the camera speed changes), this embodiment uses an event data representation method that combines TS (Time Surface) and TOS (Threshold-Ordinal Surface). Of course, other event data representation methods can also be used as long as the event image conforms to speed-invariant representations.

[0051] It should be noted that each event data point is a quadruple e = {x, y, t, p}, and all event data points are represented as a set of discrete points in three-dimensional space (the three coordinate axes are x, y, and t, and the value is p). The "image-like representation" can be understood as compressing all event data along the time axis (t-axis) into a two-dimensional plane (leaving only the x and y axes) according to various rules. Different rules will result in different images, such as TS, TOS, etc.

[0052] Due to the working mechanism of event cameras, event images can only display the edges of moving objects or relatively moving scenes (relative motion of the scene due to the camera's own motion). The aforementioned "constant velocity representation" means that the outline of the aforementioned edges is not distorted, not the speed of motion. For example, if a cube moves to the right, a series of event data can be obtained. After being converted into an event image, a quadrilateral outline is obtained. Regardless of the cube's velocity in the real scene, this outline should be a square and will not become a rectangle, parallelogram, etc.

[0053] The basic process of the feature point method is as follows: extract feature points from two adjacent frames of images, match the feature points of the two frames of images, analyze the changes in the camera position based on the matching results, and then recover the trajectory. If the feature points do not conform to the constant speed representation (i.e., contour deformation), the matching may fail.

[0054] (12) Extract feature points from the image using the Arc* corner detection algorithm and match the feature points using the BRIEF descriptor.

[0055] (13) Based on the matched feature points, the essential matrix is ​​calculated using the five-point method. During the calculation process, considering the possibility of mismatch, the Random Sample Consensus (RANSAC) algorithm is used to estimate the approximate solution.

[0056] (14) Obtain the translational velocity v from the essential matrix. e Since the carrier moves along a straight line and there is no rotation, the essential matrix is ​​an antisymmetric matrix, so the linear velocity v of translation can be directly obtained from it. e .

[0057] The specified sensor can be an IMU, LiDAR, wheel odometer, or traditional frame camera, etc., to determine the translational linear velocity v of the specified sensor. o The specific process is as follows:

[0058] Acquire data from a specified sensor, reconstruct the corresponding trajectory based on the data, and extract the translational linear velocity v from the trajectory. o .

[0059] Taking a wheel odometer as an example, wheel odometers are typically used in scenarios where wheeled mobile vehicles move in a two-dimensional plane, and can report two states: steering angle θ and wheel speed v. w =[v x v y v z Therefore, the linear velocity v recovered from this sensor o It can be directly calculated using the following formula:

[0060]

[0061] The translational linear velocity v is obtained through other sensors. o There are corresponding publicly known methods for this, which will not be elaborated here.

[0062] S102. Calculate the two sets of linear velocities at time offset t using formula (1). d The cross-covariance and autocovariance are as follows:

[0063]

[0064]

[0065]

[0066]

[0067] in, This represents the average value of the original sample. This represents the average value of the sample after time compensation, i.e. and They represent v respectively e and v o The average value, and They represent v respectively e and v o The average value after time compensation.

[0068] S103, Based on the first-order kinematic model of the two sets of linear velocities v o =R oe v e Let I be the identity matrix. 3×3 =[s1 s2 s3], outer rotation matrix Based on formula (2), the trace correlation between the two sets of linear velocities is expressed by formula (3):

[0069]

[0070] Where Tr(·) represents the trace of the matrix.

[0071] It should be noted that for vector pairs By maximizing the kinematic correlation coefficient between the two sets of linear velocities using formula (2), the kinematic correlation coefficient of each column vector r can be determined. i However, considering the existence of time offset t d The external rotation matrix obtained at this time has a certain error. Therefore, based on formula (2), the trace correlation between the two sets of linear velocity estimates is further defined.

[0072] Trace correlation can also be used as an indicator of correlation, and it has an important property: the trace correlation between two random vectors does not change after a linear transformation (e.g., scaling, rotation, translation). Therefore, r(V) o R oe V e )=r(V o V e This decouples the time offset from the unknown external rotation when evaluating the trace correlation of the two sets of linear velocities, thus transforming the motion correlation problem of the two sets of velocities into an equivalent trace correlation problem.

[0073] S104. Calculate the time offset t using formula (4). d The optimal solution:

[0074]

[0075] Canonical Correlation Analysis (CCA) is an effective tool for evaluating the linear correlation between two random vectors. Therefore, in obtaining the linear velocity v... e v o Then, the optimal solution for the time offset between the two is obtained based on the CCA method, so that the correlation between the two sets of linear velocities reaches its maximum under this time offset.

[0076] S105, Based on time offset t d The optimal solution and formula (5) are used to calculate the external rotation matrix:

[0077]

[0078] U and V are obtained by performing SVD decomposition on the following matrices:

[0079] It should be noted that, according to formula (2), the definition is... We can obtain the following from the Cauchy-Schwarz inequality:

[0080]

[0081] If and only if When the equality holds, therefore, when At this point, the two sets of linear velocity estimates reach their maximum values. Therefore, the external rotation matrix can be calculated using formula (5).

[0082] As can be seen from the above, the embodiments of this application fully consider the kinematic characteristics of the system (the two sets of linear velocities are linearly correlated) and the limited accuracy of the visual odometry based on the event camera (there is a time offset td). By utilizing the kinematic correlation between the two sets of linear velocity estimates from the event camera and the sensors on the carrier, a first-order kinematic model is established on the basis of the CCA scheme to estimate the external parameters of the event camera.

[0083] The following describes in detail one or more embodiments of an event camera extrinsic parameter calibration apparatus of the present invention. Those skilled in the art will understand that these calibration apparatuses can all be configured using commercially available hardware components through the steps taught in this solution. Figure 2 An embodiment of the present invention provides an event camera extrinsic parameter calibration device, such as... Figure 2 As shown, the calibration device includes a velocity sampling module 11, a variance calculation module 12, a trace correlation definition module 13, a time offset calculation module 14, and an external parameter calculation module 15.

[0084] The velocity sampling module 11 is used to determine the translational linear velocity v of the event camera on the carrier during the carrier's linear motion. e Determine the translational linear velocity v of the specified sensor on the carrier. o Two sets of linear velocities were obtained in sync. and Where N represents the number of samples, t i Indicates the sampling time.

[0085] In this embodiment, the translational linear velocity v is determined based on the feature point method. e The specific process is as follows:

[0086] (11) Acquire event data from the event camera and convert the event data into an image-like representation:

[0087] The event data is represented as TS and TOS images. A logical AND operation is performed on the pixel values ​​of corresponding points in the TS and TOS images to form the event image.

[0088] Considering the advantages and disadvantages of various representation methods in pattern recognition (TS is superior) and the fact that the linear velocity estimation based on the feature point method in this application scenario needs to ensure that the event image conforms to speed-invariant representations (TOS can ensure that the 2D spatial gradient of the moving edge remains constant when the camera speed changes), this embodiment uses an event data representation method that combines TS (Time Surface) and TOS (Threshold-Ordinal Surface). Of course, other event data representation methods can also be used as long as the event image conforms to speed-invariant representations.

[0089] It should be noted that each event data point is a quadruple e = {x, y, t, p}, and all event data points are represented as a set of discrete points in three-dimensional space (the three coordinate axes are x, y, and t, and the value is p). The "image-like representation" can be understood as compressing all event data along the time axis (t-axis) into a two-dimensional plane (leaving only the x and y axes) according to various rules. Different rules will result in different images, such as TS, TOS, etc.

[0090] Due to the working mechanism of event cameras, event images can only display the edges of moving objects or relatively moving scenes (relative motion of the scene due to the camera's own motion). The aforementioned "constant velocity representation" means that the outline of the aforementioned edges is not distorted, not the speed of motion. For example, if a cube moves to the right, a series of event data can be obtained. After being converted into an event image, a quadrilateral outline is obtained. Regardless of the cube's velocity in the real scene, this outline should be a square and will not become a rectangle, parallelogram, etc.

[0091] The basic process of the feature point method is as follows: extract feature points from two adjacent frames of images, match the feature points of the two frames of images, analyze the changes in the camera position based on the matching results, and then recover the trajectory. If the feature points do not conform to the constant speed representation (i.e., contour deformation), the matching may fail.

[0092] (12) Extract feature points from the image using the Arc* corner detection algorithm and match the feature points using the BRIEF descriptor.

[0093] (13) Based on the matched feature points, the essential matrix is ​​calculated using the five-point method. During the calculation process, considering the possibility of mismatch, the Random Sample Consensus (RANSAC) algorithm is used to estimate the approximate solution.

[0094] (14) Obtain the translational velocity v from the essential matrix. e Since the carrier moves along a straight line and there is no rotation, the essential matrix is ​​an antisymmetric matrix, so the linear velocity v of translation can be directly obtained from it. e .

[0095] The specified sensor can be an IMU, LiDAR, wheel odometer, or traditional frame camera, etc., to determine the translational linear velocity v of the specified sensor. o The specific process is as follows:

[0096] Acquire data from a specified sensor, reconstruct the corresponding trajectory based on the data, and extract the translational linear velocity v from the trajectory. o .

[0097] Taking a wheel odometer as an example, wheel odometers are typically used in scenarios where wheeled mobile vehicles move in a two-dimensional plane, and can report two states: steering angle θ and wheel speed v. ω =[v x v y v z Therefore, the linear velocity v recovered from this sensor o It can be directly calculated using the following formula:

[0098]

[0099] The translational linear velocity v is obtained through other sensors. o There are corresponding publicly known methods for this, which will not be elaborated here.

[0100] Variance calculation module 12 is used to calculate the time offset t of two sets of linear velocities using formula (1). d The cross-covariance and autocovariance are as follows:

[0101]

[0102]

[0103]

[0104]

[0105] in, This represents the average value of the original sample. This represents the average value of the sample after time compensation, i.e. and They represent v respectively e and v o The average value, and They represent v respectively e and v o The average value after time compensation.

[0106] Trace correlation definition module 13 is used to define the first-order kinematic model v based on two sets of linear velocities. o =R oe v e Let I be the identity matrix. 3×3 =[s1 s2 s3], outer rotation matrix Based on formula (2), the trace correlation between the two sets of linear velocities is expressed by formula (3):

[0107]

[0108]

[0109] Where Tr(·) represents the trace of the matrix.

[0110] It should be noted that for vector pairs By maximizing the kinematic correlation coefficient between the two sets of linear velocities using formula (2), the kinematic correlation coefficient of each column vector r can be determined. i However, considering the existence of a time offset t d The external rotation matrix obtained at this time has a certain error. Therefore, based on formula (2), the trace correlation between the two sets of linear velocity estimates is further defined.

[0111] Trace correlation can also be used as an indicator of correlation, and it has an important property: the trace correlation between two random vectors does not change after a linear transformation (e.g., scaling, rotation, translation). Therefore, r(V) o R oe V e )=r(V o V e This decouples the time offset from the unknown external rotation when evaluating the trace correlation of the two sets of linear velocities, thus transforming the motion correlation problem of the two sets of velocities into an equivalent trace correlation problem.

[0112] Time offset calculation module 14 is used to calculate time offset t using formula (4). d The optimal solution:

[0113]

[0114] Canonical Correlation Analysis (CCA) is an effective tool for evaluating the linear correlation between two random vectors. Therefore, in obtaining the linear velocity v... e v o Then, the optimal solution for the time offset between the two is obtained based on the CCA method, so that the correlation between the two sets of linear velocities reaches its maximum under this time offset.

[0115] extrinsic parameter calculation module 15 is used to calculate based on time offset t d The optimal solution and formula (5) are used to calculate the external rotation matrix:

[0116]

[0117] U and V are obtained by performing SVD decomposition on the following matrices:

[0118] It should be noted that, according to formula (2), the definition is... We can obtain the following from the Cauchy-Schwarz inequality:

[0119]

[0120] If and only if When the equality holds, therefore, when At this point, the two sets of linear velocity estimates reach their maximum values. Therefore, the external rotation matrix can be calculated using formula (5).

[0121] In summary, the event camera extrinsic parameter calibration device provided in the above embodiments can execute the event camera extrinsic parameter calibration methods provided in the foregoing embodiments.

[0122] Similar to the above concept, the above Figure 2 The structure of the event camera extrinsic calibration device shown can be implemented as an electronic device. Figure 3 A schematic block diagram of the structure of an electronic device provided by an embodiment of the present invention is shown.

[0123] For example, the electronic device includes a storage module 21 and a processor 22. The storage module 21 includes instructions loaded and executed by the processor 22, which, when executed, cause the processor 22 to perform the steps described in the above section of this specification, which describes a method for calibrating the extrinsic parameters of an event camera, according to various exemplary embodiments of the present invention.

[0124] It should be understood that processor 22 can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, the general-purpose processor can be a microprocessor or any conventional processor.

[0125] This invention also provides a computer-readable storage medium that stores one or more programs that, when executed by a processor, implement the steps described in the above section on an event camera extrinsic parameter calibration method according to various exemplary embodiments of the invention.

[0126] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer-readable storage media (or non-transitory media) and communication media (or transient media).

[0127] As is known to those skilled in the art, the term computer-readable storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0128] For example, the computer-readable storage medium may be an internal storage unit of the electronic device described in the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., provided on the electronic device.

[0129] The electronic devices and computer-readable storage media provided in the foregoing embodiments fully consider the kinematic characteristics of the system (two sets of linear velocities linearly correlated) and the limited accuracy of visual odometry based on event cameras (with time offset t). d By utilizing the kinematic correlation between two sets of linear velocity estimates from sensors on the event camera and the carrier, respectively, a first-order kinematic model was established based on the CCA scheme to estimate the external parameters of the event camera.

[0130] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for calibrating extrinsic parameters of an event camera, characterized in that, include: During the linear motion of the carrier, the translational linear velocity v of the event camera on the carrier is determined. e Determine the translational linear velocity v of the designated sensor on the carrier. o Two sets of linear velocities were obtained in sync. and Where N represents the number of samples, t i Indicates the sampling time; The two sets of linear velocities at time offset t are calculated using formula (1). d The cross-covariance and autocovariance are as follows: in, and They represent v respectively e and v o The average value, and They represent v respectively e and v o The average value after time compensation; Based on the first-order kinematic model of the two sets of linear velocities v o =R oe v e Let I be the identity matrix. 3×3 =[s1 s2 s3], outer rotation matrix Based on formula (2), the trace correlation between the two sets of linear velocities is expressed by formula (3): Where Tr(·) represents the trace of the matrix; The time offset t is calculated using formula (4). d The optimal solution: According to the time offset t d The optimal solution and formula (5) are used to calculate the external rotation matrix: U and V are obtained by performing SVD decomposition on the following matrices:

2. The event camera extrinsic parameter calibration method according to claim 1, characterized in that, The translational linear velocity v of the event camera on the carrier is determined. e Further including: Event data is acquired from the event camera, and the event data is converted into an image-like representation to obtain an event image, which conforms to a velocity-invariant representation. Feature points are extracted from the image using the Arc* corner detection algorithm, and the feature points are matched using the BRIEF descriptor; Based on the matched feature points, the essential matrix is ​​calculated using the five-point method; Obtain the translational velocity v from the essential matrix. e .

3. The event camera extrinsic parameter calibration method according to claim 2, characterized in that, The step of converting event data into an image-like representation to obtain an event image further includes: The event data is represented as TS images and TOS images; A logical AND operation is performed on the pixel values ​​of corresponding points in the TS image and TOS image to form an event image.

4. The event camera extrinsic parameter calibration method according to claim 1, characterized in that, The translational linear velocity v of the designated sensor on the carrier is determined. o Further including: Acquire data from the specified sensor and reconstruct the corresponding trajectory based on the data; Obtain the translational linear velocity v from the trajectory. o .

5. An event camera extrinsic parameter calibration device, characterized in that, include: The velocity sampling module is used to determine the translational linear velocity v of the event camera on the carrier during the carrier's linear motion. e Determine the translational linear velocity v of the designated sensor on the carrier. o Two sets of linear velocities were obtained in sync. and Where N represents the number of samples, t i Indicates the sampling time; The variance calculation module is used to calculate the time offset t of two sets of linear velocities using formula (1). d The cross-covariance and autocovariance are as follows: in, and They represent v respectively e and v o The average value, and They represent v respectively e and v o The average value after time compensation; The trace correlation definition module is used to define the first-order kinematic model v based on the two sets of linear velocities. o =R oe v e Let I be the identity matrix. 3×3 =[s1 s2 s3], outer rotation matrix Based on formula (2), the trace correlation between the two sets of linear velocities is expressed by formula (3): Where Tr(·) represents the trace of the matrix; The time offset calculation module is used to calculate the time offset t using formula (4). d The optimal solution: The extrinsic parameter calculation module is used to calculate the time offset t. d The optimal solution and formula (5) are used to calculate the external rotation matrix: U and V are obtained by performing SVD decomposition on the following matrices:

6. The event camera extrinsic parameter calibration device according to claim 5, characterized in that, The translational linear velocity v of the event camera on the carrier is determined. e Further including: Event data is acquired from the event camera, and the event data is converted into an image-like representation to obtain an event image, which conforms to a velocity-invariant representation. Feature points are extracted from the image using the Arc* corner detection algorithm, and the feature points are matched using the BRIEF descriptor; Based on the matched feature points, the essential matrix is ​​calculated using the five-point method; Obtain the translational velocity v from the essential matrix. e .

7. The event camera extrinsic parameter calibration device according to claim 6, characterized in that, The step of converting event data into an image-like representation to obtain an event image further includes: The event data is represented as TS images and TOS images; A logical AND operation is performed on the pixel values ​​of corresponding points in the TS image and TOS image to form an event image.

8. The event camera extrinsic parameter calibration device according to claim 5, characterized in that, The translational linear velocity v of the designated sensor on the carrier is determined. o Further including: Acquire data from the specified sensor and reconstruct the corresponding trajectory based on the data; Obtain the translational linear velocity v from the trajectory. o .

9. An electronic device, characterized in that, The system includes a storage module containing instructions loaded and executed by a processor, which, when executed, cause the processor to perform an event camera extrinsic calibration method according to any one of claims 1-4.

10. A computer-readable storage medium storing one or more programs, characterized in that, When the one or more programs are executed by the processor, they implement the event camera extrinsic parameter calibration method according to any one of claims 1-4.

Images