Methods and systems for calibrating external parameters of cameras on two-wheeled vehicles, including two-wheeled vehicles and groups.

Abstract

This application provides a method, system, two-wheeled vehicle, and group for extrinsic parameter calibration of a two-wheeled vehicle camera network, relating to the fields of photogrammetry and calibration. The method includes: receiving marker point position and direction of travel information packets and acquiring real-shot images; extracting real-shot versions of marker points from other two-wheeled vehicles from the real-shot images; parsing the marker point position and direction of travel information packets to obtain a list of neighboring two-wheeled vehicle marker points corresponding to the real-shot images; calculating the coordinates of each projected marker point in the vehicle's coordinate system with the vehicle's camera as the origin; estimating the coordinates of the projected marker points in the pixel coordinate system; and, under conditions that meet automatic calibration requirements, determining the correspondence between the real-shot marker points and the projected marker points, and calibrating the extrinsic parameters of the vehicle's camera based on this correspondence. This method reduces dependence on external calibration objects, enabling real-time calibration in dynamic environments and improving the efficiency of automatic calibration of camera extrinsic parameters.

Description

Technical Field

[0001] This application relates to the field of photogrammetry and calibration technology, specifically to a method, system, two-wheeled vehicle, and group for extrinsic parameter networking calibration of cameras on a two-wheeled vehicle. Background Technology

[0002] Vehicle cameras used for autonomous driving, driver assistance, or traffic warning systems are commonly used to perceive the vehicle's surrounding environment and targets. In the process of obtaining target information from images captured by the camera, perception algorithms require accurately calibrated internal and external parameters of the camera. The external parameters refer to the camera's position and orientation within the vehicle's coordinate system.

[0003] Currently, for situations requiring dynamic calibration while in motion (at the start of the journey), either special targets are needed along the driving path (such as clear lane lines or road signs used for distance measurement), which are usually difficult to meet at any time; or joint calibration is performed by measuring the coordinates of specific targets using lidar, which is also difficult to operate and expensive; or a method is used to estimate the camera pose in real time using visual SLAM or visual-inertial odometry, combined with vehicle motion models (such as Ackerman steering geometry) to optimize extrinsic parameters, but this method suffers from cumulative error, resulting in poor calibration results.

[0004] Therefore, for vehicle-mounted cameras, especially those used in two-wheeled vehicles for driver assistance, there is an urgent need for a highly efficient dynamic calibration method that can be provided while driving. Summary of the Invention

[0005] This application provides a method, system, two-wheeled vehicle, and group for extrinsic parameter network calibration of a two-wheeled vehicle camera. It is applicable to dynamic scenarios of vehicle movement, enabling calibration to be performed in real time in dynamic environments, thereby improving the efficiency of automatic calibration of camera extrinsic parameters.

[0006] In a first aspect, the present invention provides a method for extrinsic parameter network calibration of a two-wheeled vehicle camera, applicable to any two-wheeled vehicle in a group of two-wheeled vehicles, wherein the group comprises multiple two-wheeled vehicles, each including at least one marker point; the method includes: broadcasting its own marker point position and direction of travel information packets to other two-wheeled vehicles in the same group; receiving marker point position and direction of travel information packets sent by other two-wheeled vehicles in the group; acquiring real-shot images based on its own vehicle's camera; and extracting real-shot versions of the marker points of other two-wheeled vehicles from the real-shot images; the marker point position and direction of travel information packets include the vehicle ID, timestamp, coordinates of the vehicle's marker point, and direction of travel of the vehicle; extracting real-shot versions of the marker points of other two-wheeled vehicles from the real-shot images; and parsing the timestamp and the... The system obtains a list of nearby two-wheeled vehicle markers corresponding to the real-shot image by taking the markers of the nearest vehicles in time and their direction of travel from the captured image. This list includes multiple projected markers. The coordinates of each projected marker in the list are calculated in the vehicle's coordinate system with the camera as the origin. Based on preset camera extrinsic and intrinsic parameters, the coordinates of the projected markers in the pixel coordinate system are estimated. The system determines whether automatic calibration conditions are met based on the number of captured and projected markers. If the automatic calibration conditions are met, the correspondence between the captured and projected markers is determined. Based on this correspondence and the camera's intrinsic parameters, the system performs extrinsic parameter calibration on the camera.

[0007] According to one embodiment of the present invention, the step of calculating the coordinates of each simulated marker in the list of nearby two-wheeled vehicle markers in the vehicle body coordinate system with the vehicle camera as the origin, and estimating the coordinates of the simulated markers in the pixel coordinate system based on preset camera extrinsic and intrinsic parameters, further includes: deleting invalid markers; the invalid markers are nearby two-wheeled vehicle markers that cannot be captured by the vehicle camera.

[0008] According to one embodiment of the present invention, the deletion of invalid marker points includes: deleting marker points in the vehicle coordinate system where the angle formed by the line connecting the origin to the marker point and the vertical coordinate is greater than half of the horizontal field of view of the camera; if the vehicle camera is a front-view camera, then deleting the marker points in front of adjacent two-wheeled vehicles traveling in the same direction as the vehicle, and the marker points behind adjacent two-wheeled vehicles traveling in the opposite direction to the vehicle; if the vehicle camera is a rear-view camera, then deleting the marker points behind adjacent two-wheeled vehicles traveling in the same direction as the vehicle, and the marker points in front of adjacent two-wheeled vehicles traveling in the opposite direction to the vehicle.

[0009] According to one embodiment of the present invention, determining whether the automatic calibration conditions are met based on the number of the actual shooting version markers and the number of the deduced version markers, and determining the correspondence between the actual shooting version markers and the deduced version markers when the automatic calibration conditions are met, includes: if the number of the actual shooting version markers is inconsistent with the number of the deduced version markers, then the automatic calibration conditions are not met; if the number of the actual shooting version markers is consistent with the number of the deduced version markers, then the automatic calibration conditions are met.

[0010] According to one embodiment of the present invention, determining whether the automatic calibration conditions are met based on the number of the actual shooting version markers and the number of the deduced version markers, and determining the correspondence between the actual shooting version markers and the deduced version markers when the automatic calibration conditions are met, includes: sorting the pixel coordinate arrays of the actual shooting version markers and the pixel coordinate arrays of the deduced version markers according to the size of the horizontal coordinates to obtain the correspondence between the actual shooting version markers and the deduced version markers.

[0011] According to one embodiment of the present invention, the step of calibrating the extrinsic parameters of the vehicle camera based on the correspondence and the intrinsic parameters of the vehicle camera includes: performing an extrinsic parameter calibration operation based on the correspondence and the intrinsic parameters of the vehicle camera, and replacing the preset camera extrinsic parameters with the calibrated extrinsic parameters.

[0012] According to one embodiment of the present invention, the method further includes: comparing the difference between the pixel coordinates of the deduced version of the marker points and the pixel coordinates of the actual version of the marker points to verify the calibration result of the external parameter calibration.

[0013] Secondly, the present invention also provides a two-wheeled vehicle camera extrinsic parameter network calibration system, applicable to any two-wheeled vehicle in a two-wheeled vehicle group, wherein the two-wheeled vehicle group includes multiple two-wheeled vehicles, each two-wheeled vehicle including at least one marker point, the marker point being a position on the two-wheeled vehicle that can be determined by other nearby two-wheeled vehicles in the same two-wheeled vehicle group; the two-wheeled vehicle camera extrinsic parameter network calibration system includes: a transmitting module, used to broadcast its own marker point position and travel direction information packet to other two-wheeled vehicles in the same two-wheeled vehicle group; a receiving module, used to receive the marker point position and travel direction information packet sent by other two-wheeled vehicles in the two-wheeled vehicle group; the marker point position and travel direction information packet includes the vehicle ID, timestamp, vehicle marker point coordinates, and vehicle travel direction; an image receiving module, used to acquire a real-shot image based on the vehicle's camera, record the shooting timestamp, and send the real-shot image to a calculation module; and a parsing module, used to analyze the real-shot image... The system extracts real-world markers from other two-wheeled vehicles and parses the marker positions and travel directions of the nearest vehicles whose timestamps are closest to the real-world image, obtaining a list of neighboring two-wheeled vehicle markers corresponding to the real-world image. This list includes multiple projected markers. A calculation module calculates the coordinates of each projected marker in the list within the vehicle's coordinate system with the camera as the origin, and estimates the coordinates of the projected markers in the pixel coordinate system based on preset camera extrinsic and intrinsic parameters. A judgment module determines whether automatic calibration conditions are met based on the number of real-world and projected markers; if the conditions are met, it determines the correspondence between the real-world and projected markers. A calibration module performs extrinsic parameter calibration on the vehicle's camera based on the correspondence and the camera's intrinsic parameters.

[0014] Thirdly, the present invention also provides a two-wheeled vehicle, on which the above-mentioned two-wheeled vehicle camera extrinsic parameter networking calibration system is installed.

[0015] Fourthly, the present invention also provides a group of two-wheeled vehicles, the group of two-wheeled vehicles comprising a plurality of the above-described two-wheeled vehicles.

[0016] Compared with existing technologies, the beneficial effects of this application are as follows: By receiving marker location and direction information packets sent by other two-wheeled vehicles and combining them with real-shot images acquired by the vehicle's camera to extract real-shot markers, the system can comprehensively utilize multi-source data to provide real-time and rich input information for calibration. Parsing the marker location and direction information packets whose timestamps are closest to the real-shot images generates a list of markers from neighboring two-wheeled vehicles. This step ensures data time synchronization, reduces errors caused by vehicle movement, and is applicable to dynamic scenarios involving vehicle movement. By calculating the coordinates of the simulated markers in the vehicle's body coordinate system and estimating their positions in the pixel coordinate system based on preset camera parameters, the system establishes a geometric mapping relationship and initially aligns multi-view data. The mechanism of determining whether the automatic calibration conditions are met based on the number of real-shot and simulated markers ensures the reliability of the calibration process, avoids invalid operations when data is insufficient, and thus improves the calibration success rate. When conditions are met, the correspondence between the actual shooting markers and the inferred markers is determined, and the extrinsic parameters are calibrated using intrinsic parameters. This design reduces the reliance on external calibration fields or manual intervention, enabling calibration to be performed in real time in a dynamic environment, which can improve the efficiency of automatic calibration of camera extrinsic parameters. Attached Figure Description

[0017] Figure 1 A schematic diagram illustrating the steps of the two-wheeled vehicle camera extrinsic parameter network calibration method provided in this application embodiment.

[0018] Figure 2 This is a schematic diagram of the internal components of a two-wheeled vehicle provided in an embodiment of this application.

[0019] Figure 3 This is a schematic diagram of a group of two-wheeled vehicles consisting of several two-wheeled vehicles, provided for this application.

[0020] Figure 4 A schematic diagram showing the positional relationship between the actual and simulated markers provided in this application. Detailed Implementation

[0021] The present application will now be described in further detail with reference to experimental examples and specific embodiments. However, this should not be construed as limiting the scope of the subject matter of the present application to the following embodiments. All technologies implemented based on the content of the present application fall within the scope of protection of the present application.

[0022] Unless otherwise specified, the terms "upper," "lower," "left," "right," "center," "inner," "outer," and "side" used in the description of specific embodiments of this application to indicate orientation or positional relationships are based on the orientation or positional relationships shown in the accompanying drawings, or the orientation or positional relationship in which the product / equipment / device is usually placed during use. These terms are merely for the purpose of facilitating the description of the solution in this application or simplifying the description in specific embodiments, so as to enable those skilled in the art to quickly understand the solution, and do not indicate or imply that a particular device / component / element must have a specific orientation, or be constructed and operated in a specific positional relationship. Therefore, they should not be construed as limitations on this application.

[0023] In the description of the embodiments of this application, technical terms such as "first" and "second" only distinguish one entity or operation from another, and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary or secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0024] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0025] Please refer to Figure 1 , Figure 1 A schematic diagram illustrating the steps of the two-wheeled vehicle camera extrinsic parameter networking calibration method provided in this application embodiment. The two-wheeled vehicle camera extrinsic parameter networking calibration method may include:

[0026] S1. Broadcast its own marker position and direction of travel information packets to other two-wheeled vehicles in the same two-wheeled vehicle group, receive marker position and direction of travel information packets sent by other two-wheeled vehicles in the two-wheeled vehicle group, and acquire real-time images based on its own vehicle camera.

[0027] S2. Extract the real-shot version of the markers of other two-wheeled vehicles from the real-shot image, and parse the marker position and direction of travel information packet of the nearest vehicle whose timestamp is closest to the time of the real-shot image to obtain a list of neighboring two-wheeled vehicle markers corresponding to the real-shot image.

[0028] S3. Calculate the coordinates of each simulated marker in the list of nearby two-wheeled vehicle markers in the vehicle coordinate system with the vehicle camera as the origin, and estimate the coordinates of the simulated markers in the pixel coordinate system based on the preset camera extrinsic and intrinsic parameters.

[0029] S4. Determine whether the automatic calibration conditions are met based on the number of actual and simulated marker points. If the automatic calibration conditions are met, determine the correspondence between the actual and simulated marker points.

[0030] S5. Based on the correspondence and the intrinsic parameters of the vehicle camera, perform extrinsic parameter calibration on the vehicle camera to obtain the camera extrinsic parameters, and use them to replace the preset extrinsic parameters.

[0031] The extrinsic parameter network calibration method for two-wheeled vehicle cameras provided in this application can be applied to any two-wheeled vehicle in a group of two-wheeled vehicles. The following is a description of the corresponding application scenarios of this solution.

[0032] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the internal components of a two-wheeled vehicle provided in an embodiment of this application. Each two-wheeled vehicle has cameras (a front-view camera and a rear-view camera), a computing unit, a wireless communication unit, and a high-precision positioning and speed measurement device. The computing unit has general computing and AI computing capabilities. In this embodiment, "two-wheeled vehicle" generally refers to a motorized or electric vehicle with two wheels, specifically including motorcycles, electric bicycles, electric scooters, or autonomous two-wheeled vehicles used for testing or specific scenarios.

[0033] Each vehicle has a unique ID, which cannot be duplicated within the group; depending on the specific scenario, this ID can be derived from the vehicle's serial number, the physical address of the communication module, etc. Each two-wheeled vehicle has at least one marker point, which is a location on the two-wheeled vehicle that can be determined by neighboring two-wheeled vehicles through photography and object detection algorithms, such as... Figure 1 The preceding or following marker.

[0034] The marker has a directional attribute, indicating whether it is located in front of or behind the vehicle. A forward marker is located at the front of the vehicle and is typically only visible to the rear-view camera of a nearby two-wheeled vehicle traveling in the same direction as the vehicle, or to the front-view camera of a nearby two-wheeled vehicle traveling in the opposite direction. A rear marker is located at the rear of the vehicle and is typically only visible to the front-view camera of a nearby two-wheeled vehicle traveling in the same direction as the vehicle, or to the rear-view camera of a nearby two-wheeled vehicle traveling in the opposite direction. Examples of forward markers include: the contact point between the front wheel and the ground of a two-wheeled vehicle, headlights, and the license plate of the vehicle in front; examples of rear markers include: the contact point between the rear wheel and the ground of a two-wheeled vehicle, taillights, and the license plate of the vehicle in rear direction. The specific location used as a marker depends on the characteristics of the AI-based object detection algorithm operating on the two-wheeled vehicle.

[0035] Inside the two-wheeled vehicle, the relative positions of the positioning and speed information receiving devices (such as the modules of a GPS / BDS terminal), the camera installation positions, and the marker points are fixed and known. Changes in the vehicle's load and tilt angle will not significantly affect their relative positional relationships. The two-wheeled vehicle has a precise real-time clock, which can be sourced from the GPS / BDS terminal's timing or from an internal RTC chip; this application does not specify the clock source. The two-wheeled vehicle's wireless communication unit has the ability to send broadcast data packets to the wireless communication devices of neighboring two-wheeled vehicles and also to receive data packets from the wireless communication devices of neighboring two-wheeled vehicles. The communication delay between neighboring two-wheeled vehicles is sufficiently small to affect the system's calculation results. The two-wheeled vehicle's camera can take photos and send them to the computing unit for processing. After receiving the photos, the computing unit records the timestamp of the photo being taken. The external parameters of the two-wheeled vehicle's camera have not changed significantly from their default values, enough to cause incorrect judgments about the forward and backward positions of neighboring two-wheeled vehicles, such as the front-view camera being tilted to the point where it can capture the two-wheeled vehicle behind it.

[0036] The two-wheeled vehicle group comprises multiple vehicles. Each vehicle in the group obtains its real-time location and direction of travel from a positioning and speed measurement device. It calculates and updates its own marker coordinates (latitude and longitude) and camera location coordinates (latitude and longitude), and broadcasts its location information packet, including its vehicle ID, marker coordinates, and timestamp, along with its direction of travel, to all other two-wheeled vehicles in the group in real time. Please see [link / reference]. Figure 3 , Figure 3 This is a schematic diagram of a group of two-wheeled vehicles consisting of several two-wheeled vehicles, provided for this application. Figure 2 The system consists of a group of 15 two-wheeled vehicles, where the horizontal field of view of the front-view camera of the main vehicle is 120° and the horizontal field of view of the rear-view camera is 150°. The internal parameters of the cameras of the two-wheeled vehicles in the group are known, and the external parameters have a set of default values ​​(but need to be calibrated and updated by the method provided in this application) and can be used by the computing unit.

[0037] The two-wheeled vehicles in the group obtain their own location and direction of travel in real time from the positioning and speed measuring device, calculate and update the coordinates (latitude and longitude) of their own marker points and the coordinates (latitude and longitude) of the camera location, and broadcast the location information packet, including the vehicle ID, marker point coordinates, and timestamp, and the direction of travel, to all other two-wheeled vehicles in the group in real time. The marker point location and direction of travel information packet may include the vehicle ID, timestamp, marker point coordinates, and direction of travel.

[0038] Within the group, vehicles need to share their direction of travel, which is used as a key criterion when selecting markers. The coordinates of each vehicle's markers are expressed in latitude and longitude. These coordinates are not directly taken from raw readings of high-precision positioning devices (such as GPS / BDS terminals), but rather are calculated and processed precise values. Specifically, when updating its positioning, each two-wheeled vehicle obtains its real-time latitude and longitude coordinates and direction of travel from the positioning device. Then, based on known internal vehicle structural parameters—such as the positioning device mounting point, camera mounting point, and the fixed relative positions between markers—it uses geometric calculations to derive and update the latitude and longitude coordinates of each marker.

[0039] The cameras on the two-wheeled vehicles in the group take photos in real time and record the time of shooting. Then, based on the photos, target detection is performed to obtain the pixel coordinates of the marker points of the nearby two-wheeled vehicles, that is, the pixel coordinates of the marker points in the real shot.

[0040] In the specific implementation, each two-wheeled vehicle updates its own positioning at a certain frequency (generally not less than 50 Hz, corresponding to a period of not more than 20 milliseconds). This means receiving high-precision real-time coordinates (latitude and longitude) and the vehicle's direction of travel (true north) from its own positioning device. Each time a positioning update occurs, the two-wheeled vehicle should save the update time and, based on the known relative positions of the vehicle's internal positioning device installation point, camera installation point, and marker points, calculate and update the coordinates of the vehicle's camera and the coordinates of each marker point. (In calibration scenarios, because the positional deviation between the positioning device and the marker points is sufficient to affect the calibration effect, the vehicle's positioning information, i.e., the coordinates of the high-precision positioning device, cannot be directly used as the coordinates of the marker points.) Subsequently, the vehicle ID, timestamp (the time of the positioning update), vehicle direction of travel, marker point attribute (front or back), and marker point coordinates (latitude and longitude) are combined to form a marker point position and direction of travel information packet, which is then broadcast along with the vehicle's direction of travel to all other two-wheeled vehicles in the group. The number of markers on the two-wheeled vehicle determines the number of marker location information broadcast packets to be sent after each location update.

[0041] The two-wheeled vehicle receives real-time information packets about the location and direction of travel from neighboring two-wheeled vehicles via its communication unit. The computing unit caches this information.

[0042] The front-view and / or rear-view cameras of the two-wheeled vehicle take photos at a certain frequency (generally not less than 30 Hz, corresponding to a period of not more than 33 milliseconds) and transmit them to the computing unit through a high-speed link inside the vehicle. The computing unit records the timestamp of the photo (the moment when the image data is sent from the camera to the computing unit), and then runs a target detection algorithm to detect all targets in the photo that are classified as two-wheeled vehicles, and obtains their marker points and pixel coordinates in the image.

[0043] After obtaining the target detection results based on a certain photo, the computing unit of the two-wheeled vehicle scans all received marker location information from neighboring two-wheeled vehicles and performs preliminary data filtering according to the following principles: if multiple location information packets belong to the same two-wheeled vehicle and are located in the same direction, only the packet with the smallest absolute value of the difference between its timestamp and the photo's shooting timestamp is selected. Finally, a list of neighboring two-wheeled vehicle markers corresponding to the photo is obtained, which includes multiple extrapolated markers.

[0044] The coordinates of each marker in the list are calculated in the vehicle's coordinate system with the vehicle's camera as the origin. A planar coordinate system is then established with the vehicle's camera as the origin. If it is a front-view camera, the Y-axis of the coordinate system is aligned with the vehicle's heading angle; if it is a rear-view camera, the Y-axis is opposite to the vehicle's heading angle (rotated 180°). Note that the position of the vehicle's camera is updated during vehicle positioning updates, and the coordinates of the markers in the coordinate system are calculated. For two points A and B with known latitude and longitude, the algorithm for calculating the coordinates of point B in a planar coordinate system with point A as the origin and a given Y-axis direction (generally including projection transformation, relative offset calculation, coordinate rotation, etc., is existing technology and will not be elaborated here) is as follows:

[0045] After calculating the coordinates of all marker points, the method provided in this application may further include deleting invalid marker points, which are marker points of adjacent two-wheeled vehicles that the vehicle's camera cannot capture. Deleting invalid marker points may include: deleting marker points in the vehicle's coordinate system where the angle formed by the line connecting the origin to the marker point and the vertical coordinate is greater than half the horizontal field of view of the camera; if the vehicle's camera is a front-view camera, deleting marker points in front of adjacent two-wheeled vehicles traveling in the same direction as the vehicle, and marker points behind adjacent two-wheeled vehicles traveling in the opposite direction; if the vehicle's camera is a rear-view camera, deleting marker points behind adjacent two-wheeled vehicles traveling in the same direction as the vehicle, and marker points in front of adjacent two-wheeled vehicles traveling in the opposite direction.

[0046] Among them, the markers whose angle between the line connecting the origin to the marker and the vertical coordinate is greater than half of the camera's horizontal field of view are located behind the camera's field of view and cannot be captured, therefore they need to be deleted. Please continue to refer to [the relevant documentation]. Figure 3 For the vehicle's front-view camera, this step requires deleting the image from M. 6,R M 6,F To M 14,R M 14,F Those marker points; for a car's rearview camera, this step requires deleting the ones from M... 1,R M 1,F To M 9,R M 9,FThose markers. If the camera is a forward-facing camera, delete the markers (M) in front of neighboring two-wheeled vehicles traveling in the same direction as your vehicle. 3,F M 4,F and M 5,F ), and the rear marker (M) of a neighboring two-wheeled vehicle traveling in the opposite direction to its own vehicle. 1,R M 2,R If the camera is a rear-view camera, then delete the markers (M) behind neighboring two-wheeled vehicles traveling in the same direction as your vehicle. 13,R and M 14,R ), and the marker ahead of a nearby two-wheeled vehicle traveling in the opposite direction (M) 10,F and M 11,F ).

[0047] After deleting invalid markers, the coordinates of the derived markers in the pixel coordinate system can be estimated based on preset camera extrinsic and intrinsic parameters. For example, based on the coordinate values ​​of each marker in the planar coordinate system with the camera as the origin, a set of coarse camera intrinsic and extrinsic parameters can be used, and through a simple projection transformation, the pixel coordinates of these markers in the photograph after being captured by the camera can be obtained. The algorithm for calculating the pixel coordinates of a marker in the camera pixel coordinate system from its coordinate value in the vehicle body coordinate system with the camera as the origin is existing technology and will not be elaborated here. Although this transformation is affected by the camera's intrinsic and extrinsic parameters, in real-world scenarios, since the distance between vehicles is much greater than the height of the camera, it can be assumed that the vehicle's camera and the markers of each adjacent two-wheeled vehicle are on the same horizontal plane. Therefore, although the estimated marker pixel coordinates may have errors, compared with the marker pixel coordinates calculated based on the extrinsic parameters, the relative positional relationships of the markers will not change.

[0048] Please refer to Figure 4 , Figure 4 A schematic diagram showing the positional relationship between the actual and projected markers provided in this application. Figure 3 Based on the vehicle, the actual photo version of the marker points is as follows: Figure 4 The upper part of the text, the deduced version of the marker is... Figure 4The lower half of the text describes the process of estimating the coordinates of the simulated markers in the pixel coordinate system. It then determines whether automatic calibration is feasible. Assume that `m` real-world markers are detected using actual camera photos and AI algorithms; and `n` simulated markers are obtained by receiving location information packets from nearby vehicles and performing corresponding data filtering and transformation. If `m` ≠ `n`, it indicates that the number of markers detected by the camera image is inconsistent with the number of valid markers from neighboring vehicles. This suggests that there is mutual obstruction between adjacent two-wheeled vehicles, or that an unfamiliar two-wheeled vehicle outside the group has appeared in the camera's view. In this case, automatic calibration is not satisfied and cannot be achieved. If `m` = `n`, it indicates a high probability that the following conditions are met: all markers of neighboring two-wheeled vehicles have been captured and detected by the camera; there is no obstruction between the markers of neighboring two-wheeled vehicles, and no unfamiliar two-wheeled vehicles have entered the group.

[0049] Before performing automatic calibration of external parameters, it is also necessary to determine the correspondence between the detected marker points in the image and the adjacent two-wheeled vehicle marker points. Specifically, the pixel coordinate arrays of the actual marker points and the inferred marker points can be sorted according to the size of the horizontal coordinate to obtain the correspondence between the actual marker points and the inferred marker points.

[0050] Assume the pixel coordinate array of the marker points in the actual photograph is as follows:

[0051] Pc[0].X,Pc[0].Y; Pc[1].X,Pc[1].Y;…;Pc[m].X,Pc[m].Y.

[0052] The pixel coordinate array of the deduced marker is:

[0053] Pw[0].X,Pw[0].Y; Pw[1].X,Pw[1].Y;…;Pw[m].X,Pw[m].Y.

[0054] Sort the pixels in both arrays according to their x-coordinates from smallest to largest (corresponding to the markers from left to right), and you will get a one-to-one correspondence between the markers in the two arrays. That is, the i-th marker in the actual image from left to right corresponds to the i-th marker in the deduced image.

[0055] After determining the correspondence between the actual and simulated marker points, an extrinsic parameter calibration operation is performed based on this correspondence. Using photogrammetry principles, the observed data in the pixel coordinate system can be matched with the spatial coordinates in the vehicle body coordinate system, thereby resolving the camera's extrinsic parameters, which are the calibrated extrinsic parameters obtained in this embodiment. Furthermore, the calibrated extrinsic parameters can be used to replace the preset camera extrinsic parameters to update their values, providing a more accurate calibration basis for subsequent tasks.

[0056] Specifically, if the number of marker points is large enough (i.e. m is greater than a certain threshold such as 5), the external parameters of the camera can be calculated and saved for later use based on the pixel coordinates of each real-shot marker point and the coordinates of their corresponding deduced marker points in the vehicle body coordinate system, as well as the known internal parameters of the camera.

[0057] Since the calibration process is performed simultaneously on all two-wheeled vehicles in the group, the dynamic calibration of the extrinsic parameters of multiple two-wheeled vehicle cameras can be completed at the same time.

[0058] In some embodiments, since the vehicles in the group are moving in real time, the two-wheeled vehicle is always ready to verify the calibration results after calibration is completed. The method provided in this application may further include: comparing the difference between the pixel coordinates of the simulated marker points and the pixel coordinates of the actual photographed marker points to verify the calibration results of the extrinsic parameter calibration; the smaller the difference, the more reliable the calibration results.

[0059] When the method provided in this application is applied to dynamic calibration during driving, if camera calibration is required, the group of vehicles can be ridden to a flat open space (road, playground, etc.) before the actual journey begins, and the vehicles can be ridden alternately for a period of time to complete the external parameter calibration of each vehicle's camera.

[0060] In some other embodiments, the method provided in this application can also be applied to the calibration of camera extrinsic parameters when two-wheeled vehicles leave the factory. The calibration site is set as an open-air area, and multiple two-wheeled vehicles equipped with the two-wheeled vehicle camera extrinsic parameter networking calibration system provided in this application are arranged in rows at a relatively fixed interval and pushed to the site at a certain speed, so that calibration can be completed simultaneously. Among them, the high-precision positioning and speed measuring device can be installed on the vehicle during the calibration stage, and can be moved to other vehicles to be calibrated after the calibration is completed.

[0061] Furthermore, the calibration method provided in this application is generally not suitable for calibrating automotive cameras because automobiles are large, and the number of marker points in a single photograph taken by the camera often does not meet the requirements for extrinsic parameter calibration. However, if multiple marker points can be set on the vehicle body, this calibration method can also be used for extrinsic parameter calibration of automotive cameras, provided that the marker points on each vehicle meet the requirements and their relative positions to the mounting points of the vehicle's high-precision positioning device are fixed.

[0062] In the above implementation process, by receiving marker location and direction information packets sent by other two-wheeled vehicles and combining them with real-shot images acquired by the vehicle's camera to extract real-shot markers, the system can comprehensively utilize multi-source data to provide real-time and rich input information for calibration. The system analyzes the marker location and direction information packets whose timestamps are closest to the real-shot images to generate a list of markers from neighboring two-wheeled vehicles. This step ensures data time synchronization, reduces errors caused by vehicle movement, and is applicable to dynamic scenarios involving vehicle movement. By calculating the coordinates of the simulated markers in the vehicle's body coordinate system and estimating their positions in the pixel coordinate system based on preset camera parameters, the system establishes a geometric mapping relationship and initially aligns the multi-view data. The system determines whether the automatic calibration conditions are met based on the number of real-shot and simulated markers. This mechanism ensures the reliability of the calibration process, avoids invalid operations when data is insufficient, and thus improves the calibration success rate. When conditions are met, the correspondence between the actual shooting markers and the inferred markers is determined, and the extrinsic parameters are calibrated using intrinsic parameters. This design reduces the reliance on external calibration fields or manual intervention, enabling calibration to be performed in real time in a dynamic environment, which can improve the efficiency of automatic calibration of camera extrinsic parameters.

[0063] Based on the same concept, this application also provides a two-wheeled vehicle camera extrinsic parameter networking calibration system, which may include:

[0064] The sending module is used to broadcast its own marker location and direction of travel information packets to other two-wheeled vehicles in the same two-wheeled vehicle group;

[0065] The receiving module is used to receive the marker location and direction of travel information packets sent by other two-wheeled vehicles in the two-wheeled vehicle group; the marker location and direction of travel information packets include the vehicle ID, timestamp, marker coordinates of the vehicle, and direction of travel of the vehicle.

[0066] The image receiving module acquires real-time images based on the vehicle's camera, records the shooting timestamp, and sends the real-time images to the computing module;

[0067] The parsing module is used to extract real-shot version markers of other two-wheeled vehicles from the real-shot image, and parse the marker position and travel direction information packet of the nearest vehicle whose timestamp is closest to the time of the real-shot image, to obtain a list of neighboring two-wheeled vehicle markers corresponding to the real-shot image; the list of neighboring two-wheeled vehicle markers includes multiple simulated markers.

[0068] The calculation module is used to calculate the coordinates of each simulated marker in the list of nearby two-wheeled vehicle markers in the vehicle body coordinate system with the vehicle camera as the origin, and to estimate the coordinates of the simulated markers in the pixel coordinate system based on preset camera extrinsic and intrinsic parameters.

[0069] The judgment module is used to determine whether the automatic calibration conditions are met based on the number of real-shot markers and the number of simulated markers. If the automatic calibration conditions are met, the module determines the correspondence between the real-shot markers and the simulated markers.

[0070] The calibration module is used to calibrate the extrinsic parameters of the vehicle camera based on the correspondence and the intrinsic parameters of the vehicle camera.

[0071] Based on the same concept, this application also provides a two-wheeled vehicle equipped with the aforementioned two-wheeled vehicle camera extrinsic parameter networking calibration system.

[0072] Based on the same concept, embodiments of this application also provide a group of two-wheeled vehicles, the group of two-wheeled vehicles including a plurality of the two-wheeled vehicles described above.

[0073] It should be understood that when the various modules of the system provided in the above embodiments are working, the division of each functional module in the above description is only used as an example. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0074] The functional modules in the above embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of the embodiments of this application.

[0075] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for calibrating a two-wheeled vehicle camera extrinsic parameter network, characterized in that, Any two-wheeled vehicle applied to a two-wheeled vehicle group, the two-wheeled vehicle group comprising a plurality of two-wheeled vehicles, each two-wheeled vehicle comprising at least one marker point; The two-wheeled vehicle camera extrinsic parameter networking calibration method comprises: Broadcasting a self marker point position and travel direction information package to other two-wheeled vehicles in the same two-wheeled vehicle group, receiving marker point position and travel direction information packages sent by other two-wheeled vehicles in the two-wheeled vehicle group, and obtaining a real shot image based on a self vehicle camera; the marker point position and travel direction information package comprises a self vehicle ID, a timestamp, a self vehicle marker point coordinate, and a self vehicle travel direction; Extracting a real shot version marker point of other two-wheeled vehicles from the real shot image, and analyzing a marker point position and travel direction information package of a nearby vehicle whose timestamp is closest to a time of the real shot image to obtain a neighboring two-wheeled vehicle marker point list corresponding to the real shot image; the neighboring two-wheeled vehicle marker point list comprises a plurality of deduced version marker points; Calculating coordinates of each deduced version marker point in a vehicle body coordinate system with the self vehicle camera as an origin, and estimating coordinates of the deduced version marker point in a pixel coordinate system based on preset camera extrinsic parameters and intrinsic parameters; Determining whether an automatic calibration condition is met based on a number of the real shot version marker points and a number of the deduced version marker points, and determining a corresponding relationship between the real shot version marker points and the deduced version marker points in a case where the automatic calibration condition is met; Calibrating the self vehicle camera based on the corresponding relationship and intrinsic parameters of the self vehicle camera.

2. The two-wheeler camera extrinsic parameter networking calibration method of claim 1, wherein, The calculation of the coordinates of each deduced version marker point in the vehicle body coordinate system with the self vehicle camera as the origin, and the estimation of the coordinates of the deduced version marker point in the pixel coordinate system based on the preset camera extrinsic parameters and intrinsic parameters further comprises: Deleting invalid marker points; the invalid marker points are neighboring two-wheeled vehicle marker points that cannot be shot by the self vehicle camera.

3. The two-wheeler camera extrinsic parameter networking calibration method of claim 2, wherein, The deletion of the invalid marker points comprises: Deleting a marker point whose angle formed by a line from a coordinate origin to the marker point and a vertical coordinate is greater than half of a horizontal field of view angle of the camera; If the self vehicle camera is a front-view camera, deleting a front marker point of a neighboring two-wheeled vehicle whose travel direction is consistent with that of the self vehicle, and a rear marker point of a neighboring two-wheeled vehicle whose travel direction is opposite to that of the self vehicle; If the self vehicle camera is a rear-view camera, deleting a rear marker point of a neighboring two-wheeled vehicle whose travel direction is consistent with that of the self vehicle, and a front marker point of a neighboring two-wheeled vehicle whose travel direction is opposite to that of the self vehicle.

4. The two-wheeler camera extrinsic parameter networking calibration method of claim 1, wherein, The determination of whether the automatic calibration condition is met based on the number of the real shot version marker points and the number of the deduced version marker points, and the determination of the corresponding relationship between the real shot version marker points and the deduced version marker points in a case where the automatic calibration condition is met comprises: If the number of the real shot version marker points is inconsistent with the number of the deduced version marker points, the automatic calibration condition is not met; If the number of the real shot version marker points is consistent with the number of the deduced version marker points, the automatic calibration condition is met.

5. The two-wheeler camera extrinsic parameter networking calibration method of claim 1, wherein, The step of determining whether the automatic calibration conditions are met based on the number of the actual shooting version markers and the number of the deduced version markers, and determining the correspondence between the actual shooting version markers and the deduced version markers if the automatic calibration conditions are met, includes: The pixel coordinate arrays of the real-shot markers and the pixel coordinate arrays of the projected markers are sorted according to the size of their horizontal coordinates to obtain the correspondence between the real-shot markers and the projected markers.

6. The two-wheeler camera extrinsic parameter networking calibration method of claim 1, wherein, The step of calibrating the extrinsic parameters of the vehicle camera based on the correspondence and the intrinsic parameters of the vehicle camera includes: Based on the aforementioned correspondence and the intrinsic parameters of the vehicle camera, an extrinsic parameter calibration operation is performed, and the calibrated extrinsic parameters are used to replace the preset camera extrinsic parameters.

7. The two-wheeler camera extrinsic parameter networking calibration method according to any one of claims 1-6, characterized in that, The method further includes: The difference between the pixel coordinates of the simulated marker and the pixel coordinates of the actual marker is compared to verify the calibration results of the external parameter calibration.

8. A two-wheeled vehicle camera extrinsic parameter networking calibration system, characterized in that, The system is applied to any two-wheeled vehicle in a group of multiple two-wheeled vehicles, each including at least one marker point; the two-wheeled vehicle camera extrinsic parameter network calibration system includes: The sending module is used to broadcast its own marker location and direction of travel information packets to other two-wheeled vehicles in the same two-wheeled vehicle group; The receiving module is used to receive marker location and direction of travel information packets sent by other two-wheeled vehicles in the two-wheeled vehicle group; the marker location and direction of travel information packets include the vehicle ID, timestamp, vehicle marker coordinates and vehicle direction of travel; The image receiving module acquires real-time images based on the vehicle's camera, records the shooting timestamp, and sends the real-time images to the computing module; The parsing module is used to extract real-shot version markers of other two-wheeled vehicles from the real-shot image, and parse the marker position and travel direction information packet of the nearest vehicle whose timestamp is closest to the time of the real-shot image, to obtain a list of neighboring two-wheeled vehicle markers corresponding to the real-shot image; the list of neighboring two-wheeled vehicle markers includes multiple simulated markers; The calculation module is used to calculate the coordinates of each simulated marker in the list of adjacent two-wheeled vehicle markers in the vehicle body coordinate system with the vehicle camera as the origin, and to estimate the coordinates of the simulated markers in the pixel coordinate system based on preset camera extrinsic and intrinsic parameters. The judgment module is used to determine whether the automatic calibration conditions are met based on the number of the actual shooting version markers and the number of the deduced version markers. If the automatic calibration conditions are met, the correspondence between the actual shooting version markers and the deduced version markers is determined. The calibration module is used to calibrate the extrinsic parameters of the vehicle camera based on the correspondence and the intrinsic parameters of the vehicle camera.

9. A two-wheeled vehicle characterized by comprising: The two-wheeled vehicle is equipped with the two-wheeled vehicle camera extrinsic parameter networking calibration system as described in claim 8.

10. A group of two-wheeled vehicles characterized in that, The group of two-wheeled vehicles includes multiple two-wheeled vehicles as described in claim 9.

Images