Systems and methods for automated external calibration of lidar, camera, radar and ultrasonic sensors mounted on vehicles and robots
The automated external calibration system addresses the complexity and inaccuracies of manual sensor calibration by using a turntable and calibration target device to capture 3D images and adjust positional attributes, ensuring precise and efficient sensor alignment.
Patent Information
- Application Number
- JP2024522018
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-11-08
- Filing Date
- 2022-11-07
- Publication Date
- 2024-11-26
AI Technical Summary
Current external sensor calibration processes for vehicles and robots are complex, requiring significant manual labor and are prone to inaccuracies, which can lead to safety issues in driver assistance systems.
A system and method for automated external calibration of lidar, camera, and ultrasonic sensors using a turntable system with a calibration target device and imaging system to capture three-dimensional images, identify markers, and adjust positional attributes for precise calibration.
Enables rapid, highly accurate sensor calibration with minimal user intervention, simplifying the process and reducing the risk of errors, ensuring optimal functionality of advanced driver assistance systems.
Smart Images

Figure 2024543784000001_ABST
Abstract
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of and priority to U.S. Provisional Patent Application No. 63 / 276,823, filed November 8, 2021, the entire disclosure of which is incorporated herein by reference for any purpose. [Technical field]
[0002] Embodiments of the present disclosure relate generally to data processing systems, and more particularly to data processing systems and methods suitable for automatically performing external calibration of lidar, camera, radar, and ultrasonic sensors mounted on vehicles and robots. [Background technology]
[0003] Today's consumer vehicles are routinely equipped with sensors such as cameras, lidar, radar, and ultrasonic sensors. Many of the major Advanced Driver Assistance (ADAS) applications are enabled by these sensors, such as Backup Monitoring, Lane Keep Assist, Lane Departure Warning, Lane Centering, Automatic Emergency Braking, Forward Collision Warning, Pedestrian and Cyclist Emergency Braking, Adaptive Cruise Control, and Blind Spot Monitoring. In addition, such sensors are also used in many robotics applications, such as warehouse automation, autonomous driving, and unmanned aerial vehicles.
[0004] One of the requirements for a sensor system to function properly is precise external and internal calibration. Internal calibration is required to understand the relationship between the transmitters, detectors, and pixels inside the system and the direction of the input signals that ultimately form a 2D or 3D world image. External calibration, on the other hand, is performed to precisely perform the spatial transformation between the sensor and the vehicle or robot body frame to which the sensor is rigidly mounted. The resulting extrinsic parameters change whenever the sensor is moved, reinstalled, or repositioned relative to the vehicle frame. For example, the extrinsic parameters of a camera mounted on a vehicle's windshield change when the windshield is replaced or reinstalled. Also, a change in vehicle height due to replacement of larger wheels may cause the sensor to see the road from a higher ground clearance, which will require an update to the external calibration. These are just a few examples of situations where an external calibration of a sensor is required.
[0005] Inaccurate calibration can lead to catastrophic driver assistance events: For example, a forward-facing camera or lidar calibration that is off by just two degrees can cause a vehicle 200 feet ahead to be mistakenly identified as being in the adjacent lane, potentially causing a crash.
[0006] However, even with the latest solutions on the market, the process of external static sensor calibration is still highly manual. A typical solution requires the following steps: Measure the vehicle centerline using a plumb line and a laser pointer or line marker; Using multiple tape measures to measure precise distances, an adjustable fixture is manually positioned at specific distances in front of the vehicle; Using wheel clamps and a manual dot laser type system, align the fixture so that it is perpendicular to the vehicle centerline. Manually adjust the fixture height to the exact height required for the specific vehicle calibration procedure; Manually place the printed stationary target in the fixture with the correct orientation, Ensure that the area behind the calibration area is well lit and clear of clutter, and Ensure there is enough work area in a well-lit area with a completely level ground.
[0007] Each of these requirements makes the calibration process very complicated. Meeting these requirements requires many hours of human intervention and relies heavily on the technician's measurement skills and attention to detail. Even the slightest mistake or omission could result in an inaccurate calibration, which could have serious consequences for the camera-based driver assistance application. Summary of the Invention [Problem to be solved by the invention]
[0008] In view of the above, there is a need for improved systems and methods for automated exo-calibration of vehicle and robot-mounted lidar, camera, radar, and ultrasonic sensors that overcome the above-mentioned barriers and deficiencies of currently available systems. [Means for solving the problem]
[0009] The present disclosure relates to a system for calibrating a sensor system associated with a device under test, and methods for making and using the same. The sensor calibration system may include a turntable system for supporting and rotating the device under test relative to at least one calibration target system, and one or more imaging systems distributed around a periphery of the turntable system. The calibration target system may include a calibration target device having calibration indicia, and a calibration target positioning system for positioning the calibration target device relative to the sensor system. Meanwhile, the imaging system may capture images of the device under test while the turntable system rotates the device under test. In certain embodiments, the calibration target system may advantageously calibrate a sensor system supporting one or more advanced driver assistance (ADAS) and / or autonomous vehicle (AV) applications associated with a passenger or commercial vehicle.
[0010] According to a first aspect of the present disclosure, a method for automatically calibrating a sensor system associated with a device under test is provided, the method comprising: scanning for devices under test to identify the devices under test; configuring a calibration target system to cooperate with the identified device under test; and / or The method may include a step of calibrating the sensor system using the set calibration target system.
[0011] In some embodiments of the method according to the first aspect of the present disclosure, the step of scanning the device under test may include capturing a three-dimensional image of the device under test and / or identifying the device under test based on the captured three-dimensional image. Optionally, the step of capturing a three-dimensional image may include rotating the device under test relative to the imaging means and / or capturing a three-dimensional point cloud registration result image of the rotating device under test by the imaging means. The step of capturing a three-dimensional point cloud registration result image may include, for example, capturing a sparse three-dimensional point cloud registration result image of the rotating device under test by the imaging means and / or capturing a dense three-dimensional point cloud registration result image of the rotating device under test by the imaging means. In certain embodiments, capturing the 3D point cloud registration result image may include recording multiple encoder angles of a turntable system for rotating the device under test, capturing a sequence of image frames of the rotating device under test, generating multiple rigid body transformation results from the captured image frames and the recorded encoder angles corresponding to each image frame, and / or fitting the generated rigid body transformation results into a preselected coordinate system to provide the 3D point cloud registration result image.
[0012] Additionally or alternatively, identifying the device under test may include extracting at least one device marker from the captured three-dimensional image and / or identifying the device under test based on the extracted device marker. Optionally, extracting at least one device marker may include extracting a device mirror, a device bumper, a device wheel, a device axle center, a device logo, a device wheel center, a device thrust line, a device door, and a device pillar from the captured three-dimensional image. In certain embodiments, extracting at least one device marker may include training a machine learning framework with device data for a plurality of devices under test of different types and / or extracting at least one device marker from the captured three-dimensional image with the trained machine learning framework.
[0013] For example, identifying the device under test may include identifying the sensor system based on the captured three-dimensional image.
[0014] In some embodiments of the method according to the first aspect of the present disclosure, the step of configuring the calibration target system may include selecting a calibration target device having calibration indicia suitable for calibrating the sensor system of the identified device under test, positioning the selected calibration target device in a calibration target positioning system of the calibration target system, and / or setting at least one position attribute of the selected calibration target device relative to the sensor system by the calibration target positioning system. Optionally, the step of selecting the calibration target device may include selecting the calibration target device from among a plurality of calibration target devices having different calibration indicia.
[0015] Additionally or alternatively, setting at least one position attribute of the selected calibration target device may include radially translating and / or three-dimensionally rotating the calibration target positioning system relative to the device under test. In certain embodiments, setting the calibration target system may include determining a travel path that moves the calibration target system to a state of the set at least one position attribute while avoiding collisions between the selected calibration target device and the device under test. Optionally, determining the travel path may also include solving forward and inverse kinematics of the calibration target system and a turntable system for rotating the device under test. In certain embodiments, setting at least one position attribute may include setting, by the calibration target positioning system, at least one orientation attribute of the selected calibration target device relative to the sensor system.
[0016] In some embodiments of the method according to the first aspect of the present disclosure, the step of setting the calibration target system comprises: creating a virtual sensor calibration environment by placing, by a processing means, a virtual calibration target device adjacent to a virtual device under test, the virtual device under test being associated with the virtual sensor system and comprising a type of device under test; A step of simulating an external calibration process for the virtual sensor system using the constructed virtual sensor calibration environment; adjusting, by the processing means, at least one three-dimensional position attribute of the virtual calibration target device based on the step of simulating the external calibration process; and / or The calibration target system may include a step of configuring the calibration target system by positioning a calibration target device associated with the calibration target system relative to the device under test according to at least one three-dimensional position attribute of the adjusted virtual calibration target device.
[0017] In some embodiments of the method according to the first aspect of the present disclosure, calibrating the sensor system may include calibrating an advanced driver assistance system (ADAS) sensor system located on a passenger vehicle and / or a commercial vehicle with the configured calibration target system.
[0018] In some embodiments of the method according to the first aspect of the present disclosure, calibrating the sensor system may include calibrating an autonomous vehicle (AV) sensor system located on a passenger vehicle and / or a commercial vehicle with the configured calibration target system.
[0019] According to a second aspect of the present disclosure, there is provided a system for automatically calibrating a sensor system associated with a device under test, the system comprising means for performing embodiments of the method according to the first aspect.
[0020] According to a third aspect of the present disclosure, a computer program product for automatically calibrating a sensor system associated with a device under test is provided. The computer program product may include instructions for performing embodiments of the method according to the first aspect. Optionally, the computer program product according to the third aspect may be encoded on one or more non-transitory machine-readable storage media.
[0021] According to a fourth aspect of the present disclosure, there is provided a computer program product for automatically calibrating a sensor system associated with a device under test, the computer program product comprising: instructions to scan for devices under test to identify the devices under test; instructions for configuring a robotic calibration target system to cooperate with the identified device under test; and / or The calibration target system may include instructions to calibrate the sensor system using the set calibration target system.
[0022] Also, optionally, the computer program product according to the fourth aspect may be encoded on one or more non-transitory machine-readable storage media.
[0023] According to a fifth aspect of the present disclosure, there is provided a system for automatically calibrating a sensor system associated with a device under test, the system comprising: a central turntable system for rotating the device under test; an articulated robotic calibration target system having an end effector member for mounting a calibration target device; and / or The rotary table system may include a first imaging means and a second imaging means configured to scan the device under test to generate a three-dimensional image of the device under test while the rotary table system rotates the device under test, and the first imaging means, the second imaging means and the robotic calibration target system are positioned around the periphery of the rotary table system. Additionally or alternatively, the system may comprise control means for identifying a device under test based on the three-dimensional image and configuring the robotic calibration target system to cooperate with the identified device under test; The sensor system is calibrated using the set calibration target system.
[0024] In some embodiments of the system according to the fifth aspect of the present disclosure, the articulated robotic calibration target system may have between 3 and 9 degrees of freedom, and optionally, the articulated robotic calibration target system may include at least one rotary joint member, at least one prismatic joint member, or both.
[0025] In some embodiments of the system described in the fifth aspect of the present disclosure, the first imaging means and the second imaging means are each selected from a group of imaging means consisting of camera imaging means, light detection and ranging (lidar) imaging means, radio detection and ranging (radar) imaging means, and ultrasound imaging means.
[0026] In some embodiments of the system described in the fifth aspect of the present disclosure, the first imaging means and the robotic calibration target system may be positioned in a first plane passing through a central region of the rotating table system, and the second imaging system may be positioned in a second plane perpendicular to the first plane passing through the central region of the rotating table system. [Brief description of the drawings]
[0027] [Figure 1] FIG. 1 is a schematic block diagram illustrating an exemplary embodiment of an automated external calibration system for a device under test associated with a sensor system disposed on a rotating platform system; [Figure 2A] FIG. 2 is a schematic block diagram illustrating an alternative exemplary embodiment of the automated external calibration system shown in FIG. 1, where the automated external calibration system includes a robotic arm for positioning a first selected calibration target device at a first predetermined distance from the device under test. [Figure 2B] FIG. 2B is a schematic block diagram illustrating an alternative exemplary embodiment of the automated external calibration system shown in FIG. 2A, in which a robotic arm positions a second selected calibration target device at a second predetermined distance from the device under test; [Diagram 3] FIG. 2 is a detailed diagram illustrating another exemplary alternative embodiment of the automated external calibration system shown in FIG. 1, in which the automated external calibration system comprises a pair of light sensing and ranging (LIDAR) systems positioned around the periphery of the rotating platform system. [Figure 4A] 2 is a schematic flow chart illustrating an exemplary embodiment of a method for automatic external calibration of a device under test shown in FIG. [Figure 4B] FIG. 4B is a schematic flow chart illustrating an exemplary alternative embodiment of the automatic external calibration method shown in FIG. 4A, in which the automatic external calibration method includes scanning the device under test. [Figure 5A] FIG. 4C is a schematic flow chart illustrating an exemplary alternative embodiment of the automatic external calibration method shown in FIG. 4B, in which the automatic external calibration method includes identifying the device under test. [Figure 5B]FIG. 4B is a schematic flow chart illustrating another exemplary alternative embodiment of the automatic external calibration method shown in FIG. 4B, in which the automatic external calibration method includes positioning a selected calibration target device relative to a device under test. [Figure 6A] FIG. 2 is a top detailed view of yet another exemplary alternative embodiment of the automatic external calibration system shown in FIG. 1, in which the automatic external calibration system is configured to calibrate a sensor system associated with a passenger vehicle. [Figure 6B] Detailed side view of the automated external calibration system shown in Figure 6A. [Figure 7] FIG. 2 is a detailed diagram illustrating yet another exemplary alternative embodiment of the automatic external calibration system shown in FIG. 1, in which the automatic external calibration system is configured to perform static sensor system calibration. [Figure 8A] FIG. 2C is a detailed diagram illustrating an alternative exemplary embodiment of the automated external calibration system shown in FIGS. 2A and 2B, in which the calibration target device comprises a calibration target device having a first calibration indicia for calibrating a camera imaging system associated with a device under test; [Figure 8B] FIG. 8B is a detailed view of an exemplary alternative embodiment of a calibration target device for calibrating the camera imaging system shown in FIG. 8A, in which the calibration target device has a second calibration indicia for calibrating the camera imaging system; [Figure 9A] FIG. 2C is a detailed view of another exemplary alternative embodiment of the automated external calibration system shown in FIGS. 2A and 2B, in which the calibration target device comprises a calibration target device having a third calibration indicia for calibrating a lidar imaging system associated with the device under test. [Figure 9B] FIG. 9B is a detailed view of an example alternative embodiment of a calibration target device for calibrating the LIDAR imaging system shown in FIG. 9A, in which the calibration target device has a fourth calibration indicia for calibrating the LIDAR imaging system; [Figure 10A] FIG. 2C is a detailed diagram illustrating yet another exemplary alternative embodiment of the automatic external calibration system shown in FIGS. 2A and 2B, in which the calibration target device comprises a calibration target device having a fifth calibration indicia for calibrating a radar imaging system associated with a device under test. [Figure 10B] FIG. 10B is a detailed view of an alternative exemplary embodiment of a calibration target device for calibrating the radar imaging system shown in FIG. 10A, in which the calibration target device has a sixth calibration indicia for calibrating the radar imaging system; [Figure 11A] FIG. 2 is a detailed top view of yet another exemplary alternative embodiment of the automatic external calibration system shown in FIG. 1, in which the automatic external calibration system is configured to calibrate a sensor system associated with a front region of a device under test; [Figure 11B] FIG. 2 is a top detailed view of yet another exemplary alternative embodiment of the automatic external calibration system shown in FIG. 1, in which the automatic external calibration system is configured to calibrate a sensor system associated with a side region of a device under test; [Figure 11C] FIG. 2 is a top detailed view of yet another exemplary alternative embodiment of the automatic external calibration system shown in FIG. 1, in which the automatic external calibration system is configured to calibrate a sensor system associated with a rear region of a device under test; [Figure 12]FIG. 4 is a detailed diagram illustrating an embodiment of a control system for the automated external calibration system shown in FIG. [Figure 13] FIG. 13 is a detailed diagram illustrating an embodiment of a 3D reconstruction system capable of reconstructing a point cloud registration result image of a device under test in the control system illustrated in FIG. [Figure 14] FIG. 13 is a detailed diagram of one embodiment of a machine learning system capable of deciphering one or more aspects and / or segments of a device under test based on a point cloud registration result image in the control system shown in FIG. [Figure 15] FIG. 13 is a detailed diagram of one embodiment of a target placement simulation system that can be used to design a calibration procedure for a particular type of device under test and sensor system in the control system shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0028] It should be noted that the drawings are not to scale, and that for convenience of explanation, elements having similar structures or functions may be collectively represented by similar reference numerals throughout the drawings. It should also be noted that the drawings are merely intended to facilitate explanation of the preferred embodiments. The drawings do not depict all aspects of the embodiments described herein, and are not intended to limit the scope of the present disclosure.
[0029] Given the complex manual nature of conventional extrinsic static sensor calibration processes, a need exists for a system and method for performing automated extrinsic calibration of lidar, camera, radar, and ultrasonic sensors that can provide the basis for a wide range of applications, such as the calibration of sensors associated with vehicles and / or robots. According to one embodiment disclosed herein, such an effect can be achieved by a sensor calibration system 100 shown in FIG.
[0030] 1, the illustrated sensor calibration system 100 includes a turntable system 110 for supporting and / or rotating a device under test 200. The device under test 200 may be associated with a sensor system (or sensor means) 210. Exemplary devices under test 200 may include, but are not limited to, passenger vehicles, autonomous vehicles, unmanned aerial vehicles, and robotic systems. The sensor system 210 may be located at any suitable location on the device under test 200. In certain embodiments, the sensor system 210 may be located on a preselected surface of the device under test 200 and / or at least a portion of the sensor system 210 may be located within a preselected area of the device under test 200. The sensor system 210 may be located, for example, on a roof, bumper, grill, side mirror, rearview mirror, or hood of the passenger vehicle.
[0031] Exemplary sensor system 210 may include, but is not limited to, one or more camera systems, one or more Light Detection and Ranging (LIDAR) systems, one or more Radio Detection and Ranging (RADAR) systems, and / or one or more ultrasonic systems. For example, in the case of a sensor system 210 associated with a passenger vehicle or commercial vehicle, it may support one or more Advanced Driver Assistance System (ADAS) applications, such as backup monitoring, lane keeping assist, lane departure warning, lane centering, automatic emergency braking, forward collision warning, pedestrian and bicycle emergency braking, speed and distance control, blind spot monitoring, and the like. Additionally or alternatively, sensor system 210 may comprise an Autonomous Vehicle (AV) sensor system that may be associated with any suitable type of device under test 200, such as a passenger vehicle, commercial vehicle, or the like.
[0032] The sensor calibration system 100 has the advantage of being able to perform external calibration of the sensor system 210. External calibration of the device under test 200, the sensor system 210, or both may be performed at any suitable time. For example, the sensor calibration system 100 may perform an initial external calibration upon initial installation of the sensor system 210 to the device under test 200 (when the sensor system 210 is installed as an original equipment (OE) item or when the sensor system 210 is added as an aftermarket item). Optionally, the sensor calibration system 100 may also support time-based external calibration, such as periodic external calibration of the sensor system 210, for example, external calibration that is performed as part of scheduled maintenance of the device under test 200. Additionally or alternatively, the sensor calibration system 100 may also support event-based external calibration of the sensor system 210, for example, external calibration after the device under test 200 or the sensor system 210 has been damaged, such as by being involved in a crash event.
[0033] The turntable system 110 may include a turntable motor system 116 (shown in FIG. 6A ) for rotating the test area 112. During external calibration, the device under test 200 may be placed in the test area 112. If the test area 112 includes a concave test surface or an elevated test surface as shown in FIG. 1 , the turntable system 110 may include a tilt area 114 such as a ramp, so that the device under test 200 can be easily placed on the test area 112. The tilt area 114 may be provided separately from the turntable system 110, or at least a part of the tilt area 114 may be provided integrally with the turntable system 110. The dimensions of the test area 112, such as the size, diameter, shape, etc., may preferably be such that various devices under test 200 having different shapes, sizes, and masses can be accommodated.
[0034] The sensor calibration system 100 may also include at least one calibration target system 120. The calibration target system 120 may be disposed adjacent to the rotating bed system 110. The calibration target system 120 may be associated with one or more calibration target devices 122. Each calibration target device 122 may have at least one calibration indicia 124, such as a predefined grid pattern. At least one calibration target device 122 may be selected for mounting to a calibration target positioning system 126. In certain embodiments, the calibration target devices 122 may have calibration indicia 124 associated with each of the devices under test 200 and / or sensor systems 210 such that the calibration indicia 124 of the selected calibration target device 122 are suitable for calibrating the devices under test 200 and / or sensor systems 210 disposed on the rotating bed system 110. Optionally, the calibration target device 122 may also have calibration indicators 124 associated with two or more devices under test 200 and / or sensor systems 210 so that the calibration target device 122 is suitable for use in calibrating two or more devices under test 200 and / or sensor systems 210.
[0035] 1, the calibration target positioning system 126 can position the selected calibration target device 122 such that the selected calibration target device 122 has predetermined relative position attributes such as distance D, altitude, orientation, angle, etc. with respect to the device under test 200 and / or the sensor system 210. In other words, the calibration target positioning system 126 can raise and lower the selected calibration target device 122 relative to the rotational table system 110, the device under test 200 or the sensor system 210 disposed on the rotational table system 110, or any combination thereof. Additionally or alternatively, the calibration target positioning system 126 can rotate the selected calibration target device 122 horizontally, diagonally, and / or vertically relative to the rotational table system 110, the device under test 200 or the sensor system 210 disposed on the rotational table system 110, or any combination thereof. Additionally or alternatively, the calibration target positioning system 126 may translate the selected calibration target device 122 relative to the rotate bed system 110, the device under test 200 or the sensor system 210 disposed on the rotate bed system 110, or any combination thereof. In certain embodiments, the calibration target positioning system 126 itself may undergo relative movement, such as translation, with respect to the device under test 200 and / or the sensor system 210 to position the selected calibration target device 122 such that a predetermined position attribute, such as distance D, is achieved. The calibration target positioning system 126 may, for example, be translated radially toward and / or away from the rotate bed system 110.
[0036] The translational movement of the selected calibration target device 122 may include, for example, moving the selected calibration target device 122 toward and / or away from the rotate bed system 110, the device under test 200 or the sensor system 210 disposed on the rotate bed system 110, or a combination thereof. In certain embodiments, the calibration target positioning system 126 may position the selected calibration target device 122 such that at least one position attribute of the selected calibration target device 122 is relative to a predefined point or location of the device under test 200 and / or the sensor system 210. Also, optionally, the calibration target positioning system 126 may adjust at least one position attribute, such as the distance D, within a preselected range of position attribute values.
[0037] 1, the calibration target positioning system 126 may include a positioning system base 128 for supporting the calibration target positioning system 126, and a positioning system member 127 for mounting the selected calibration target device 122 to the positioning system base 128. The positioning system base 128 may be an elongated base having a proximal end region adjacent to the turntable system 110 and a distal end region remote from the turntable system 110. Stated slightly differently, the proximal end region of the positioning system base 128 may be located between the distal end region of the positioning system base 128 and the turntable system 110. This allows the positioning system member 127 to move back and forth between the proximal end region and the distal end region of the positioning system base 128 to adjust the distance D between the selected calibration target device 122 and the turntable system 110, the device under test 200, and / or the sensor system 210.
[0038] In a particular embodiment, the calibration target positioning system 126 can automatically position the selected calibration target device 122 such that at least one position attribute of the selected calibration target device 122 is determined relative to the device under test 200 and / or the sensor system 210. For example, the calibration target positioning system 126 can include a robot. In that case, as shown in FIGS. 2A and 2B, the positioning system member 127 includes a robot arm 125. Referring to FIG. 2A, the robot arm 125 shown in FIG. 2A is configured as a multi-joint robot arm having a plurality of arm segment members 125A and arm joint members 125B disposed between each pair of adjacent segment members 125A. In a particular embodiment, at least one of the arm joint members 125B can be configured as at least one of a rotary joint member, a rotational joint member, and a prismatic joint member. The proximal arm segment member 125A may be coupled to the positioning system base 128 either directly or via an arm articulation member 125B as shown in Figure 2A, while the distal arm segment member 125A may include an end effector member 125C for mounting a selected calibration target device 122.
[0039] The sensor calibration system 100 has the advantage of being highly versatile and capable of supporting a wide range of calibration procedures. For example, the sensor calibration system 100 can perform a calibration procedure for a camera-, radar-, and / or lidar-based sensor system 210. The sensor calibration system 100 can also be programmably controlled to additionally or alternatively perform a calibration procedure for any device under test 200 associated with any brand, type, or other identification attribute.
[0040] For example, FIGS. 2A and 2B show the sensor calibration system 100 accommodating two different devices under test 200A and 200B, respectively. A first sensor system 210A of the first device under test 200A may be different from a second sensor system 210B of the second device under test 200B. For example, a first sensor type of the first sensor system 210A may be different from a second sensor type of the second sensor system 210B. Additionally or alternatively, the first sensor system 210A may be located at a first position of the first device under test 200A, while the second sensor system 210B may be located at a second position of the second device under test 200B.
[0041] A first sensor type, which is a type of the first sensor system 210A, may be a sensor type that requires selection of a first calibration target device 122A having a first calibration indicia 124A. Similarly, a second sensor type, which is a type of the second sensor system 210B, may be a sensor type that requires selection of a second calibration target device 122B having a second calibration indicia 124B. In certain embodiments, as described in detail above with reference to FIG. 1, the calibration target positioning system 126 determines the distance D of the first calibration target device 122A relative to the first device under test 200A and / or the first sensor system 210A. A The first calibration target device 122A may be positioned such that a first position attribute, such as altitude, orientation, angle, etc., is predetermined.
[0042] The first position attribute of the first calibration target device 122A can be based, for example, on a first sensor type, a first location that is the location of the first sensor system 210A, or both. Similarly, as described above, the calibration target positioning system 126 may determine the distance D of the second calibration target device 122B relative to the second device under test 200B and / or the second sensor system 210B. B The second calibration target device 122B may be positioned such that a second location attribute, such as a distance, an altitude, an orientation, an angle, etc., is predetermined. For example, the second location attribute of the second calibration target device 122B may be based on a second sensor type, a second location, which is the location of the second sensor system 210B, or both. A and a predetermined second distance D B may comprise any suitable predetermined distance. Exemplary predetermined distances may include, but are not limited to, one or more predetermined distances (e.g., 1500 millimeters, 3000 millimeters) or at least one predetermined distance range (e.g., a range between 1500 millimeters and 3000 millimeters) (including any preselected sub-ranges within the predetermined distance range).
[0043] Referring again to FIG. 1, one or more imaging systems (or imaging means) 130 are shown disposed adjacent to the rotating bed system 110. Exemplary imaging systems 130 may include, but are not limited to, a lidar imaging system (or lidar imaging means), a radar imaging system (or radar imaging means), and a camera system (or camera means). In certain embodiments, the lidar imaging system may include a high-precision 3D lidar imaging system or an industrial grade 2D lidar imaging system, while the camera imaging system may include a high-fidelity camera. The imaging systems 130 are preferably homogenous imaging systems, but may include at least one different imaging system in certain embodiments. The calibration target system 120 and the imaging systems 130 may be distributed around the periphery of the rotating bed system 110 in any suitable manner. The imaging systems 130 may be positioned such that the relative position attributes, such as distance, elevation, orientation, angle, etc., of each imaging system 130 are predetermined with respect to the device under test 200 and / or the sensor system 210.
[0044] The turntable system 110 may rotate the device under test 200 and / or the sensor system 210 relative to the calibration target system 120 and the imaging system 130 during the external sensor calibration process. If desired, the turntable system 110 may rotate the device under test 200 and / or the sensor system 210 in a clockwise direction, a counterclockwise direction, or both, about a central axis of rotation of the turntable system 110. The turntable system 110 may also rotate the device under test 200 and / or the sensor system 210 one or more full revolutions, or may additionally or alternatively rotate the device under test 200 and / or the sensor system 210 up to a fraction of a revolution. The direction and amount of rotation may be based at least in part on the requirements of the external sensor calibration process. In certain embodiments, while the turntable system 110 rotates the device under test 200 and / or the sensor system 210, at least one of the calibration target system 120 and the imaging system 130 can be kept stationary (not moving) or in motion (moving).
[0045] Preferably, the external sensor calibration process includes an automated external sensor calibration process. The sensor calibration system 100 shown in FIG. 1 includes an optional control system (or control means) 140 for automating, such as by automatic control, the external sensor calibration process. For example, the control system 140 can be configured to control the operation of the rotating bed system 110, the calibration target system 120, the imaging system 130, or any combination thereof. The control system 140 can include a processing system (or processing means), such as any conventional processing system suitable for controlling the rotating bed system 110, the calibration target system 120, the imaging system 130, or any combination thereof, such as a computer server system, a personal computing system, a notebook computing system, a tablet computing system, a mobile phone system, etc. Preferably, the control system 140 can include a high-precision controller-encoder system (or high-precision controller-encoder means) (not shown), which can additionally or alternatively include one or more fast-clocked central processing units (CPUs) and / or graphics processing units (GPUs) with access to the Internet. In certain embodiments, the external sensor calibration process may include a computer-implemented external sensor calibration process.
[0046] The control system 140 can communicate with the rotation table system 110, the calibration target system 120, the imaging system 130, or any combination thereof in any conventional wired and / or wireless manner. For example, the control system 140 can exchange data with the rotation table system 110, the calibration target system 120, the imaging system 130, or any combination thereof, directly or via a computer network (not shown), such as the Internet. As shown in FIG. 3, the control system 140 can include a rotation table control system (or a rotation table control means) 142 for controlling the operation of the rotation table system 110, a calibration target control system (or a calibration target control means) 144 for controlling the operation of the calibration target system 120, and / or an imaging control system (or an imaging control means) (not shown) for controlling the imaging system 130.
[0047] The rotary table control system 142 may include an optional high precision controller-encoder system (or high precision controller-encoder means) 142A (shown in FIG. 13 ), for example, to record encoder angles during rotation of the rotary table system 110. The rotary table control system 142, the calibration target control system 144, and the imaging control system may each comprise separate control systems, or may additionally or alternatively be integrated at least in part into at least one combined control system. In certain embodiments, the control system 140 may include a control station for providing a user interface 148, which may enable a system user (not shown), such as a system operator, to interact with the sensor calibration system 100.
[0048] 3, the sensor calibration system 100 may include a calibration target system 120 and a pair of imaging systems 130 arranged around the periphery of a central turntable system 110. The calibration target system 120 may include a robotic arm 125 having a selected calibration target device 122 mounted on an end effector member 125C. The illustrated imaging system 130 includes a first imaging system 130A and a second imaging system 130B. The first imaging system 130A and the second imaging system 130B may be arranged in any suitable configuration relative to the calibration target system 120.
[0049] The first imaging system 130A is provided, for example, on the opposite side of the central rotating platform system 110 from the calibration target system 120, as shown. In other words, the first imaging system 130A and the calibration target system 120 can be arranged in a plane that includes both ends of the diameter of the central rotating platform system 110 and passes through the center (or central region) 115 (shown in FIG. 6A ) of the rotating platform system 110. Meanwhile, the second imaging system 130B can be arranged at a position perpendicular to the plane in which the first imaging system 130A and the calibration target system 120 are arranged. In other words, stated slightly differently, the second imaging system 130B and the first imaging system 130A can be at right angles with the central region 115 as the center, and in addition or instead, the second imaging system 130B and the calibration target system 120 can be at right angles with the central region 115 as the center.
[0050] 1 to 3, the sensor calibration system 100 is illustrated as including a single calibration target system 120 and a single imaging system 130, but such an illustration is for illustrative purposes only. The sensor calibration system 100 may include any suitable number of calibration target systems 120 and any suitable number of imaging systems 130. The calibration target systems 120 may include a first calibration target system 120 and a second calibration target system 120, and the first calibration target system 120 and the second calibration target system 120 may be the same or different. For example, the first calibration target system 120 may be associated with a first selected calibration target device 122A (shown in FIG. 2A), while the second calibration target system 120 may be associated with a second selected calibration target device 122B (shown in FIG. 2B) that is different from the first selected calibration target device 122A.
[0051] Additionally or alternatively, the imaging system 130 may include a first imaging system 130A (shown in FIG. 3) and a second imaging system 130B (shown in FIG. 3), which may be the same or different. For example, if the first imaging system 130A includes a first LIDAR system, then if the first imaging system 130A and the second imaging system 130B are the same, then the second imaging system 130B may include a second LIDAR system. On the other hand, if the first imaging system 130A and the second imaging system 130B are different, then the second imaging system 130B may include a RADAR system.
[0052] It should be noted that while the sensor calibration system 100 is shown and described as having a central turntable 110 for rotating the device under test 200 and / or the sensor system 210, this is for illustrative purposes only. In certain embodiments, the sensor calibration system 100 may be configured to rotate the calibration target system(s) 120 and / or the imaging system(s) 130 about a stationary device under test 200 and / or sensor system 210.
[0053] In operation, the sensor calibration system 100 may advantageously perform an external calibration method for the sensor system 210. An exemplary external calibration method 300 is illustrated in Figures 4A and 4B. The external calibration method 300 may include instructions for configuring and controlling the sensor calibration system 100. In certain embodiments, the sensor calibration system 100 may be configured and controlled via software that may be executed by the control system 140 (shown in Figure 1). In other words, in certain embodiments, the external calibration method 300 may include a computer-implemented external calibration method.
[0054] 4A, the external calibration method 300 shown in FIG. 4A includes a step 320 of configuring the sensor calibration system 100 (shown in FIG. 1) to calibrate the sensor system 210 (shown in FIG. 1) associated with a target device under test 200 (shown in FIG. 1). Then, in a step 330, the configured sensor calibration system 100 can calibrate the sensor system 210 of the device under test 200. In a particular embodiment, the external calibration method 300 can include an automated external sensor calibration process. In other words, the external calibration method 300 can be automated, such as by automatic control, by a processing system (or processing means) such as the control system 140 (shown in FIG. 1).
[0055] 4B illustrates an alternative embodiment of an external calibration method 300. As shown in FIG. 4B, the external calibration method 300 may include step 310 of scanning a target device under test 200 (shown in FIG. 1) associated with a sensor system 210 (shown in FIG. 1). Then, in step 320, the sensor calibration system 100 may be configured for the device under test 200. In other words, in step 320A, the sensor calibration system 100 may be configured to calibrate the sensor system 210 of the scanned device under test 200. Then, in step 330A, the configured sensor calibration system 100 may calibrate the sensor system 210 of the scanned device under test 200.
[0056] In step 310, the target device under test 200 may be scanned in any suitable manner. With reference to FIG. 5A, for example, in step 312, the target device under test 200 may be scanned by placing the target device under test 200 on the turntable system 110 (shown in FIG. 1). In a particular embodiment, the target device under test 200 may be placed on the test area 112 (shown in FIG. 1) of the turntable system 110. Then, in step 314, the turntable system 110 with the target device under test 200 thereon may be rotated at a predetermined rotational speed to capture a three-dimensional image of the target device under test 200. In step 314, the three-dimensional image of the target device under test 200 may be captured, for example, by the imaging system(s) 130 (shown in FIG. 1).
[0057] When the imaging system 130 includes a 3D LIDAR imaging system, the rotational platform system 110 can preferably rotate at a constant slow rotational speed. As the target device under test 200 rotates with the rotational platform system 110, the 3D LIDAR imaging system can construct, or otherwise image, a complete 3D point cloud registration result image 136 (shown in FIG. 13 ) of the target device under test 200. Exemplary rotational speeds include, but are not limited to, any rotational speed within a rotational speed range of 1 / 20 revolutions per second to 10 revolutions per second. In certain embodiments, the rotational speed can include a rotational speed within a rotational speed range of 1 / 10 revolutions per second to 4 revolutions per second. Capturing the 3D point cloud registration result image can optionally include capturing a sparse 3D point cloud registration result image of the rotating device under test 200 and can additionally or alternatively include capturing a dense 3D point cloud registration result image of the rotating device under test 200. Optionally, filtering may also be used to remove high noise points in the 3D point cloud registration result image 136 of the target device under test 200.
[0058] Then, in step 316, the device under test 200 can be identified based on the captured three-dimensional image of the device under test 200. The device under test 200 can be identified, for example, by extracting device components 220 (shown in FIG. 14 ), such as one or more associated markers, from the captured three-dimensional image of the device under test 200. Exemplary markers when the target device under test 200 is configured in a passenger vehicle can include, but are not limited to, bumpers, side mirrors, wheel centers, axle centers, vehicle logos, vehicle thrust lines, doors, and pillars. Based on the extracted markers, device identification (or device attribute) information, such as manufacturer and model, of the target device under test 200 can be identified. Optionally, sensor identification information, such as sensor type and sensor location of the sensor system 210 can also be identified based on the extracted markers, device identification information, the identified device under test 200, or any combination thereof.
[0059] Once the target device under test 200 has been identified, the sensor calibration system 100 (shown in FIG. 1) can be configured for the target device under test 200 in step 320A (shown in FIG. 4B). In other words, the sensor calibration system 100 can be configured to calibrate the sensor system 210 of the scanned device under test 200. As shown in FIG. 5B, once the target device under test 200 has been identified, for example, at least one calibration target device 122 (shown in FIG. 1) can be selected for mounting on the calibration target positioning system 126 (shown in FIG. 1) in step 322. Then, the selected calibration target device 122 can be placed on the calibration target positioning system 126 (shown in FIG. 1) in step 324.
[0060] In certain embodiments, the selected calibration target device 122 may be mounted on the calibration target positioning system 126 in the manner described in more detail above with reference to Figures 1, 2A and 2B. Then, in step 326, the calibration target positioning system 126 may position the selected calibration target device 122 relative to the device under test 200 disposed on the turntable system 110. In certain embodiments, the calibration target positioning system 126 may position the selected calibration target device 122 such that relative position attributes such as distance D, altitude, orientation, angle, etc. of the selected calibration target device 122 are predefined with respect to the device under test 200 and / or the sensor system 210, as described in more detail above with reference to Figures 1, 2A and 2B.
[0061] Returning now to FIG. 4B, at step 330A, after the selection and positioning of the calibration target device 122, the sensor system 210 of the scanned device under test 200 may be calibrated. For example, the control system 140 (shown in FIG. 1) may automate, such as by automatic control, the external sensor calibration process. The control system 140 may be configured to control the operation of the turntable system 110 (shown in FIG. 1), the calibration target system 120 (shown in FIG. 1), the imaging system 130 (shown in FIG. 1), or any combination thereof. In certain embodiments, the control system 140 may adjust at least one position attribute, path planning, velocity, or any combination thereof, of the calibration target system 120 when calibrating the sensor system 210 at step 330A.
[0062] The acquisition of the calibration data can be initiated manually and / or automatically at the device under test 200. In certain embodiments, the device under test 200 can image and / or store the calibration data for the calibration of the sensor system 210. The calibration procedure of the sensor system 210 can be performed in real time at the device under test 200 or can additionally or alternatively be uploaded for later execution and verification of the calibration procedure of the sensor system 210. For example, the calibration procedure can be uploaded to the device under test 200 or can be uploaded to a separate data storage system (or data storage means) (not shown), such as the cloud. Once the calibration procedure is completed, the device under test 200 can be removed from the turntable system 110 and the sensor calibration system 100 can be returned to a default state.
[0063] Thus, the sensor calibration system 100 and / or external calibration method 300 can perform external calibration or recalibration of the sensor system 210 as needed. The sensor calibration system 100 and / or external calibration method 300 can quickly perform a highly accurate factory level calibration of the sensor system 210 to restore or maintain optimal functionality with minimal or no user intervention. The sensor calibration system 100 and / or external calibration method 300 has the advantage of being able to utilize a highly accurate calibration target system 120 and imaging system 130 to perform highly accurate calibration procedures for a variety of sensor systems 210 and devices under test 200.
[0064] In a particular embodiment, the sensor calibration system 100 can calibrate a sensor system 210 associated with a passenger vehicle, such as a car, truck, van, etc. The sensor calibration system 100 shown in Figures 6A and 6B is configured to calibrate a sensor system 210 (shown in Figure 1) associated with a passenger vehicle. With reference to Figure 6A, the illustrated sensor calibration system 100 includes a calibration target system 120 and a pair of imaging systems 130 arranged around the periphery of a central rotating platform system 110 in the manner described in detail above with reference to Figure 3. The calibration target system 120 can include, for example, a robotic arm 125 having an end effector member 125C for mounting a selected calibration target device 122 (shown in Figure 3). As shown in Figure 6A, the imaging system 130 can include a first imaging system 130A and a second imaging system 130B. The first imaging system 130A and the second imaging system 130B may comprise, for example, a LIDAR imaging system and may additionally or alternatively be arranged in any suitable configuration relative to the calibration target system 120.
[0065] The illustrated rotating platform system 110 includes a test area 112. The test area 112 has a predetermined diameter W that is capable of supporting, rotating, etc., accommodating a passenger vehicle. T It has a given diameter W T can be in the diameter range of 1 meter to 10 meters, or greater than 10 meters in diameter. In certain embodiments, the predetermined diameter W T can be within a preselected partial diameter range within the above diameter range, for example, within a preselected partial diameter range of 4 meters to 6 meters (range width: 2 meters), or within a preselected partial diameter range of 4 meters to 8 meters (range width: 4 meters). TPreferably, the vehicle hub 100 may be appropriately sized to accommodate a variety of passenger vehicles having different shapes, sizes, and masses.
[0066] The periphery of the test area 112 may be surrounded by the test apron area 118, such as by having the test apron area 118 surround at least a portion of the periphery of the test area 112. In other words, in certain embodiments, the test apron area 118 may be concentric with the turntable system 110. The test apron area 118 may extend beyond a predetermined diameter W of the test area 112. T and a predetermined outer diameter W that is greater than the predetermined inner diameter of the test apron region 118. AC The width of the test apron area 118 can be in the range of 1 meter to 10 meters, or greater than 10 meters. In certain embodiments, the predetermined outer diameter W AC can be within a preselected sub-diameter range within the above diameter range, for example, from the above-mentioned predetermined inner diameter to the above-mentioned predetermined outer diameter W of the test apron region 118. AC For example, the predetermined diameter W of the test area 112 may be within a preselected part diameter range (range width: 4 meters). T is 6 meters, the predetermined inner diameter of the test apron area 118 is a diameter approximately equal to 6 meters, and the predetermined outer diameter W AC can be 12 meters.
[0067] As shown in FIG. 6A, at least one calibration target system 120, at least one imaging system 130, or both may be positioned within the test apron area 118. In certain embodiments, the calibration target system 120, the imaging system 130, or both may be fixedly positioned on the test apron area 118. In the figure, a first imaging system 130A is positioned on the test apron area 118 on the opposite side of the rotating bed system 110 from the calibration target system 120. In other words, the first imaging system 130A and the calibration target system 120 may be positioned within a plane that includes both ends of the diameter of the central rotating bed system 110 and passes through a central area 115 of the rotating bed system 110. In the figure, the calibration target positioning system 126 of the calibration target system 120 is positioned at a predetermined distance W from the central area 115 of the rotating bed system 110. C The first imaging system 130A is spaced a predetermined distance W from the central region 115 of the rotating stage system 110. A The electrodes are spaced apart by only a small distance.
[0068] Meanwhile, the second imaging system 130B can be located in a position within the test apron area 118 perpendicular to the plane in which the first imaging system 130A and the calibration target system 120 are located. Stated slightly differently, the first imaging system 130A and the second imaging system 130B can form a right angle about the central area 115, and additionally or alternatively, the calibration target system 120 and the second imaging system 130B can form a right angle about the central area 115. In the figure, the second imaging system 130B is located at a predetermined distance W from the central area 115 of the turntable system 110. B The electrodes are spaced apart by only a small distance.
[0069] In a particular embodiment, a predetermined distance W between the first imaging system 130A and the central region 115 Ais a predetermined distance W between the second imaging system 130B and the central region 115. B The predetermined distance W between the calibration target system 120 and the central region 115 can be equal to C is a predetermined distance W between the first imaging system 130A and the central area 115 and / or between the second imaging system 130B and the central area 115. A For example, the distance may be greater than, less than, or equal to the given diameter W of the test area 112. T If is 6 meters, the given distance W A , W B , W C At least one of the predetermined distances W may be within a predetermined distance range of 3 meters to 10 meters. A , W B , W C can be within a preselected partial distance range within the above-mentioned predetermined distance range, for example, within a preselected partial distance range of 3 meters to 6 meters (range width: 3 meters).
[0070] The calibration target system 120 shown in FIG. 6B has a predetermined height H C In a particular embodiment, the calibration target positioning system 126 of the calibration target system 120 is positioned at a predetermined height H C The height H C can be configured at an appropriate predetermined height such that the calibration indicia 124 can be appropriately positioned relative to the vehicle or sensor system 210 (shown in FIG. 1). C The height H may include, but is not limited to, a height within a predetermined height range of 0 meters to 6 meters. In certain embodiments, the predetermined height H Ccan be within a preselected partial height range within the above-mentioned predetermined height range, for example, within a preselected partial height range of 0 meters to 1 meter (range width: 1 meter).
[0071] The first imaging system 130A is positioned at a predetermined height H from the test apron area 118. A The second imaging system 130B is positioned at a predetermined height H B As shown in FIG. 6B, a first support member 132A and a second support member 132B may be disposed within the test apron area 118, with the first support member 132A supporting a first imaging system 130A and the second support member 132B supporting a second imaging system 130B. In certain embodiments, each support member 132 may be fixedly disposed on the test apron area 118 and configured to alternately (sequentially) couple to at least one selected imaging system 130. Stated somewhat differently, each support member 132 may couple to at least one of a plurality of imaging systems 130 of the same type and / or different types.
[0072] First predetermined height H A and a second predetermined height H B can be configured at an appropriate predetermined height such that the first imaging system 130A and the second imaging system 130B can be appropriately positioned relative to the vehicle or the sensor system 210. The first predetermined height H A is the second predetermined height H of the second imaging system 130B. B The first predetermined height H may be the same as or different from the first predetermined height H. A and a second predetermined height H B The first predetermined height H may include, but is not limited to, a height within a predetermined height range of 0 meters to 6 meters. In certain embodiments, the first predetermined height H A and a second predetermined height H Bcan be within a preselected partial height range within the above-mentioned predetermined height range, for example, within a preselected partial height range of 1.5 meters to 3 meters (range width: 1.5 meters).
[0073] Additionally or alternatively, the first imaging system 130A is oriented at a first predetermined imaging angle Θ with respect to the test area 112. A and a second imaging system 130B is positioned at a second predetermined imaging angle Θ B For example, as shown in FIG 6B, a first imaging system 130A can be coupled to a first support member 132A and positioned at a first predetermined imaging angle Θ A and a second imaging system 130B is coupled to a second support member 132B to adjust a second predetermined imaging angle Θ B The first predetermined imaging angle Θ A and a second predetermined imaging angle Θ B may be configured at any predetermined imaging angle suitable for imaging the vehicle or sensor system 210. The first predetermined imaging angle Θ of the first imaging system 130A A is the second predetermined imaging angle Θ of the second imaging system 130B. B The angles may be the same as or different from each other.
[0074] Exemplary First Predetermined Imaging Angle Θ A and a second predetermined imaging angle Θ B The first predetermined imaging angle Θ may include, but is not limited to, an imaging angle within a predetermined imaging angle range of 0 degrees to 60 degrees. In certain embodiments, the first predetermined imaging angle Θ A and a second predetermined imaging angle Θ B can be within a preselected partial imaging angle range within the above-mentioned predetermined imaging angle range, for example, within a preselected partial range of 0 degrees to 30 degrees (range width: 30 degrees). Note that in FIG. 6B, the first predetermined imaging angle Θ A and a second predetermined imaging angle Θ BWhile the first predetermined imaging angle Θ is illustrated as being comprised of a depression imaging angle, such description is for illustrative purposes only. A , the second predetermined imaging angle Θ B , or both may be configured with an elevation imaging angle.
[0075] Additionally or alternatively, the sensor calibration system 100 may be configured to perform static sensor system calibration. In other words, the sensor calibration system 100 may support the implementation of a robot that performs static sensor calibration on the sensor system 210 and / or the device under test 200, as shown in FIG. 7. Referring to FIG. 7, the illustrated sensor calibration system 100 includes a turntable system 110 for supporting and / or rotating the device under test 200 and / or the sensor system 210, and a calibration target system 120 disposed adjacent to the turntable system 110. The sensor calibration system 100 of FIG. 7 includes a robot arm 125. In a particular embodiment, the robot arm 125 may be an articulated robot arm 125 including an end effector member 125C in the manner shown and described with reference to FIGS. 2A and 2B.
[0076] 7, the robot arm 125 can extend the end effector member 125C above the positioning system base 128 to a first height difference Y1, and can additionally or alternatively extend the end effector member 125C below the positioning system base 128 to a second height difference Y2. An exemplary first height difference Y1 can include, but is not limited to, a predetermined first height difference within a predetermined first height difference range of 0 meters to 3 meters, and the second height difference Y2 can include, but is not limited to, a predetermined first height difference within a predetermined first height difference range of 0 meters to 3 meters. CIn a particular embodiment, the first elevation difference Y1 may be configured with a predetermined first elevation difference of 1.786 meters, and the second elevation difference Y2 may be configured with a predetermined second elevation difference of 0.720 meters.
[0077] Additionally or alternatively, the robot arm 125 can extend the end effector member 125C a first distance X1 away from the rotating bed system 110, and additionally or alternatively, the end effector member 125C can extend a second distance X2 toward the rotating bed system 110. An exemplary first distance X1 can include, but is not limited to, a predetermined first distance within a predetermined first distance range of 0 meters to 3 meters, and the second distance X2 can include a predetermined second distance within a predetermined second distance range of 0 meters to 3 meters. In a particular embodiment, the first distance X1 can be configured as a predetermined first distance of 1.150 meters, and the second distance X2 can be configured as a predetermined second distance of 1.450 meters.
[0078] Optionally, the robot arm 125 may have 2 to 9 degrees of freedom, and may additionally or alternatively have an effective range radius of 0 meters to [the sum of the first distance X1 and the second distance X2] and / or 0 meters to [the sum of the first height difference Y1 and the second height difference Y2]. Continuing with the numerical example above, the effective range radius of the robot arm 125 may be in the range of 0.5 meters to 3 meters. The combined movement of the turntable system 110 and the robot arm 125 provides the advantage of a hemispherical effective calibration reachable volume (RV) centered on the device under test 200 and / or the sensor system 210, as shown in FIG. 7.
[0079] The effective calibration reachable area RV of a hemisphere is a given diameter Z RVIn certain embodiments, the predetermined diameter Z RV can be a diameter equal to the sum of the first distance X1 and the second distance X2 and / or the sum of the first height difference Y1 and the second height difference Y2. Following the numerical example above, the predetermined diameter Z of the calibration reachable region RV obtained by the combined movements of the rotating platform system 110 and the robot arm 125 can be RV As the diameter, a predetermined diameter within a predetermined diameter range of 6 meters to 12 meters can be mentioned.
[0080] As described in detail above with reference to Figures 1, 2A and 2B, the calibration target system 120 can be associated with one or more calibration target devices 122 having calibration indicia 124 suitable for calibrating the device under test 200 and / or the sensor system 210. The calibration indicia 124 of the calibration target device 122 may be different for each type of device under test 200 and / or sensor system 210 to be calibrated, for example. This allows a suitable calibration target device 122 to be selected based on the type of device under test 200 and / or sensor system 210 to be calibrated, and the selected suitable calibration target device 122 can be removably attached to the end effector member 125C of the calibration target system 120 to calibrate the device under test 200 and / or the sensor system 210, as shown in Figures 8A and 8B, 9A and 9B, and 10A and 10B.
[0081] To allow for quick reconfiguration of the sensor calibration system 100 between external sensor calibration processes, the selected calibration target device 122 is preferably configured to be easily attached to and detached from the end effector member 125C. For example, the calibration target device 122 may be positioned within the reachable area of the robot arm 125 such that the robot arm 125 can retrieve the selected calibration target device 122 for use in a calibration process and return the selected calibration target device 122 to a stowed position after the calibration process is completed. The selected calibration target device 122 may include any predetermined number and / or type of calibration indicia 124 suitable for calibrating the sensor system(s) 210 (shown in FIG. 1 ) of interest. The selected calibration target device 122 may include, for example, one or more calibration indicia 124 that are different for each type of device under test 200 and / or sensor system(s) 210 to be calibrated.
[0082] When the sensor system 210 includes a camera imaging system, the selected calibration target device 122 may include a first calibration indicia 124U suitable for calibrating the camera imaging system, as shown in FIG. 8A. Alternatively, the selected calibration target device 122 may include a second calibration indicia 124V suitable for calibrating the camera imaging system, as shown in FIG. 8B. Optionally, when a lidar imaging system is disposed on the device under test 200 (shown in FIG. 1) as the sensor system 210 (shown in FIG. 1), the selected calibration target device 122 may include a third calibration indicia 124W suitable for calibrating the lidar imaging system, as shown in FIG. 9A. Alternatively, the selected calibration target device 122 may include a fourth calibration indicia 124X suitable for calibrating the lidar imaging system, as shown in FIG. 9B. Additionally or alternatively, if the sensor system 210 (shown in FIG. 1) includes a radar imaging system, the selected calibration target device 122 may include a fifth calibration indicia 124Y suitable for calibrating the radar imaging system, as shown in FIG 10A, or alternatively, the selected calibration target device 122 may include a sixth calibration indicia 124Z for calibrating the radar imaging system, as shown in FIG 10B.
[0083] Advantageously, the sensor system 210 can be positioned at a suitable position relative to the selected calibration target device 122 in the calibration target system 120. In other words, the movement, such as rotation, of at least one of the turntable system 110 and the calibration target system 120 can be controlled so that the device under test 200 and / or the sensor system 210 are positioned at suitable positions within the calibration reachable range RV (shown in FIG. 7 ) of the turntable system 110 and the calibration target system 120. As shown in FIG. 11A , when the sensor system 210 is positioned at the front region 201 of the device under test 200, the turntable system 110 can rotate the device under test 200 until the front region 201 is adjacent to the calibration target system 120. The calibration target system 120 can adjust the relative position of the selected calibration target device 122 relative to the front region 201 of the device under test 200 and / or the sensor system 210. This allows the sensor calibration system 100 located in the front area 201 to be positioned appropriately relative to the selected calibration target device 122 so that the sensor calibration system 100 can proceed with the calibration process.
[0084] 11B, when the sensor system 210 is disposed in the side region 202 of the device under test 200, the turntable system 110 can rotate the device under test 200 until the side region 202 is adjacent to the calibration target system 120. The calibration target system 120 can adjust the relative position of the selected calibration target device 122 with respect to the side region 202 of the device under test 200 and / or the sensor system 210, as necessary. This allows the sensor calibration system 100 disposed in the side region 202 to be positioned appropriately with respect to the selected calibration target device 122 so that the sensor calibration system 100 can proceed with the calibration process.
[0085] 11C, when the sensor system 210 is disposed in the rear region 203 of the device under test 200, the turntable system 110 can rotate the device under test 200 until the rear region 203 is adjacent to the calibration target system 120. The calibration target system 120 can adjust the relative position of the selected calibration target device 122 with respect to the rear region 203 of the device under test 200 and / or the sensor system 210. This allows the sensor calibration system 100 disposed in the rear region 203 to be positioned in an appropriate position with respect to the selected calibration target device 122 so that the sensor calibration system 100 can proceed with the calibration process. In other words, by controlling the movement, such as the positioning movement, of at least one of the turntable system 110 and the calibration target system 120, the device under test 200 and / or the sensor system 210 can be positioned in an appropriate orientation with respect to the selected calibration target device 122 regardless of the position of the sensor system 210 in the device under test 200.
[0086] An exemplary control system 140 of the sensor calibration system 100 is shown in FIG. 12. Referring to FIG. 12, the illustrated control system 140 includes a sensor data imaging and control system (or sensor data imaging and control means) 143 in communication with a master control system (or master control means) 141. The sensor data imaging and control system 143 can receive sensor data, such as point cloud images, from each imaging system 130. As shown in FIG. 12, the sensor data imaging and control system 143 can receive sensor data, such as a first point cloud image, from a first imaging system 130A, and can additionally or alternatively receive sensor data, such as a second point cloud image, from a second imaging system 130B. Optionally, the sensor data, such as point cloud images, received from the imaging systems 130 can include synchronized sensor data, such as point cloud images. The sensor data imaging and control system 143 can provide one or more imaging control commands to an imaging control system (not shown) for controlling the positioning and / or operation of each imaging system 130.
[0087] Additionally or alternatively, the sensor data imaging and control system 143 may communicate with a turntable control system 142 for controlling the positioning and movement of the turntable system 110, and additionally or alternatively, may communicate with a calibration target control system 144 for controlling the positioning and movement of the calibration target system 120. For example, the sensor data imaging and control system 143 may provide one or more turntable control commands to the turntable control system 142 for controlling the turntable system 110. Similarly, the sensor data imaging and control system 143 may provide one or more target control commands to the calibration target control system 144 for controlling the positioning, such as the orientation, of the calibration target system 120.
[0088] For example, the sensor data imaging and control system 143 can provide one or more turntable control commands to the turntable control system 142 for controlling the turntable system 110. In addition, in the case where the calibration target system 120 includes a robot arm 125 having an end effector member 125C for mounting a selected calibration target device 122 (shown in FIG. 3 ), the sensor data imaging and control system 143 can provide, to the calibration target control system 144, target control commands that issue commands to the robot arm 125 to mount the selected calibration target device 122 on the end effector member 125C or to move the end effector member 125C to a predetermined position or orientation.
[0089] In certain embodiments, communication between the sensor data imaging and control system 143 and the turntable control system 142 and the calibration target control system 144 may include bidirectional communication. This allows the sensor data imaging and control system 143 to receive turntable feedback data from the turntable control system 142 and to receive target feedback data from the calibration target control system 144. Exemplary turntable feedback data may include, but is not limited to, the current orientation and rotational speed of the turntable control system 142. Meanwhile, the target feedback data may include data regarding whether a calibration target device 122 is currently attached to the end effector member 125C, identification information regarding the currently attached calibration target device 122, and the current position and orientation of the end effector member 125C. This has the advantage that the sensor data imaging and control system 143 may utilize the received turntable feedback data to control the turntable system 110. Similarly, the sensor data imaging and control system 143 may utilize the received target feedback data to control the calibration target system 120.
[0090] The sensor data imaging and control system 143 shown in FIG. 12 supports bidirectional communication for sending and receiving data and commands to and from the master control system 141. The master control system 141 can communicate with one or more other subsystems of the control system 140. As shown in FIG. 12, for example, the master control system 141 can communicate with a 3D reconstruction system (or 3D reconstruction means) 145 for reconstructing a point cloud registration result image 136 (shown in FIG. 13) of the device under test 200 placed on the rotating stage system 110 in the manner described above with reference to FIG. 5A. Advantageously, the 3D reconstruction system 145 can receive a number of measurements of the device under test 200 from a variety of different viewpoints and convert the received measurements into the point cloud registration result image 136.
[0091] Figure 13 illustrates an exemplary 3D reconstruction system 145 of the control system 140. With reference to Figure 13, the illustrated 3D reconstruction system 145 includes a point cloud registration system (or point cloud registration means) 160 for receiving rotate stage encoder angle data from the rotate stage system 110 (shown in Figure 1) and image frame data 134 from at least one imaging system 130 (shown in Figure 1). The image frame data 134 may include a number of image frames 1341, 1342, ..., 134, such as a sequence of image frames. N may include.
[0092] As the turntable system 110 with the device under test 200 rotates, the high precision controller and encoder system 142A of the turntable control system 142 can record a number of encoder angles, such as the encoder angle sequence of the turntable system 110, in the manner described above with reference to FIGS. 1 and 3. The high precision controller and encoder system 142A can then provide the recorded encoder angles to the point cloud registration system 160. In certain embodiments, the high precision controller and encoder system 142A can provide the recorded encoder angles to the point cloud registration system 160 via the sensor data imaging and control system 143 (shown in FIG. 12), the master control system 141 (shown in FIG. 12), or both. The point cloud registration system 160 can receive the recorded encoder angles via a pose integration system (or pose integration means) 162. In certain embodiments, the pose integration system 162 can comprise a rigid body pose integration system.
[0093] Similarly, the imaging system 130 may capture image frame data 134 of the device under test 200 as it is rotated by the turntable system 110. Stated slightly differently, the image frame data 134 may include a number of images of the device under test 200 captured by the imaging system 130 from different angles while the device under test 200 is rotated by the turntable system 110. The imaging system 130 and / or an imaging control system associated with the imaging system 130 may provide the captured image frame data 134 to the point cloud registration system 160. In certain embodiments, the imaging system 130 and / or the imaging control system may provide the captured image frame data 134 to the point cloud registration system 160 via the sensor data imaging and control system 143, the master control system 141, or both. The point cloud registration system 160 may receive the captured image frame data 134 via a rigid body (RB) transformation system (or rigid body transformation means) 164.
[0094] As shown in FIG. 13, the rigid transformation system 164 transforms image frames 1341, 1342, . . . , 134 N a plurality of rigid body transformation subsystems 1641, 1642, ..., 164 for receiving one of the N The rigid transformation system 164 may integrate the captured image frame data 134 with the recorded encoder angles obtained from the pose integration system 162 to generate a rigid transformation result of the captured image frame data 134 that accounts for the movement, such as the rotation, of the turntable system 110. Specifically, the first rigid transformation subsystem 1641 may integrate the first captured image frame data 1341 with the first recorded encoder angle obtained from the pose integration system 162 to generate a first rigid transformation result of the captured image frame data 134. Then, the second rigid transformation subsystem 1642 may integrate the second captured image frame data 1342 with the second recorded encoder angle obtained from the pose integration system 162 to generate a second rigid transformation result of the captured image frame data 134, and so on until the Nth rigid transformation subsystem 164. N The Nth captured image frame data 134 N and the Nth recorded encoder angle obtained from the pose integration system 162, an Nth rigid body transformation result of the captured image frame data 134 can be generated.
[0095] Image frames 1341, 1342, …, 134 N The rigid body transformations of each of the above can be combined at the location indicated by reference numeral 166, and advantageously, the image frames 1341, 1342, ..., 134 NThe result of the composite rigid transformation of the above-mentioned can be transformed and / or accumulated by the transformation frame accumulation system (or transformation frame accumulation means) 168 to the preselected coordinate system. In this way, the transformation frame accumulation system 168 can generate the 3D point cloud registration result image 136 of the device under test 200 in the preselected coordinate system. In other words, stated slightly differently, the 3D point cloud registration result image 136 can include a dense point cloud image of the device under test 200 in the selected reference coordinate frame. Optionally, filtering can also be used to remove high noise points included in the 3D point cloud registration result image 136 of the device under test 200.
[0096] Returning now briefly to FIG. 12, optionally, the master control system 141 may communicate with a machine learning system (or machine learning means) 146 for deciphering one or more sides and / or segments of the device under test 200 disposed on the rotating platform system 110. As shown in FIG. 14, the machine learning system 146 may extract device components 220, such as one or more device parts, features, markers, etc., of the device under test 200. When the device under test 200 is configured as a passenger vehicle as shown in FIG. 14, exemplary device components 220 of the passenger vehicle may include, but are not limited to, at least one vehicle mirror 222, at least one vehicle bumper 224, at least one wheel 226 (or center of wheel 226), at least one vehicle thrust line, at least one vehicle door, and at least one vehicle pillar. The machine learning system 146 shown in FIG. 14 extracts a left side mirror 222L, a right side mirror 222R, a front bumper 224F, a rear bumper 224R, a center of a left front wheel 226LF, a center of a right front wheel 226RF, and a center of a right rear wheel 226RR.
[0097] The machine learning system 146 may extract the device components 220 of the device under test 200 in any suitable manner. For example, as shown in FIG. 14 , the machine learning system 146 may extract the device components 220 based on the 3D point cloud registration result image 136 of the device under test 200. In a particular embodiment, the machine learning system 146 may include a neural network-based machine learning framework (not shown) that is pre-trained with data related to a plurality of types of devices under test 200, such as vehicles. The training data may include component labels associated with one or more device components 220 for each type of device under test 200, such as vehicles. Stated slightly differently, each device under test 200 in the training data may include at least one device component 220, and each device component 220 may have a component label associated with it.
[0098] The neural network-based machine learning framework enables neural network inference to be performed on the 3D point cloud registration result image 136 of the passenger vehicle. Based on the neural network inference, the neural network-based machine learning framework can compare the 3D point cloud registration result image 136 with the labeled training data. In other words, the neural network-based machine learning framework can identify known passenger vehicles in the labeled data that have a set of labeled device components 220 that most closely match the device components 220 extracted from the passenger vehicle depicted in the 3D point cloud registration result image 136. In this way, the neural network-based machine learning framework can identify and locate one or more device components 220 of the passenger vehicle in the 3D point cloud registration result image 136.
[0099] Returning again to FIG. 12, optionally, the master control system 141 may communicate with a robot and turntable path planning system 147 for solving the forward and inverse kinematics of the turntable system 110 and the calibration target system 120. The robot and turntable path planning system 147 may advantageously control the movement of the turntable system 110 and the calibration target system 120 to a desired orientation and position while avoiding collisions with objects in the sensor calibration environment in which the sensor calibration system 100 is located, such as the device under test 200 resting on the turntable system 110. Stated slightly differently, the robot and turntable path planning system 147 may avoid the calibration target system 120 from travelling along a path of movement that may cause a collision between the selected calibration target device 122 (shown in FIG. 3) or the calibration target positioning system 126 (shown in FIG. 3) and objects in the sensor calibration environment, such as the device under test 200 oriented in a predetermined direction on the turntable system 110.
[0100] The illustrated master control system 141 is shown in communication with a user interface 148 to allow a system user (not shown), such as a system operator, to interact with the sensor calibration system 100. For example, the master control system 141 may allow instructions to be provided to the sensor calibration system 100 by the system user. Exemplary instructions may include, but are not limited to, instructions for setting up the sensor calibration system 100 to perform an external sensor calibration process on the device under test 200 (shown in FIG. 1 ) and / or the sensor system 210 (shown in FIG. 1 ), instructions for initiating an external sensor calibration process, and instructions for terminating an external sensor calibration process. Instructions for setting up the sensor calibration system 100 may include, but are not limited to, instructions for selecting a calibration target device 122 (shown in FIG. 3 ) to be placed on the calibration target positioning system 126 (shown in FIG. 3 ) and instructions for loading the selected calibration target device 122 onto the calibration target positioning system 126. In certain embodiments, the user interface 148 may visually and / or audibly present one or more results of the external sensor calibration process. Optionally, the user interface 148 may also present diagnostic information associated with the external sensor calibration process.
[0101] As shown in Fig. 12, the master control system 141 may optionally communicate with a target placement simulation system (or target placement simulation means) 149. Advantageously, the target placement simulation system 149 may assist in designing an external sensor calibration process procedure for a particular type of device under test 200 and / or sensor system 210. For example, as shown in Fig. 15, the target placement simulation system 149 may provide at least one tool 149A for positioning one or more virtual calibration target devices 122V around the periphery of a virtual device under test 200V associated with a virtual sensor system 210V. The virtual device under test 200V and / or virtual sensor system 210V may closely resemble the real (i.e., physical) device under test 200 (shown in Fig. 1) and / or sensor system 210 (shown in Fig. 1) that are the subject of the external sensor calibration process. In other words, the virtual device under test 200V may have a model of the real device under test 200 that is being subjected to an external sensor calibration process, and the virtual sensor system 210V may have a model of the real sensor system 210 that is being subjected to an external sensor calibration process.
[0102] Advantageously, the target placement simulation system 149 can present or otherwise provide a virtual sensor calibration environment for the real device under test 200 and / or sensor system 210. The tool 149A can also advantageously allow a system user to design the virtual sensor calibration environment by configuring and adjusting the number, selection, position and orientation of the virtual calibration target devices 122V with the tool 149A. If desired, the sensor calibration system 100 can also accurately replicate the virtual sensor calibration environment by utilizing a real turntable system 110 (shown in FIG. 1 ) and one or more real calibration target systems 120 (shown in FIG. 1 ) that perform an external sensor calibration process on the real device under test 200 and / or sensor system 210 in the real sensor calibration environment.
[0103] Stated somewhat differently, the number, selection, location, and orientation of the virtual calibration target devices 122V can be transferred to the calibration target system 120. In certain embodiments, the robot and turntable path planning system 147 (shown in FIG. 12) can control the movement of the turntable system 110 and / or the calibration target system 120 to a desired orientation and location in the manner described in detail above with reference to FIG. 12. Optionally, the tool 149A can also comprise a computer-implemented tool that is presented or otherwise provided by a processing system (or processing means) 149B, which may be, for example, any suitable conventional processing system, such as a computer server system, a personal computing system, a notebook computing system, a tablet computing system, a mobile phone system, or the like.
[0104] 12 and the description given with reference thereto, the master control system 141, the turntable control system 142, the sensor data imaging and control system 143, the calibration target control system 144, the imaging control system, the 3D reconstruction system 145, the machine learning system 146, the robot and turntable path planning system 147, the user interface 148, and the target placement simulation system 149 are described as being composed of separate systems, but such description is merely exemplary. Alternatively, one or more of these systems may be integrated in whole or at least in part into a composite system.
[0105] Although particular embodiments of the sensor calibration system 100 and / or external calibration method 300 have been described above with reference to particular numerical values and / or numerical ranges, it will be understood that these numerical values are exemplary only and are not intended to be limiting. The numerical values and numerical ranges associated with particular features of the sensor calibration system 100 and / or external calibration method 300 may vary depending on the particular application of the sensor calibration system 100 and / or external calibration method 300, such as the mass, size, diameter, shape, and other characteristics of the device under test 200 that it accommodates. It will also be understood that in some cases, such numerical values and numerical ranges may fall outside of the ranges described herein or may additionally or alternatively fall within preselected subranges of the ranges described herein.
[0106] In certain embodiments, one or more of the features disclosed herein may be provided as a computer program product encoded on one or more non-transitory machine-readable storage media. A phrase of the form "at least one of A, B, C, and D" herein should be interpreted as meaning "one or more A, one or more B, one or more C, and / or one or more D". Similarly, a phrase of the form "A, B, C, or D" herein should be interpreted as meaning "A, or B, or C, or D". For example, a phrase of the form "A, B, C, or combinations thereof" herein should be interpreted as meaning "A, or B, or C, or any combination of A, B, and / or C".
[0107] While the present disclosure is susceptible to various modifications and variations, specific examples thereof have been shown by way of example in the drawings and are described in detail herein, however, it should be understood that the present disclosure is not to be limited to the particular forms or methods disclosed, but rather, the present disclosure is to embrace all modifications, equivalents, and alternatives. [Explanation of symbols]
[0108] 100 Sensor Calibration System 110 Rotating Table System 112 Test Area 114 Slope area 115 Central area 116 Rotating table motor system 118 Test Apron Area 120 Calibration Target System 122 Calibration Target Device 122A First Calibration Target Device 122B Second Calibration Target Device 122V Virtual Calibration Target Device 124 Calibration Index 125 Robot Arm 125A Arm segment parts 125B Arm joint member 125C End effector member 126 Calibration Target Positioning System 127 Positioning system components 128 Positioning system base 130 Imaging System 130A First Imaging System 130B Second Imaging System 132 Support member 132A First Support Member 132B Second support member 134 Captured image frame data 1341 first captured image frame data 1342 second captured image frame data 134 N Nth captured image frame data 136 3D point cloud registration result image 140 Control System 141 Master Control System 142 Rotating table control system 142A Encoder System 143 Sensor Data Imaging and Control System 144 Calibration Target Control System 145 3D Reconstruction System 146 Machine Learning Systems 147 Rotating Table Path Planning System 148 User Interface 149 Target Placement Simulation System 149A Tools 149B Processing Systems 160 Point Cloud Registration System 162 Posture Integration System 164 Rigid Body Transformation System 1641 First Rigid Body Transformation Subsystem 1642 Second Rigid Body Transformation Subsystem 164 N Nth rigid body transformation subsystem 168 Conversion Frame Accumulation System 200 Devices Under Test 200A First Device Under Test 200B Second Device Under Test 200V Virtual Device Under Test 201 Anterior area 202 Side Area 203 Posterior area 210 Sensor System 210A First Sensor System 210B Second Sensor System 210V Virtual Sensor System 220 Device Components 222 Vehicle mirror 222L Left side mirror 222R Right side mirror 224 Vehicle Bumper 224F front bumper 224R Rear bumper 226 Wheel (centre) 226LF Left front wheel (centre) 226RF Right front wheel (centre) 226RR Right rear wheel (centre)
Claims
1. 1. A method for automatically calibrating a sensor system associated with a device under test, comprising: scanning the device under test to identify the device under test; configuring a calibration target system to cooperate with the identified device under test; calibrating the sensor system using the set calibration target system; The method includes:
2. the step of scanning the device under test further comprising: capturing a three-dimensional image of the device under test; identifying the device under test based on the captured three-dimensional image; The method of claim 1 , comprising:
3. The step of capturing a three-dimensional image comprises: rotating the device under test relative to an imaging means; capturing an image of a three-dimensional point cloud registration result of the device under test while the device is rotating by the imaging means; The method of claim 2 , comprising:
4. 4. The method of claim 3, wherein said step of capturing said 3D point cloud registration result image comprises the step of capturing, by said imaging means, a sparse 3D point cloud registration result image of said device under test in a rotating state.
5. 5. The method of claim 3, wherein the step of capturing the 3D point cloud registration result image comprises the step of capturing, by the imaging means, a dense 3D point cloud registration result image of the device under test in a rotating state.
6. The step of capturing the three-dimensional point cloud registration result image comprises: recording a plurality of encoder angles of a turntable system for rotating the device under test; capturing a sequence of image frames of the device under test as it rotates; generating a plurality of rigid body transformations from the captured image frames and the recorded encoder angles corresponding to each image frame; fitting the generated rigid transformation result into a preselected coordinate system to provide the 3D point cloud registration result image; The method according to any one of claims 3 to 5, comprising:
7. 4. The method of claim 3, wherein the step of identifying the device under test comprises: extracting at least one device marker from the captured three-dimensional image; identifying the device under test based on the extracted device markers; The method according to any one of claims 2 to 6, comprising:
8. 8. The method of claim 7, wherein the step of extracting the at least one device marker comprises the step of extracting a device mirror, a device bumper, a device wheel, a device wheel center, a device axle center, a device logo, a device thrust line, a device door, and a device pillar from the captured three-dimensional image.
9. The step of extracting the at least one device marker further comprises: training a machine learning framework with device data for a plurality of devices under test of different types; extracting the at least one device marker from the captured three-dimensional image using the trained machine learning framework; The method of claim 7 or 8, comprising:
10. The method according to any one of claims 2 to 9, wherein the step of identifying the device under test comprises identifying the sensor system based on the captured three-dimensional image.
11. said step of configuring said calibration target system further comprising: selecting a calibration target device having calibration indicia suitable for calibrating the sensor system of the identified device under test; placing the selected calibration target device in a calibration target positioning system of the calibration target system; establishing, by the calibration target positioning system, at least one position attribute of the selected calibration target device relative to the sensor system; The method according to any one of claims 1 to 10, comprising:
12. The method of claim 11 , wherein the step of selecting the calibration target device comprises the step of selecting the calibration target device from among a plurality of calibration target devices having different calibration indicia.
13. 13. The method of claim 11 or 12, wherein the step of setting the at least one position attribute of the selected calibration target device comprises the steps of radially translating and three-dimensionally rotating the calibration target positioning system relative to the device under test.
14. 14. The method according to claim 11, wherein the step of setting the calibration target system includes a step of determining a travel path that moves the calibration target system to a state of the at least one position attribute that has been set while avoiding collisions between the selected calibration target device and the device under test.
15. The method of claim 14 , wherein the step of determining the travel path comprises solving forward and inverse kinematics of the calibration target system and a rotation stage system for rotating the device under test.
16. 16. The method of claim 11, wherein the step of setting the at least one position attribute comprises setting, by the calibration target positioning system, at least one orientation attribute of the selected calibration target device relative to the sensor system.
17. said step of configuring said calibration target system further comprising: creating a virtual sensor calibration environment by placing, by a processing means, a virtual calibration target device adjacent to a virtual device under test, the virtual device under test being associated with a virtual sensor system and having a type of the device under test; A step of simulating an external calibration process for the virtual sensor system using the constructed virtual sensor calibration environment; adjusting, by said processing means, at least one three-dimensional position attribute of said virtual calibration target device based on said step of simulating said external calibration process; configuring the calibration target system by positioning a calibration target device associated with the calibration target system relative to the device under test according to the at least one three-dimensional position attribute of the adjusted virtual calibration target device; The method according to any one of claims 1 to 16, comprising:
18. The method according to any one of claims 1 to 17, wherein the step of calibrating the sensor system includes a step of calibrating an advanced driver assistance system (ADAS) sensor system disposed on a passenger vehicle using the configured calibration target system.
19. 19. The method of claim 1, wherein the step of calibrating the sensor system comprises a step of calibrating an autonomous vehicle (AV) sensor system disposed on a passenger vehicle with the configured calibration target system.
20. A system for automatically calibrating a sensor system associated with a device under test, the system comprising means for implementing the method according to any one of claims 1 to 19.
21. A computer program product for automatically calibrating a sensor system associated with a device under test, the computer program product comprising instructions for carrying out the method of any one of claims 1 to 19.
22. 22. The computer program product of claim 21, encoded on one or more non-transitory machine-readable storage media.
23. 1. A computer program product for automatically calibrating a sensor system associated with a device under test, comprising: instructions to scan for the device under test to identify the device under test; instructions for configuring a robotic calibration target system to cooperate with the identified device under test; An instruction to calibrate the sensor system using the set calibration target system; A computer program product comprising:
24. 24. The computer program product of claim 23, encoded on one or more non-transitory machine-readable storage media.
25. 1. A system for automatically calibrating a sensor system associated with a device under test, the system comprising: a central turntable system for rotating the device under test; an articulated robot type calibration target system having an end effector member for mounting a calibration target device; a first imaging means and a second imaging means configured to scan the device under test to generate a three-dimensional image of the device under test while the turntable system rotates the device under test; and control means for identifying the device under test based on the three-dimensional image and configuring the robotic calibration target system to cooperate with the identified device under test; the first imaging means, the second imaging means, and the robotic calibration target system are arranged around a periphery of the turntable system; A system in which the sensor system is calibrated using the set calibration target system.
26. 26. The system of claim 25, wherein the articulated robotic calibration target system has between 3 and 9 degrees of freedom.
27. 27. The system of claim 25 or 26, wherein the articulated robotic calibration target system comprises at least one revolute joint member, at least one prismatic joint member, or both.
28. The system according to any one of claims 25 to 27, wherein the first imaging means and the second imaging means are each selected from the group of imaging means consisting of a camera imaging means, a light detection and ranging (lidar) imaging means, a radio detection and ranging (radar) imaging means, and an ultrasonic imaging means.
29. the first imaging means and the robotic calibration target system are disposed in a first plane passing through a central region of the rotating stage system; A system according to any one of claims 25 to 28, wherein the second imaging means is arranged in a second plane perpendicular to the first plane and passing through the central region of the rotating stage system.
Citation Information
Patent Citations
Vehicle type identification method and system and electronic equipment
CN112712023A
Three-dimensional scanner
JP2015040856A
Vehicle inspection device
JP2020051792A
Sensor solution determination method, device, instrument and storage medium
JP2021098506A
Enhanced three-dimensional training data generation
US20190311487A1
