Interactive calibration method and device of roadside multi-sensing device and electronic device

Abstract

The application discloses an interactive calibration method and device of a road end multi-perception device and electronic equipment, the method is applied to an interactive calibration device, the interactive calibration device is in communication connection with an RTK device, a radar and a camera at a road end, and the interactive calibration method comprises the following steps: receiving a dotting time sent by the RTK device, and receiving point cloud frames and image frames obtained by the radar and the camera based on the dotting time; target detection is respectively performed on the point cloud frames and the image frames, and a detection result of a probe point of the RTK device is obtained; if the detection result indicates that the probe point detection fails, prompt information of dotting failure is generated and sent to the RTK device, so that the RTK device re-performs dotting. Through the above scheme, the technical problems of low calibration accuracy and high calibration cost of the road end multi-perception device in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, in particular to an interactive calibration method and device for a roadside multi-sensing device and an electronic device. BACKGROUND

[0002] Vehicle-road cooperation has become a key to the landing of the domestic automatic driving industry. Vehicle-road cooperation refers to the combination of smart vehicles and intelligent roads to achieve efficient cooperation of "people-vehicles-roads-cloud", greatly increasing the detection range and reducing the computational burden of the vehicle end, making it easier and less costly for automatic driving to land, and providing more safety redundancy. Intelligent roads need to deploy various sensors and edge computing devices on the road end. Various sensing devices provide accurate and comprehensive sensing data to the edge computing device, which performs multi-sensing data fusion, detection, decision-making, etc. based on the data, and then shares relevant data with the vehicle end. Multi-sensor fusion requires multi-sensing device calibration.

[0003] Existing road end multi-sensing device calibration usually collects a batch of calibration data at the road end, then manually screens the calibration data, including deleting sensing data that is obviously obscured, unclear target, and timestamp mismatch, and finally calibrates based on the screened calibration data. The present inventor found during the implementation of the present application that the calibration data after preliminary screening still has a lot of data that cannot determine the key points, and the calibration based on this has generally low accuracy. In actual engineering, workers have to repeatedly go to the end to calibrate multiple times until the calibration result meets the basic requirements, resulting in generally high calibration cost. SUMMARY

[0004] The embodiments of the present application provide an interactive calibration method and device for a road end multi-sensing device and an electronic device, to solve the technical problems of low calibration accuracy and high calibration cost of the road end multi-sensing device in the prior art.

[0005] In a first aspect, the embodiments of the present application provide an interactive calibration method for a road end multi-sensing device, applied to an interactive calibration device, the interactive calibration device being in communication connection with an RTK device, a radar, and a camera at the road end, and the interactive calibration method comprising:

[0006] receiving a dotting time sent by the RTK device, and receiving a point cloud frame and an image frame obtained by the radar and the camera based on the dotting time;

[0007] performing target detection on the point cloud frame and the image frame respectively to obtain a detection result of a probe point of the RTK device;

[0008] If the detection result indicates that the probe point detection fails, a dotting failure prompt information is generated and sent to the RTK device, so that the RTK device re-dots.

[0009] Optionally, the method further comprises:

[0010] After receiving the dotting time, triggering the radar and the camera to collect perception data;

[0011] And receiving the perception data fed back by the radar and the camera, obtaining the point cloud frame and the image frame closest to the dotting time from the time stamp in the perception data.

[0012] Optionally, if the detection result indicates that the probe point detection fails, a dotting failure prompt information is generated, specifically comprising:

[0013] Judging whether the RTK device is blocked based on the detection result of the image frame;

[0014] If the RTK device is blocked, a dotting failure blocking prompt information is generated to make the RTK device adjust the dotting position and re-dot.

[0015] Optionally, the RTK device comprises a cross-shaped calibration rod composed of a centering rod and a horizontal rod, the centering rod and the horizontal rod are telescopic, and / or the horizontal rod can rotate left and right relative to the centering rod;

[0016] If the RTK device is not blocked, the dotting failure prompt information comprises:

[0017] Generating a first prompt information that the dotting failure needs to adjust the centering rod and / or the horizontal rod, and / or,

[0018] Generating a second prompt information that the dotting failure needs to adjust the horizontal angle of the horizontal rod.

[0019] Optionally, the method further comprises:

[0020] If the detection result indicates that the probe point detection succeeds, pairing and saving the point cloud coordinates, image coordinates and GPS coordinates of the probe point;

[0021] When the number of paired points is greater than a set threshold, calculating an initial conversion matrix required for coordinate point conversion between any two perception devices;

[0022] Based on the initial conversion matrix, verifying the next pair of paired points and obtaining a mapping error;

[0023] If the mapping error is greater than a set error threshold, a continue dotting prompt is generated and sent, and if the mapping error is less than or equal to the set error threshold, an abort dotting prompt is generated and sent.

[0024] Optionally, if the mapping error is greater than a set error threshold, the method further comprises:

[0025] Based on the initial conversion matrix, all paired points are verified, and n paired points with the largest mapping error are obtained, n≥3;

[0026] Based on the n paired points, a target dotting area is determined, and the target dotting area is sent to the RTK device, so that the RTK device continues to dot in the target dotting area.

[0027] Optionally, if the mapping error is greater than a set error threshold, the method further comprises:

[0028] Based on all paired points, a dotting map is constructed;

[0029] The dotting map is compared with a target detection area, and an area in the target detection area that is not covered or has a coverage rate lower than a coverage threshold by dotting is taken as a target dotting area;

[0030] The target dotting area is sent to the RTK device, so that the RTK device continues to dot in the target dotting area.

[0031] In a second aspect, an embodiment of the present application provides an interactive calibration device, which is in communication connection with an RTK device, a radar and a camera at a road end, and the interactive calibration device comprises:

[0032] A receiving unit is configured to receive a dotting time sent by the RTK device, and receive a point cloud frame and an image frame obtained by the radar and the camera based on the dotting time;

[0033] A detection unit is configured to perform target detection on the point cloud frame and the image frame respectively, and obtain a detection result of a probe point of the RTK device;

[0034] A generating unit is configured to generate a dotting failure prompt information in a case where the detection result indicates that the probe point detection fails;

[0035] A sending unit is configured to send to the RTK device, so that the RTK device re-dots.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, which comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to implement any method of the first aspect.

[0037] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement any method of the first aspect.

[0038] The one or more technical solutions in the embodiments of the present application have at least the following technical effects:

[0039] The embodiments of the present application provide an interactive calibration method of a road end multi-sensing device. The communication connection between an RTK device, a radar, a camera and a calibration device is established, the probe point detection, that is, the detection of key points required for calibration, is performed while the calibration is performed, the re-punching point prompt is performed on the punching position of which the detection fails, so that the final obtained punching data is all valid punching data. The calibration based on the punching data can effectively improve the calibration accuracy. In addition, the method can obtain a large amount of valid punching data at one time, and the staff does not need to repeatedly go to the road end to punch, which greatly reduces the calibration cost. It can be seen that the above technical solutions provided by the embodiments of the present application solve the technical problems of low calibration accuracy and high calibration cost of the road end multi-sensing device in the prior art, and improve the calibration accuracy of the road end multi-sensing device and reduce the calibration cost. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 A flowchart of an interactive calibration method of a road end multi-sensing device provided by the embodiments of the present application is shown in the figure;

[0041] Figure 2 An interactive flowchart of a punching trigger provided by the embodiments of the present application is shown in the figure;

[0042] Figure 3 A frame diagram of an interactive calibration device of a road end multi-sensing device provided by the embodiments of the present application is shown in the figure;

[0043] Figure 4 A structure diagram of an electronic device provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0044] Before introducing the embodiments of the present disclosure, it should be explained that:

[0045] Some embodiments of the present disclosure are described as a process flow, although the operation steps of the flow can be implemented in parallel, concurrently or simultaneously.

[0046] The term “and / or” can be used in the embodiments of the present disclosure, and “and / or” includes any and all combinations of one or more of the associated features listed.

[0047] It should be understood that when describing the connection or communication relationship between two components, unless it is explicitly stated that the two components are directly connected or communicate directly, the connection or communication between the two components can be understood as a direct connection or communication, or it can be understood as an indirect connection or communication through an intermediate component.

[0048] To make the technical solutions and advantages of the embodiments of this disclosure clearer, the exemplary embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this disclosure can be combined with each other.

[0049] Example

[0050] Please refer to Figure 1 This embodiment provides an interactive calibration method for roadside multi-sensor devices, applied to an interactive calibration device that is communicatively connected to RTK equipment, radar, and cameras at the roadside. RTK equipment is a Real-Time Kinematic Positioning (RTK) tool that uses GPS satellite signal carrier phase to perform real-time correction of GPS position information, providing GPS location information with very high positioning and measurement accuracy. Based on this RTK equipment, the interactive calibration method includes:

[0051] S110, Receive the dot-mapping time sent by the RTK device, and receive the point cloud frames and image frames obtained by the radar and the camera based on the dot-mapping time;

[0052] S120. Target detection is performed on the point cloud frame and the image frame respectively to obtain the detection results of the probe points of the RTK device;

[0053] S130. If the detection result indicates that the probe point detection failed, a prompt message indicating failure to mark the point is generated and sent to the RTK device so that the RTK device can re-mark the point.

[0054] By establishing a communication connection between RTK equipment, radar, camera, and calibration device, probe point detection (i.e., detection of key points required for calibration) is performed simultaneously during the calibration process. Points that fail to be detected are prompted to be re-marked, ensuring that all the final point data are valid. Calibration based on this point data can effectively improve calibration accuracy. Furthermore, this method can obtain a large amount of valid point data at once, eliminating the need for staff to repeatedly go to the roadside to mark points, thus greatly reducing calibration costs.

[0055] In the implementation process, the RTK device can send the dotting time to the radar, camera and interactive calibration device when dotting. This method requires the radar and camera to be in the environment perception state before dotting, and due to network delay and other reasons, there is always a certain error between the dotting data and the perception data collected by the radar and camera. In order to avoid or reduce this error, the embodiment provides a new dotting method. The RTK device sends the dotting time to the interactive calibration device, camera and radar before dotting, so that the camera and radar collect perception data at the dotting time and send the corresponding perception data to the calibration device. As another optional embodiment, the embodiment can also perform the following steps to obtain the point cloud frame and image frame. Please refer to Figure 2 :

[0056] S111. The RTK device sends the dotting time to the calibration device before dotting.

[0057] S112. The calibration device triggers the radar and camera to collect perception data after receiving the dotting time. Specifically, the radar and camera can be triggered to collect perception data a short time (such as 10s) before the dotting time.

[0058] S113. The camera and radar enter the perception data collection mode after receiving the trigger information of the calibration device, and feed back the perception data to the calibration device. Specifically, the calibration device can send the dotting time to the radar and camera when triggering the perception data collection, so that the radar and camera can collect the perception data of a period of time before and after the dotting time and feed back.

[0059] S114. The calibration device obtains the point cloud frame and image frame with the closest timestamp to the dotting time from the perception data.

[0060] After obtaining the point cloud frame and image frame in S110, S120 is executed to detect the point cloud frame and image frame respectively to obtain the detection result of the probe point of the RTK device. The probe point detection can use a deep learning model to detect, can also use a traditional segmentation detection, and can also use a deep learning method to detect a large target such as an RTK device or a calibration rod, and then use a traditional point cloud segmentation and image segmentation method in the area of the detected RTK device or calibration rod to detect the probe point. It should be noted that the embodiment does not limit the specific method of probe point detection, and any probe point detection method in the prior art can be applicable. After detecting the detection result of the probe point, S130 is executed.

[0061] If the detection result in S130 indicates that the probe point detection failed, a failure message is generated and sent to the RTK device so that the RTK device can re-perform the detection. Conversely, if the detection result indicates that the probe point detection was successful, the current point cloud frame, point cloud image, detection time, and detection result are saved, and the process proceeds to the next detection point. Probe point detection failures typically occur in point cloud frames. Due to various reasons such as the target being too far away, insufficient radar lines, or environmental obstructions, probe point detection often fails. Such point cloud frames and their corresponding image frames and GPS points are invalid for joint calibration between radar, camera, and GPS. Using this data may also lead to calibration errors. Therefore, this embodiment detects and excludes such data in S130, prompting the RTK device to re-perform the detection, thereby enabling the radar and camera's perception data to be re-acquired and detected.

[0062] In practice, when generating prompts, only a prompt to re-mark points can be generated. Whether the RTK device needs adjustment during re-marking and how to adjust it can be decided by the staff based on the on-site situation. As a preferred implementation, this embodiment can also determine whether the RTK device is occluded based on the detection results of the image frame. If the RTK device is occluded, an occlusion prompt message indicating marking failure is generated, so that the RTK device can adjust the marking position and re-mark points. If the RTK device is not occluded, a first prompt message and / or a second prompt message are generated to adjust the attitude of the RTK device's calibration rod.

[0063] To improve the detection rate of probe points, this embodiment provides a calibration rod, including an RTK receiver, a centering rod, and a crossbar. The RTK receiver is connected to one end of the centering rod, and the tip of the other end of the centering rod is the probe point. The centering rod and the crossbar can form a cross shape, thus forming a cross-shaped calibration rod. The centering rod and the crossbar are telescopic, and / or the crossbar can rotate left and right relative to the centering rod. When the detection result indicates that the RTK device is not obstructed and a prompt message indicating a failure to mark a probe point is generated, a first prompt message indicating that the centering rod and / or the crossbar needs adjustment can be generated, and / or a second prompt message indicating that the horizontal angle of the crossbar needs adjustment can be generated. The prompt messages specify the specific adjustment method for the calibration rod, which helps to quickly obtain effective sensing data.

[0064] The embodiment can further perform calibration verification on the basis of real-time probe point detection for each dotting to exclude invalid dotting data, determine whether to suspend calibration based on the calibration verification result, and avoid going back to the field for re-doting due to the fact that the calibration error does not meet the requirements. Specifically, if the detection result indicates that the probe point detection is successful, the point cloud coordinates, image coordinates and GPS coordinates of the probe point are paired and saved; when the number of paired points is greater than a set threshold, an initial conversion matrix required for coordinate point conversion between any two sensing devices is calculated; the next pair of paired points is verified based on the initial conversion matrix and a mapping error is obtained; if the mapping error is greater than a set error threshold, a continue-doting prompt is generated and sent, and if the mapping error is less than or equal to the set error threshold, a dotting suspension prompt is generated and sent. The mapping error, also known as the calibration error, is generally obtained by converting the to-be-verified A point using the initial conversion matrix to obtain a point A1 in another coordinate system, calculating the distance between the actually detected point A2 in another coordinate system and A1, and the distance is the calibration error, which is theoretically as small as possible. The calibration error is related to the number of paired points and the accuracy of the calibration point. Generally, a set threshold corresponding to the number of paired points is greater than or equal to 3 to realize calibration, but in order to obtain better calibration results, the set threshold can be set to 10, 15 or 20. The specific value of the set threshold can be set according to the calibration error range, if the calibration error range is less than 1 m, the set threshold can be 10; if the calibration error range is less than 0.5 m, the set threshold can be 15; if the calibration error range is less than 0.3 m, the set threshold can be 20. Correspondingly, the set error threshold can be taken as the corresponding calibration error range.

[0065] When the mapping error is greater than the set error threshold, the embodiment of the application further makes a dotting decision in any of the following ways.

[0066] Method one: verify all paired points based on the initial conversion matrix, obtain n paired points with the largest mapping error, n≥3; determine a target dotting area based on the n paired points, and send the target dotting area to the RTK device, so that the RTK device continues to dot in the target dotting area. By detecting and sorting the dotting error, the area corresponding to the point pair with larger error is obtained, and the area with larger error is dotted, i.e., the target dotting area is dotted again, and the conversion matrix is verified again based on the re-dotted data and the previous dotting data. This method can quickly obtain a conversion matrix that meets the mapping error and improve the calibration efficiency.

[0067] The second mode is to construct a dotting map based on all the matching points, compare the dotting map with the target detection area, and take the area in the target detection area which is not covered by the dots or has a coverage rate lower than a coverage threshold as a target dotting area; and send the target dotting area to the RTK device, so that the RTK device continues to dot in the target dotting area. Different from the first mode, the second mode is to dot as comprehensively and uniformly as possible in the target detection area, so that the coordinate conversion of the calibration result meets the mapping error requirement for the entire area, and the overall coverage of the calibration is improved. Of course, when the mapping error does not meet the requirement, the re-dotted position can also be manually selected by the staff according to the on-site environment.

[0068] In the above embodiments, by establishing the communication connection among the RTK device, the radar, the camera and the calibration device, the probe point detection, i.e., the detection of the key points required for calibration, is performed while dotting in the calibration process, the re-dotted position is prompted for the dotting position for which the detection fails, so that the final obtained dotting data are all valid dotting data, and the calibration based on the dotting data can effectively improve the calibration accuracy. Moreover, the method can obtain a large amount of valid dotting data at one time, and the staff does not need to repeatedly go to the road end for dotting, which greatly reduces the calibration cost. It can be seen that the above technical solution provided in the embodiments of the application solves the technical problems of low calibration accuracy and high calibration cost of the road end multi-perception device in the prior art, and improves the calibration accuracy of the road end multi-perception device and reduces the calibration cost.

[0069] Based on Figure 1 The interactive calibration method for the road end multi-perception device is provided, and the embodiments also correspondingly provide an interactive calibration device which is in communication connection with the RTK device, the radar and the camera at the road end. Please refer to Figure 3 The device comprises:

[0070] The receiving unit 31 is configured to receive the dotting time sent by the RTK device, and receive the point cloud frame and the image frame obtained by the radar and the camera based on the dotting time;

[0071] The detection unit 32 is configured to perform target detection on the point cloud frame and the image frame respectively, and obtain the detection result of the probe point of the RTK device;

[0072] The generation unit 33 is configured to generate a dotting failure prompt information in the case that the detection result indicates that the probe point detection fails;

[0073] The sending unit 34 is configured to send to the RTK device, so that the RTK device re-dots.

[0074] As an optional implementation, after receiving the dotting time, the receiving unit 31 can trigger the radar and the camera to collect perception data through the sending unit 34; and receive the perception data fed back by the radar and the camera, and obtain the point cloud frame and the image frame closest to the dotting time from the perception data.

[0075] As an optional implementation, if the detection result indicates that the probe point detection fails, the generating unit 33 is further configured to: determine whether the RTK device is blocked based on the detection result of the image frame; if the RTK device is blocked, generate a blocking prompt information of dotting failure, so that the RTK device adjusts the dotting position and then re-dots.

[0076] As an optional implementation, the RTK device comprises a cross-shaped calibration rod composed of a plumb rod and a horizontal rod, the plumb rod and the horizontal rod are telescopic, and / or the horizontal rod can rotate left and right relative to the plumb rod; the generating unit 34 is further configured to: in the case that the RTK device is not blocked, generate a first prompt information that the dotting fails and the plumb rod and / or the horizontal rod need to be adjusted, and / or generate a second prompt information that the dotting fails and the horizontal rod needs to be adjusted.

[0077] As an optional implementation, the detection unit 32 is further configured to, when the detection result indicates that the probe point detection succeeds, pair and save the point cloud coordinates, image coordinates and GPS coordinates of the probe point; when the number of paired points is greater than a set threshold, calculate an initial conversion matrix required for coordinate point conversion between any two perception devices; verify the next pair of paired points based on the initial conversion matrix and obtain a mapping error; the generating unit 34 is further configured to: if the mapping error is greater than a set error threshold, generate a continue dotting prompt and send it, and if the mapping error is less than or equal to the set error threshold, generate a dotting termination prompt and send it.

[0078] If the mapping error is greater than a set error threshold, the detection unit 32 is further configured to verify all paired points based on the initial conversion matrix, obtain n paired points with the largest mapping error, n≥3; determine a target dotting area based on the n paired points; and the sending unit 34 is further configured to send the target dotting area to the RTK device, so that the RTK device continues to dot in the target dotting area.

[0079] As another optional implementation, if the mapping error is greater than a set error threshold, the detection unit 32 is further configured to construct a dotting map based on all the matched points; compare the dotting map with the target detection area, and take an area in the target detection area which is not covered by dots or has a coverage rate lower than a coverage threshold as a target dotting area; and the sending unit 34 is further configured to send the target dotting area to the RTK device, so that the RTK device continues dotting in the target dotting area.

[0080] As to the apparatus in the above embodiments, the specific manners in which the various units perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0081] Figure 4 FIG. 4 is a block diagram of an electronic device 400 for an interactive calibration method according to an example embodiment. For example, the electronic device 400 can be an industrial computer, a computer, an edge server, an edge computing device, etc.

[0082] Referring to Figure 4 The electronic device 400 can include one or more of the following components: a processing component 402, a memory 404, a power supply component 406, an input / output (I / O) interface 408, and a communication component 410.

[0083] The processing component 402 generally controls the overall operation of the electronic device 400 such as operations associated with data computing, control, instruction issuing, and camera triggering. The processing component 402 can include one or more processors 420 to execute instructions to complete all or part of steps of the above method. In addition, the processing component 402 can include one or more modules to facilitate the interaction between the processing component 402 and other components.

[0084] The memory 404 is configured to store various types of data to support the operation of the device 400. Examples of these data include instructions for any application or method operating on the electronic device 400, image data, associated data, configuration data, etc. The memory 404 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0085] The power supply component 406 provides power for the various components of the electronic device 400. The power supply component 406 can include a power management system, one or more power supplies, and other components associated with generating, managing and distributing power for the electronic device 400.

[0086] The communication component 410 is configured to facilitate wired or wireless communication between the electronic device 400 and other devices. The electronic device 400 can access a wireless network based on a communication standard, such as WiFi, 2G, or 4G, or a combination thereof. In an exemplary embodiment, the communication component 410 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 410 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technology.

[0087] In an exemplary embodiment, the electronic device 400 can be implemented with one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements, for performing the above-described methods.

[0088] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 404 including instructions, is also provided, which can be executed by the processor 420 of the electronic device 400 to complete the above-described methods. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc. The instructions in the non-transitory computer-readable storage medium are executed by the processor 420 of the electronic device 400, and the point cloud denoising method in the above-described embodiments can be implemented.

[0089] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims. Other objects, features, aspects and advantages of the present application will become apparent to one of ordinary skill in the art, upon reference to the following detailed description. As such, the application should not be limited by the above description, but should be given broad scope.

[0090] It is to be understood that the application is not limited to the precise construction herein described and as shown in the attached drawings, and that various modifications and changes can be effected therein by those skilled in the art without departing from the scope of the application. The scope of the application is to be limited only by the claims appended hereto and construed according to the customary canons of claim interpretation. The above description is meant to be exemplary only and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the present application.

Claims

1. An interactive calibration method for roadside multi-sensor devices, characterized in that, The interactive calibration method is applied to an interactive calibration device that is communicatively connected to RTK equipment, radar, and cameras at the roadside. Receive the dot-mapping time sent by the RTK device, and receive the point cloud frames and image frames obtained by the radar and the camera based on the dot-mapping time; Target detection is performed on the point cloud frame and the image frame respectively to obtain the detection results of the probe points of the RTK device; If the detection result indicates that the probe point detection has failed, a failure message is generated and sent to the RTK device so that the RTK device can re-perform the point marking.

2. The interactive calibration method as described in claim 1, characterized in that, The method further includes: Upon receiving the timing data, the radar and camera are triggered to collect sensor data. It receives perception data from the radar and the camera, and obtains the point cloud frame and image frame whose timestamp is closest to the point-marking time from the perception data.

3. The interactive calibration method as described in claim 1, characterized in that, If the detection result indicates that the probe point detection failed, a prompt message indicating failure to mark the point is generated, specifically including: The detection results of the image frames determine whether the RTK device is occluded; If the RTK device is obstructed, an obstruction prompt message is generated indicating that the RTK device has failed to mark points, so that the RTK device can adjust the marking position and re-mark points.

4. The interactive calibration method as described in claim 3, characterized in that, The RTK device includes a cross-shaped calibration rod consisting of a centering rod and a crossbar, wherein the centering rod and the crossbar are telescopic, and / or the crossbar is rotatable to the left and right relative to the centering rod; If the RTK device is not obstructed, the prompt message indicating failure to generate the dot matrix includes: If the generation of the dot notification fails, the first prompt message indicating that the centering rod and / or the crossbar need to be adjusted should be provided, and / or... A second message appears indicating that the generation of dots failed and the horizontal angle of the crossbar needs to be adjusted.

5. The interactive calibration method as described in any one of claims 1-4, characterized in that, The method further includes: If the detection result indicates that the probe point was successfully detected, the point cloud coordinates, image coordinates, and GPS coordinates of the probe point are paired and saved; When the number of paired points exceeds a set threshold, calculate the initial transformation matrix required for coordinate point transformation between any two sensing devices; Verify the next pair of paired points based on the initial transformation matrix and obtain the mapping error; If the mapping error is greater than the set error threshold, a prompt to continue marking is generated and sent; if the mapping error is less than or equal to the set error threshold, a prompt to stop marking is generated and sent.

6. The interactive calibration method as described in claim 5, characterized in that, If the mapping error is greater than a set error threshold, the method further includes: Based on the initial transformation matrix, all paired points are verified to obtain the n paired points with the largest mapping error, where n≥3; The target marking area is determined based on the n pairing points, and the target marking area is sent to the RTK device so that the RTK device can continue marking within the target marking area.

7. The interactive calibration method as described in claim 5, characterized in that, If the mapping error is greater than a set error threshold, the method further includes: Build a dotted map based on all paired points; The dotted map is compared with the target detection area, and the areas in the target detection area that are not covered by dots or whose coverage is lower than the coverage threshold are taken as the target dotted area; The target marking area is sent to the RTK device so that the RTK device can continue marking within the target marking area.

8. An interactive calibration device, characterized in that, The interactive calibration device is communicatively connected to the RTK equipment, radar, and camera at the roadside. The interactive calibration device includes: The receiving unit is used to receive the dot-mapping time sent by the RTK device, and to receive the point cloud frames and image frames obtained by the radar and the camera based on the dot-mapping time. The detection unit is used to perform target detection on the point cloud frame and the image frame respectively, and obtain the detection results of the probe points of the RTK device; The generation unit is used to generate a prompt message indicating that the probe point detection has failed when the detection result indicates that the probe point detection has failed. A sending unit is used to send data to the RTK device so that the RTK device can re-mark data.

9. An electronic device, characterized in that, It includes a memory and one or more programs, wherein one or more programs are stored in the memory and are configured to be executed by one or more processors to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images