Data acquisition system

Abstract

The embodiment of the present application discloses a data acquisition system, which comprises: a laser radar; at least one camera; wherein the first camera is included; a trigger module, which is used for triggering the laser radar scanning to carry out point cloud acquisition, and triggering the first camera to carry out image acquisition when the scanning visual angle of the laser radar is in the visual angle range of the first camera, and triggering the first camera to carry out at least one additional image acquisition when the scanning visual angle of the laser radar is not in the visual angle range of the first camera, thereby realizing the synchronous acquisition of the point cloud and the image of the laser radar and the camera in the same visual angle range, and also realizing the additional image acquisition of the camera when the laser radar is in the visual angle range, so that more abundant image materials can be provided for the device, a higher frame rate playback video can be provided, and the accuracy and response speed of the algorithm related to the collected data are further improved.

Description

Technical Field

[0001] This invention relates to the field of electronic circuit technology, and in particular to a data acquisition system. Background Technology

[0002] Currently, radar and cameras can be installed in vehicles undergoing road testing or normal driving to simultaneously collect data. The radar can detect the corresponding point cloud, and the camera can capture the corresponding image. Since the viewing angles of both radar and camera are fixed, in the process of achieving synchronous data collection, the viewing angles of radar and camera can be set to be the same or similar, and the frequency of radar point cloud acquisition can be pre-set to be the same as the frequency of camera image acquisition to achieve synchronized exposure.

[0003] However, if a mechanical LiDAR is used, it needs to scan a circle to collect point clouds. After the LiDAR is triggered but before it reaches the camera's field of view, there is a period of time before it reaches the camera's field of view. This means that the LiDAR and the camera cannot synchronously acquire data from the same viewpoint. Therefore, when using a mechanical LiDAR and camera for image acquisition, it is often impossible to accurately achieve synchronous acquisition of point clouds and images from the same viewpoint, and to accurately achieve synchronous exposure of point clouds and images from the same viewpoint. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a data acquisition system that can not only achieve synchronous acquisition by lidar and camera from the same perspective, but also enable the camera to acquire additional images.

[0005] To address the aforementioned problems, in a first aspect, embodiments of the present invention provide a data acquisition system, comprising:

[0006] LiDAR;

[0007] At least one camera; including a first camera;

[0008] The triggering module is used to trigger the LiDAR to scan for point cloud acquisition, and to trigger the first camera to acquire images when the scanning angle of the LiDAR is within the field of view of the first camera, and to trigger the first camera to acquire images at least once when the scanning angle of the LiDAR is not within the field of view of the first camera.

[0009] Furthermore, in the data acquisition system, the triggering module includes a radar triggering unit and at least one camera triggering unit;

[0010] The camera trigger unit corresponds to the camera, and the camera trigger unit corresponding to the first camera is the target trigger unit; the radar trigger unit is electrically connected to the lidar, and the camera trigger unit is electrically connected to the corresponding camera;

[0011] The radar triggering unit is used to trigger the lidar to start scanning from the initial position using a first trigger signal;

[0012] The target triggering unit is used to acquire a second trigger signal and use the second trigger signal to trigger the first camera to perform image acquisition; the time difference between the second trigger signal and the first trigger signal is matched with the time it takes for the scanning angle of the lidar to scan from the initial position to the target position; the target position is within the field of view of the first camera;

[0013] The target triggering unit is also used to trigger the first camera to perform the additional image acquisition when the scanning angle of the lidar is not within the field of view of the first camera, based on the second trigger signal.

[0014] Furthermore, in the data acquisition system, the target triggering unit includes a delay module and an output module;

[0015] The delay module is electrically connected to the output module, and the output module is electrically connected to the first camera.

[0016] The delay module is used to acquire the second trigger signal, and after acquiring the second trigger signal for a target duration, output a third trigger signal to the output module;

[0017] The output module is used to trigger the first camera to perform image acquisition when the second trigger signal is received; and to trigger the first camera to perform the additional image acquisition when the third trigger signal is received.

[0018] Furthermore, in the data acquisition system, there are multiple delay modules, and the target duration of each delay module is different.

[0019] Furthermore, in the data acquisition system, the target triggering unit further includes at least one switch, and the delay module includes a delay unit;

[0020] Each of the delay units is connected between a switch and the output module;

[0021] The switch is used to transmit the second trigger signal to the corresponding delay only when it is turned on, so that the corresponding delay can receive the second trigger signal;

[0022] The delay unit is used to acquire the second trigger signal, delay the second trigger signal for the target duration, and then use the delayed second trigger signal as the third trigger signal to send the third trigger signal to the output module.

[0023] Furthermore, the data acquisition system also includes a configuration module for configuring the target duration.

[0024] Furthermore, in the data acquisition system, the configuration module configures the target duration based on at least one of the first trigger signal, the exposure duration of a single image acquisition by the first camera, and the requirement for the number of additional image acquisitions.

[0025] Furthermore, in the data acquisition system, the target triggering unit includes at least one trigger, and the delay module includes at least one timer; the output terminal of the trigger is electrically connected to the input terminal of the corresponding timer.

[0026] The trigger is used to acquire the second trigger signal and, in response to the second trigger signal, to trigger the corresponding timer to restart timing;

[0027] The timer is used to output the third trigger signal to the output module when the timer reaches the target duration.

[0028] Furthermore, in the data acquisition system, the number of triggers in the target triggering unit is two, namely a first trigger and a second trigger, and the number of timers in the delay module is two, namely a first timer connected to the first trigger and a second timer connected to the second trigger.

[0029] The input of the second flip-flop is also connected to the output of the first timer, and the input of the first flip-flop is also connected to the output of the second timer;

[0030] The second trigger is also used to acquire the third trigger signal output by the first timer, and in response to the acquired third trigger signal, to trigger the second timer to restart timing;

[0031] The first trigger is also configured to acquire a third trigger signal output by the second timer, and to trigger the first timer to restart timing in response to the acquired third trigger signal.

[0032] Furthermore, the data acquisition system also includes a configuration module;

[0033] The configuration module is used to acquire external environmental information of the vehicle and determine additional exposure information of the first camera based on the external environmental information; the additional exposure information is used to at least indicate whether the first camera needs to perform the additional image acquisition.

[0034] The triggering module only triggers the first camera to perform at least one additional image acquisition when the scanning angle of the lidar is not within the field of view of the first camera, and only when the additional exposure information indicates that the first camera needs to perform the additional image acquisition.

[0035] The data acquisition system provided in this application includes a LiDAR, at least one camera, and a trigger module. The trigger module is used to trigger the LiDAR to scan for point cloud acquisition. When the scanning angle of the LiDAR is within the field of view of a camera, the camera is triggered to acquire images. When the scanning angle of the LiDAR is not within the field of view of the camera, the camera is triggered to acquire at least one additional image. This achieves synchronous acquisition of point clouds and images from the LiDAR and camera within the same field of view. It also enables the camera to acquire additional images when the LiDAR is within its field of view, increasing the number of images acquired by the camera. This provides the device with richer image material, higher frame rate playback video, and further improves the accuracy and response speed of algorithms related to the acquired data. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic block diagram of a data acquisition system provided in an embodiment of the present invention;

[0038] Figure 2 This is a schematic block diagram illustrating the setup of a lidar and a camera according to an embodiment of the present invention.

[0039] Figure 3 This is a schematic block diagram of a data acquisition system provided in an embodiment of the present invention;

[0040] Figure 4 A timing diagram showing the synchronous acquisition of data by a lidar and a camera, provided in an embodiment of the present invention;

[0041] Figure 5 Another schematic block diagram of the data acquisition system provided in the embodiment of the present invention;

[0042] Figure 6 Another schematic block diagram of the data acquisition system provided in the embodiment of the present invention;

[0043] Figure 7 This is a schematic block diagram of the target triggering part provided in an embodiment of the present invention;

[0044] Figure 8 A schematic block diagram of a target triggering part provided in an embodiment of the present invention;

[0045] Figure 9 This is another schematic block diagram of the target triggering part provided in an embodiment of the present invention;

[0046] Figure 10 This is another schematic block diagram of the target triggering part provided in an embodiment of the present invention;

[0047] Figure 11 This is a timing diagram of camera image acquisition provided in an embodiment of the present invention;

[0048] Figure 12 This is a timing diagram of image acquisition by a camera provided in an embodiment of the present invention;

[0049] Figure 13 This is another timing diagram of camera image acquisition provided in an embodiment of the present invention;

[0050] Figure 14 This is another timing diagram of camera image acquisition provided in an embodiment of the present invention;

[0051] Figure 15 Another timing diagram for image acquisition by a camera provided in an embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0054] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0055] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0056] As analyzed in the background section of this application, in the prior art, during the synchronous acquisition process of LiDAR and camera from the same perspective, the number of images acquired by the camera is relatively small. To solve the above-mentioned technical problem, this application provides a data acquisition system.

[0057] Please see Figure 1 , Figure 1 This is a schematic block diagram of a data acquisition system provided in an embodiment of the present invention. Figure 1 As shown, a data acquisition system includes:

[0058] LiDAR 10;

[0059] At least one camera; including a first camera 20;

[0060] The trigger module 30 is used to trigger the LiDAR 10 to scan for point cloud acquisition, and to trigger the first camera 20 to acquire images when the scanning angle of the LiDAR 10 is within the field of view of the first camera 20, and to trigger the first camera 20 to acquire images at least once when the scanning angle of the LiDAR 10 is not within the field of view of the first camera 20.

[0061] The scanning angle can be understood as the direction of the laser emitted by the lidar, or the rotation angle of the lidar around its axis. For example, the laser points of the lidar are arranged to form a scanning vertical line. Rotating around the axis once can satisfy a 360-degree scanning range. Under different rotation angles of the lidar, the scanning vertical line faces different directions, which is the scanning angle. For another example, the scanning angle can also be understood as the direction of the laser point at the center of the lidar.

[0062] In one embodiment, such as Figure 2As shown, three cameras can be installed on the vehicle. The arrangement of the three cameras and the LiDAR 10 can be configured as follows: the three cameras are arranged around the LiDAR 10, or the three cameras are located at different positions on the outer perimeter of the vehicle and face different directions, with the LiDAR 10 installed on the roof. Regardless of the layout, in one example, the three cameras can achieve 360-degree image acquisition, meaning that the field of view of the three cameras overlaps, which is suitable for stitching together to form a 360-degree surround view image. Of course, the field of view of the three cameras may not be suitable for stitching, for example, each covering different, non-overlapping, and non-connectable field of view. Furthermore, any one or more of these cameras can serve as the first camera mentioned in this specification. Figure 2 This example uses only one first camera. In actual applications, two or three cameras can also be used as first cameras. The number of cameras in the data acquisition system can be 1, 2, 3, 4 or more, and the number of first cameras can be 1, 2, 3, 4 or more.

[0063] The lidar 10 is a mechanical lidar. During the scanning process initiated by the trigger module 30, the scanning angle of the lidar 10 is... Figure 2 When the first camera 20 is within the field of view of any of the cameras, such as the first camera 20, the trigger module 30 can trigger the first camera 20 to perform image acquisition. At the same time, when the first camera 20 is not within its field of view, the trigger module 30 can also trigger the first camera 20 to perform at least one additional image acquisition.

[0064] Preferably, after the scanning angle of the lidar 10 leaves the field of view of the first camera 20, or before, the trigger module 30 triggers the first camera 20 to perform at least one additional image acquisition.

[0065] The trigger module 30 can detect the rotation angle of the LiDAR 10 when triggering the LiDAR 10 and the camera to perform corresponding data acquisition. For example, when the rotation angle of the LiDAR 10 reaches the viewing angle range of a certain camera, the trigger module 30 triggers the corresponding camera to perform image acquisition. Simultaneously, the trigger module 30 can also output corresponding waveform signals according to the scanning cycle of the LiDAR 10. The time difference between the rising edges of the waveform signals used to trigger the LiDAR 10 and the camera must meet the following condition: the LiDAR 10 must scan into the viewing angle range corresponding to a certain camera before triggering the corresponding camera to expose.

[0066] It should be noted that the number of cameras and the position between the cameras and the LiDAR 10 in this application can be selected according to the actual application, and this application does not impose specific limitations.

[0067] For example, the number of cameras in this application can be one. That is, when the scanning angle of the LiDAR 10 is within the field of view of the camera, the trigger module 30 can trigger the camera to perform image acquisition; when the scanning angle of the LiDAR 10 leaves the field of view of the camera, the trigger module 30 can trigger the camera to perform additional image acquisition. At this time, the image acquired by the camera does not have a matching point cloud, but this type of image can effectively enrich the number of image materials. On the same basis, this type of image can also be used for playback, thereby providing higher frame rate video for playback. This type of image can also be used for algorithm recognition, control, and even back-injection, thereby improving the accuracy and response speed of algorithms that require this type of image.

[0068] For example, the number of cameras in this application can be two, meaning that an additional camera is added to the first camera 20. During the process of triggering the LiDAR 10 to scan for point cloud acquisition, when the scanning angle of the LiDAR 10 is within the field of view of the first camera 20, the trigger module 30 can trigger the first camera 20 to acquire images, and may or may not trigger the other camera to acquire images. When the scanning angle of the LiDAR 10 is not within the field of view of the first camera 20, the trigger module 30 can either trigger the first camera 20 to acquire images additionally, or trigger the other camera to acquire images. In this case, the trigger module 30 can trigger the other camera to acquire images either as an additional acquisition or because the LiDAR 10's scanning angle is within the field of view of the other camera.

[0069] The data acquisition system provided in this application consists of a LiDAR 10, at least one camera, and a trigger module 30. The trigger module 30 is used to trigger the LiDAR 10 to scan for point cloud acquisition. When the scanning angle of the LiDAR 10 is within the field of view of a camera, the camera is triggered to acquire images. When the scanning angle of the LiDAR 10 is not within the field of view of the camera, the camera is triggered to acquire at least one additional image. This achieves synchronous acquisition of point clouds and images from the LiDAR 10 and the camera within the same field of view. It also enables the camera to acquire additional images when the LiDAR 10 is within its field of view, increasing the number of images acquired by the camera. This provides the device with richer image materials and higher frame rate playback video, further improving the accuracy and response speed of algorithms related to the acquired data.

[0070] In some embodiments, such as Figure 3As shown, the trigger module 30 includes a radar trigger unit 301 and at least one camera trigger unit; wherein, the camera trigger unit corresponds to a camera, and the camera trigger unit corresponding to the first camera 20 is a target trigger unit 302; the radar trigger unit 301 is electrically connected to the lidar 10, and the camera trigger unit is electrically connected to the corresponding camera; the radar trigger unit 301 is used to trigger the lidar 10 to start scanning from an initial position using a first trigger signal, which can also be understood as sending a first trigger signal (e.g., rising edge or falling edge) to the lidar 10 to trigger the lidar 10 to start scanning from an initial position; the target trigger unit 302 is used to acquire a second trigger signal, and in response to the second trigger signal, trigger the first camera 20 to perform image acquisition, which can also be understood as: in response to the second trigger signal, triggering the first camera 20 to perform image acquisition. The first camera 20 is triggered to acquire an image by sending a trigger signal (e.g., a rising edge or a falling edge). This trigger signal can be the second trigger signal itself, or a rising edge or falling edge generated in response to the second trigger signal. The time of its transmission can be understood as being approximately the same as the time of acquiring the second trigger signal. In addition, the first trigger signal can also be given to the camera triggering unit as the second trigger signal. The time difference between the second trigger signal and the first trigger signal is matched with the time it takes for the scanning angle of the lidar 10 to scan from the initial position to the target position. The target position is within the field of view of the first camera 20. The target triggering unit 302 is also used to trigger the first camera 20 to acquire additional images when the scanning angle of the lidar 10 is not within the field of view of the first camera 20 based on the second trigger signal.

[0071] In this embodiment, the radar triggering unit 301 mainly uses the first trigger signal to trigger the lidar 10 to scan for point cloud acquisition, and the camera triggering unit mainly responds to the second trigger signal to trigger its corresponding camera to perform image acquisition, or triggers its corresponding camera to perform additional image acquisition. The time difference between the second trigger signal and the first trigger signal is matched with the time it takes for the lidar 10 to scan the target position from the initial position.

[0072] Among them, the camera triggering unit corresponding to the first camera 20 is the target triggering unit 302. When the scanning angle of the lidar 10 is within the field of view of any camera, such as the field of view of the first camera 20, the target triggering unit 302 responds to the second trigger signal to trigger the first camera 20 to acquire an image. At the same time, the other camera triggering units use the second trigger signal to trigger the corresponding camera to perform at least one additional image acquisition. When the scanning angle of the lidar 10 is not within the field of view of the first camera 20, the target triggering unit 302 can use the second trigger signal to trigger the first camera 20 to perform at least one additional image acquisition. At the same time, the other camera triggering units select whether to use the second trigger signal to trigger the corresponding camera to perform image acquisition or additional image acquisition based on whether the scanning angle of the lidar 10 is within the field of view of the corresponding camera.

[0073] In addition, this application also allows the camera triggering unit to trigger the camera corresponding to the viewpoint to perform data acquisition simply by using the second trigger signal when the scanning viewpoint of the lidar 10 is within the viewpoint range of any camera. After the scanning viewpoint of the lidar 10 leaves the viewpoint range of one camera and enters the viewpoint range of another camera, the camera triggering unit uses the second trigger signal to trigger the other camera to perform image acquisition.

[0074] It should be noted that, regardless of whether it is the first camera 20 involved in this specification, each camera can be triggered by the corresponding camera trigger unit in response to the second trigger signal. For example... Figure 4 As shown, cameras 1, 2, and 3 are arranged around the lidar 10. In the initial stage of lidar 10 scanning and acquiring point clouds, triggered by the radar triggering unit 301, if the scanning angle of lidar 10 is within the field of view of camera 1, the camera triggering unit of camera 1 uses a second trigger signal to trigger camera 1 to acquire images. Lidar 10 continues to rotate and scan. If the scanning angle of lidar 10 leaves the field of view of camera 1 and moves into the field of view of camera 2, the camera triggering unit of camera 2 uses a second trigger signal to trigger camera 2 to acquire images. Similarly, if the scanning angle of lidar 10 leaves the field of view of camera 2 and moves into the field of view of camera 3, the camera triggering unit of camera 3 uses a second trigger signal to trigger camera 3 to acquire images. After lidar 10 completes one cycle of scanning, if the radar triggering unit 301 triggers lidar 10 to scan and acquire point clouds again using the first trigger signal, the above process is repeated, and lidar 10 performs one cycle of scanning.

[0075] In this embodiment, both the first trigger signal and the second trigger signal are high-level signals (or can be understood as rising edge signals). The time difference between the rising edges of the second trigger signals that trigger cameras 1, 2, and 3 can be equal. The first trigger signal can be directly used as a second trigger signal to trigger a camera to acquire images. At the same time, a time difference can be formed between the first trigger signal and the second trigger signal. This time difference can be equal to the time it takes for the scanning angle of the lidar 10 to scan the target position from the initial position. The specific time difference can be selected according to the actual application. This embodiment does not impose any specific limitations.

[0076] In some embodiments, the radar trigger unit 301 may also be electrically connected to some or all of the camera trigger units. For example... Figure 5 As shown, the radar trigger unit 301 is electrically connected to the target trigger unit 302, and also electrically connected to other camera trigger units. That is to say, when the radar trigger unit 301 uses the first trigger signal to trigger the lidar 10 to scan and collect point clouds, the radar trigger unit 301 can also trigger the camera trigger units, such as triggering the target trigger unit 302, so that the target trigger unit 302 can use the second trigger signal to trigger the first camera 20 to collect images or trigger the first camera 20 to collect additional images.

[0077] In some embodiments, the radar trigger unit 301 may also be electrically connected to some or all of the camera trigger units via the lidar 10, such as... Figure 6 As shown, the radar trigger unit 301 is electrically connected to the target trigger unit 302 via the lidar 10, and is also electrically connected to other camera trigger units via the lidar 10.

[0078] In other words, when the radar triggering unit 301 uses the first trigger signal to trigger the lidar 10 to scan and collect point clouds, it also triggers the camera triggering unit through the lidar 10, such as triggering the target triggering unit 302 through the lidar 10, thereby causing the target triggering unit 302 to use the second trigger signal to trigger the first camera 20 to collect images or to trigger the first camera 20 to collect additional images.

[0079] In some embodiments, such as Figure 7 As shown, the target triggering unit 302 includes a delay module 3021 and an output module 3022; wherein, the delay module 3021 is electrically connected to the output module 3022, and the output module 3022 is electrically connected to the first camera 20; the delay module 3021 is used to acquire a second trigger signal, and after a target duration following the acquisition of the second trigger signal, outputs a third trigger signal to the output module 3022; the output module 3022 is used to trigger the first camera 20 to perform image acquisition when the second trigger signal is acquired, and to trigger the first camera 20 to perform additional image acquisition when the third trigger signal is acquired.

[0080] In this embodiment, the first trigger signal can be directly used as the second trigger signal. Thus, the first camera 20 can be triggered to perform image acquisition in response to the second trigger signal. At the same time, after being delayed by the delay module 3021, the second trigger signal can be used as a delayed second trigger signal (i.e., the third trigger signal) to trigger the first camera 20 to perform at least one additional image acquisition.

[0081] When the second and third trigger signals are high (or understood as rising edges), the output module 3022 can be an OR gate, but is not limited to an OR gate. For example, when the output module 3022 receives the high level of the second trigger signal, it can output a high level to the first camera 20 to trigger the first camera 20 to perform image acquisition. At the same time, the second trigger signal can be delayed by controlling the delay module 3021, and after the delay is completed, it becomes the third trigger signal. When the output module 3022 receives the high level of the third trigger signal, it can output a high level to the first camera 20 to trigger the first camera 20 to perform additional image acquisition.

[0082] In addition, there can be multiple delay modules 3021, and the target duration of different delay modules 3021 can be different, thereby enabling the first camera 20 to perform at least one additional image acquisition.

[0083] In some embodiments, such as Figure 7 As shown, the data acquisition system also includes a configuration module 3023, which is used to configure the target duration.

[0084] In this embodiment, the target duration is the delay duration of the delay module 3021. The configuration module 3023 is mainly used to configure the target duration of the delay module 3021 and / or whether to perform a delay. The target duration is mainly to enable the camera to perform additional image acquisition. That is, after the configuration module 3023 completes the configuration of the target duration, the delay module 3021 obtains the second trigger signal, and after obtaining the target duration after the second trigger signal, it can output the third trigger signal to the output module 3022. Through the third trigger signal, additional image acquisition by the camera can be achieved.

[0085] Furthermore, in one embodiment, the configuration module 3023 configures the target duration based on at least one of the first trigger signal, the exposure duration of a single image acquisition by the camera, and the requirement for the number of additional image acquisitions (which can be understood as representing the number of additional image acquisitions during the process of the lidar rotating 10 revolutions per cycle).

[0086] Preferably, this application can calculate the scanning period t or trigger frequency 1 / t of the lidar 10 based on the first trigger signal, determine the minimum exposure time based on the exposure time of the next image acquisition by the camera, and finally determine the number of times the camera needs to be configured to acquire additional images based on the number of additional image acquisitions.

[0087] In some embodiments, the configuration module 3023 is further configured to acquire external environment information of the vehicle and determine additional exposure information of the camera based on the external environment information; the additional exposure information is used to at least indicate whether the camera needs to perform additional image acquisition; the trigger module 30 triggers the camera to perform at least one additional image acquisition only when the additional exposure information indicates that the camera needs to perform additional image acquisition, and the scanning angle of the lidar 10 is not within the field of view of the camera.

[0088] Furthermore, additional exposure information can also be used to characterize the number of additional image acquisitions. Moreover, the number of acquisitions can be determined based on external environmental information.

[0089] In this embodiment, the number of additional image acquisitions performed by the camera can be a fixed value, or it can be an adaptive value that varies according to the vehicle's driving conditions and the external environment. It can be set by analyzing the point cloud previously acquired by the lidar 10 and / or the images acquired by the camera.

[0090] For example, the number of times the camera performs additional image acquisition during the day can be 0, and the number of times it performs additional image acquisition at night can be 1, 2, or 3. That is, the camera may not be triggered to perform additional image acquisition during the day, but may be triggered to perform 1, 2, or 3 additional image acquisitions at night.

[0091] For example, the camera can perform additional image acquisition once during the day and twice or three times at night. That is, the camera can be triggered to perform additional image acquisition both during the day and at night, but it is triggered more times at night.

[0092] It should be noted that the above-mentioned daytime and nighttime can also be replaced by ambient brightness, and the specific choice can be made according to the actual application. This embodiment does not make specific limitations.

[0093] As can be seen, external environmental information includes current time information and / or ambient brightness. Current time information can be used to indicate whether it is day or night, or what time it is. Ambient brightness can be used to distinguish whether it is day or night.

[0094] For example, after detecting that the vehicle is not moving, this application may also begin to determine whether any target object (such as a person, vehicle, etc.) has moved relative to the vehicle, and then determine the number of times the camera will perform additional image acquisition based on the movement.

[0095] Specifically, the number of additional image captures by the camera can be determined by the magnitude of the target object's positional movement and the number of target objects that have moved.

[0096] For example, if no target object moves, the camera will capture 0 additional images, meaning no additional image capture is required. If the target object moves, but the magnitude and number of target objects do not exceed the upper limit, the camera can capture additional images, but the number of additional image captures can be a small value, such as 1. If the target object moves, but the magnitude or number exceeds the upper limit, the camera can also capture additional images, and the number of additional image captures can be a large value, such as 2. If the vehicle starts moving, the camera can also capture additional images, and the number of additional image captures can be an even larger value, such as 3.

[0097] This shows that by setting the number of additional image captures by the camera based on whether the target object and the vehicle are moving, the camera can avoid constantly performing a large number of additional captures, thereby reducing the rate of camera aging and wear.

[0098] It is evident that external environmental information includes at least one of the following: information on whether a target object has moved; information on the number, magnitude, and type of the moving target object (e.g., information indicating people or vehicles).

[0099] In addition, the number of times the lidar 10 collects point clouds can also be designed with reference to the scheme of the additional image collection by the camera in this embodiment.

[0100] Specifically, after detecting that the vehicle is not moving, it can be determined whether any target object has moved relative to the vehicle. Then, based on the movement, the number of point cloud acquisitions required by the LiDAR 10 can be determined. For example, the number of point cloud acquisitions by the LiDAR 10 can be preset as multiple levels. If no target object moves, the lowest level 1 can be used; if movement occurs but the magnitude and number of target objects do not exceed the upper limit, level 2 can be used; if movement occurs and the magnitude or number exceeds the upper limit, level 3 can be used; and if the vehicle starts moving, level 4 can be used. In other words, as the trigger frequency of the LiDAR 10 changes, the additional acquisitions by the camera 20 can also be adaptively adjusted.

[0101] In some embodiments, such as Figure 8 As shown, the target triggering unit 302 also includes at least one switch, and the delay module 3021 includes a delay unit; each delay unit is connected between a switch and the output module 3022; the switch is used to transmit the second trigger signal to the corresponding delay unit only when it is turned on, so that the corresponding delay unit can obtain the second trigger signal; the delay unit is used to obtain the second trigger signal, delay the second trigger signal for a target duration, and use the delayed second trigger signal as the third trigger signal to send the third trigger signal to the output module 3022.

[0102] In this embodiment, the camera can perform additional image acquisition using a timer and a switch K. The on / off state of switch K can be configured using the configuration module 3023. After the first trigger signal enters the target trigger unit 302, if the camera needs to perform additional image acquisition, switch K can close upon receiving the first trigger signal and transmit the second trigger signal to the corresponding delay unit. After being delayed by the delay unit to the target duration Δt, the delay unit sends the second trigger signal as the third trigger signal to the output module 3022, so that the output module 3022 triggers the camera to perform additional image acquisition.

[0103] For example, taking the scanning period t of the lidar 10 as an example, if the camera needs to perform two additional image acquisitions, the number of delay units needs to be set to at least two. The target duration Δt of the two delay units can be t / 3. Then, after the control switch K is turned on at different times, the second trigger signal can be sent to the output module 3022 through the two delay units respectively, and the output module 3022 will sequentially trigger the camera to perform two additional image acquisitions.

[0104] In some embodiments, such as Figure 9 As shown, the target triggering unit 302 includes at least one trigger 3024, and the delay module 3021 includes at least one timer; the output terminal of the trigger 3024 is electrically connected to the input terminal of the corresponding timer; wherein, the trigger 3024 is used to acquire a second trigger signal and trigger the corresponding timer to restart timing in response to the second trigger signal; the timer is used to output a third trigger signal to the output module 3022 when the timing reaches the target duration.

[0105] The input terminal of the timer can be understood as the connection terminal that can trigger the timer to restart timing, or it can be described as the trigger terminal.

[0106] In this embodiment, when the camera performs additional image acquisition, it can be achieved through a trigger 3024 and a timer. The first trigger signal can be directly used as the second trigger signal and sent to the trigger 3024. The trigger 3024 responds to the second trigger signal to trigger the corresponding timer to restart the timing. After the corresponding timer reaches the required target duration Δt, the second trigger signal can be sent to the output module 3022. The output module 3022 can directly use the second trigger signal as the third trigger signal to trigger the camera to perform additional image acquisition.

[0107] Different timers can be set to time different target durations Δt, and trigger 3024 can be selectively activated to trigger the camera to perform different numbers of additional image acquisitions.

[0108] It should be noted that the number of timers and delay units, as well as the target duration Δt for timing and delay, can all be selected according to requirements. These can be set manually or automatically, referencing the cycle of the mechanical LiDAR 10. For example, if the cycle of the mechanical LiDAR 10 is t, the number of additional image acquisitions by the camera is 1, and the number of timers / delay units is 1, the target duration Δt can be t / 2. If the number of additional image acquisitions by the camera is not less than 2, then the number of timers / delay units usually also needs to be greater than or equal to 2.

[0109] It should also be noted that the switch K mentioned in the above embodiments can be a transistor, MOSFET, relay, or other components. The selection of switch K can be made according to the actual application, and this application does not impose any specific limitations.

[0110] In addition, the configuration module 3023 can also determine whether to activate other triggers, such as trigger 2, based on the number of times the camera performs additional image acquisition. For example, if the camera performs one additional image acquisition, then trigger 2 and its corresponding timer 2 do not need to be activated.

[0111] In one embodiment, such as Figure 10 As shown, the target triggering unit 302 has two triggers 3024, namely a first trigger and a second trigger. The delay module 3021 has two timers, namely a first timer connected to the first trigger and a second timer connected to the second trigger. The input terminal of the second trigger is also connected to the output terminal of the first timer, and the input terminal of the first trigger is also connected to the output terminal of the second timer. The second trigger is also used to acquire a third trigger signal output by the first timer, and in response to the acquired third trigger signal, it triggers the second timer to restart timing. The first trigger is also used to acquire a third trigger signal output by the second timer, and in response to the acquired third trigger signal, it triggers the first timer to restart timing.

[0112] In this embodiment, the target triggering unit 302 employs two triggers 3024 and two timers. The two triggers 3024 can be triggered alternately, thereby enabling the camera to perform at least two additional image acquisitions. This allows for the use of fewer device resources and less area consumption to satisfy various possible values ​​for the number of additional image acquisitions by the camera.

[0113] The trigger 3024 can be configured by the configuration module 3023 to trigger the corresponding timer to restart when a rising edge is detected at the input of the trigger 3024. This function can be achieved by selecting appropriate components or by properly configuring the peripheral circuitry of the timer.

[0114] Simultaneously, the timer can also be configured by the configuration module 3023 to: once the timer is triggered by the trigger, it needs to restart timing regardless of whether it was timing before; and the timer will only emit a pulse when the timing reaches the target duration Δt, at which point a high-level signal output will occur. Before and after emitting the pulse, the timer can maintain a low-level output. This function can be achieved by selecting a suitable timer or by properly configuring the timer's peripheral circuitry.

[0115] For example, if the target triggering unit 302 includes timer 1, timer 2, trigger 1 and trigger 2, the output of timer 1 can be input to the input terminal of trigger 2, and the output of timer 2 can be input to the input side of trigger 1. Then, the rising edge of the pulse output by timer 1 can trigger trigger 2 to restart timer 2, and the rising edge of the pulse output by timer 2 can also trigger trigger 1 to restart timer 1. At the same time, the rising edge of the pulse of the second trigger signal can also trigger trigger 1 to restart timer 1.

[0116] For ease of understanding of the above embodiments, as follows Figure 11 , Figure 12 as well as Figure 13As shown, if the camera needs to perform two additional image acquisitions, the period of the LiDAR 10 is t. The target duration Δt of the two timers can be approximately t / 3. Both timers can be set to t / 3 or slightly greater than t / 3. After a pulse of the first trigger signal is input to the camera trigger unit, it can achieve one image acquisition by the camera via the output module 3022. Simultaneously, the first trigger signal can be detected at the input of trigger 1, causing trigger 1 to restart timer 1. When timer 1 counts to t / 3 or slightly greater than t / 3, it outputs a pulse. This pulse then achieves one additional image acquisition by the camera via the output module 3022. At the same time, the pulse output by timer 1 is also input to trigger 2. When trigger 2 detects a rising edge, it triggers timer 2 to restart timer 2. When timer 2 counts to t / 3 or slightly greater than t / 3, it outputs a pulse. This pulse then achieves another additional image acquisition by the camera via the output module 3022. Through the above process, the camera can achieve one synchronous image acquisition and two additional image acquisitions within one scanning cycle of the lidar 10. Then, the first trigger signal of the next scanning cycle of the lidar 10 will be input to the camera trigger unit, and the trigger 1 will trigger the timer 1 to start timing again, repeating the above process.

[0117] Among them, with Figure 11 compared to, Figure 12 The target duration Δt of trigger 1 is slightly greater than t / 3. Figure 13 The target duration Δt of trigger 2 is slightly greater than t / 3. The main purpose of this design is to ensure that tx in the figure is less than t / 3, thereby avoiding the timer 1 from triggering trigger 2 with a rising edge at time t. If trigger 2 is triggered, the above cycle will not be realized, and the calibration function for each cycle of the lidar 10 will also not be effectively realized.

[0118] In one embodiment, such as Figure 14 As shown, when the camera needs to capture three additional images, it can use... Figure 14 It is implemented using a timing diagram.

[0119] exist Figure 14In the illustrated embodiment, the target duration of both timers can be t / 4, meaning the target duration Δt of both timers can be set to t / 4. After the first trigger signal is input to the camera trigger unit, it achieves the first image acquisition of the camera within one scan cycle of the LiDAR 10 via the output module 3022. Simultaneously, when the input terminal of the trigger 1 detects the first trigger signal, the trigger 1 can trigger the timer 1 to start timing. When the timer 1 counts to t / 4, it outputs a pulse. At this time, the pulse achieves the first additional image acquisition of the camera within one scan cycle of the LiDAR 10 via the output module 3022. Meanwhile, the output of the timer 1... The output pulse is also input to trigger 2. When trigger 2 detects a rising edge, it triggers timer 2 to restart the timing. When timer 2 counts to t / 4, it outputs a pulse. This pulse, through output module 3022, enables the second additional image acquisition by the camera within one scan cycle of the LiDAR 10. At this time, the pulse output by timer 2 is also sent to trigger 1, which triggers timer 1 to restart the timing. When timer 1 counts to t / 4, it outputs a pulse. This pulse, through output module 3022, enables the third additional image acquisition by the camera within one scan cycle of the LiDAR 10. Through this process, four image acquisitions by the camera can be achieved within one scan cycle of the LiDAR 10, including three additional image acquisitions. Finally, the first trigger signal of the next scan cycle of the LiDAR 10 is input to the camera trigger unit, which in turn triggers timer 1 to restart the timing, repeating the aforementioned process.

[0120] In the above embodiment, trigger 2 can only be activated when the number of additional image acquisitions by the camera is greater than 1, and can only participate in the aforementioned process after being activated.

[0121] In one embodiment, such as Figure 15 As shown, when the camera needs to perform four additional image captures, it can use... Figure 15 This is implemented using a sequence diagram. Figure 15 In the illustrated embodiment, the process of the camera acquiring images can be understood by referring to the process of the camera acquiring images twice more, and will not be described in detail here.

[0122] As can be seen from the above embodiments, when the number of additional image acquisitions by the camera is even, the target duration Δt of any trigger 3024 can preferably be slightly greater than t / (n+1), where n is the number of additional image acquisitions by the camera. The magnitude of "slightly greater" can be arbitrarily chosen according to requirements, as long as it does not affect the number of additional image acquisitions by the camera. For example, it can be 105% or 110% of t / (n+1).

[0123] It should be noted that, in one embodiment, each cycle can be delayed and timed based on the first trigger signal (or can be understood as the rising edge), so that each cycle will have a calibration, thereby avoiding the adverse effect of the cumulative error of delay and timing on the trigger timing.

[0124] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A data acquisition system, characterized by, include: LiDAR; At least one camera; including a first camera; The triggering module is used to trigger the LiDAR to scan for point cloud acquisition, and to trigger the first camera to acquire images when the scanning angle of the LiDAR is within the field of view of the first camera, and to trigger the first camera to acquire images at least once when the scanning angle of the LiDAR is not within the field of view of the first camera. Wherein, the camera triggering unit corresponding to the first camera is a target triggering unit; the target triggering unit includes a delay module; the target triggering unit has two triggers, namely a first trigger and a second trigger, and the delay module has two timers, namely a first timer connected to the first trigger and a second timer connected to the second trigger; The input of the second flip-flop is also connected to the output of the first timer, and the input of the first flip-flop is also connected to the output of the second timer; The second trigger is also used to acquire the third trigger signal output by the first timer, and in response to the acquired third trigger signal, to trigger the second timer to restart timing; The first trigger is also configured to acquire a third trigger signal output by the second timer, and to trigger the first timer to restart timing in response to the acquired third trigger signal.

2. The data acquisition system of claim 1, wherein, The triggering module includes a radar triggering unit and at least one camera triggering unit; The camera trigger unit corresponds to the camera, the radar trigger unit is electrically connected to the lidar, and the camera trigger unit is electrically connected to the corresponding camera. The radar triggering unit is used to trigger the lidar to start scanning from the initial position using a first trigger signal; The target triggering unit is used to acquire a second trigger signal and, in response to the second trigger signal, trigger the first camera to perform image acquisition; the time difference between the second trigger signal and the first trigger signal is matched with the time it takes for the scanning angle of the lidar to scan from the initial position to the target position; the target position is within the field of view of the first camera; The target triggering unit is also used to trigger the first camera to perform the additional image acquisition when the scanning angle of the lidar is not within the field of view of the first camera, based on the second trigger signal.

3. The data acquisition system of claim 2, wherein, The target triggering unit includes an output module; The delay module is electrically connected to the output module, and the output module is electrically connected to the first camera. The delay module is used to acquire the second trigger signal, and after acquiring the second trigger signal for a target duration, output a third trigger signal to the output module; The output module is used to trigger the first camera to perform image acquisition when the second trigger signal is received; and to trigger the first camera to perform the additional image acquisition when the third trigger signal is received.

4. The data acquisition system of claim 3, wherein, There are multiple delay modules, and the target duration of each delay module is different.

5. The data acquisition system of claim 3, wherein, The target trigger unit further includes at least one switch, and the delay module includes a delay unit; Each of the delay units is connected between a switch and the output module; The switch is used to transmit the second trigger signal to the corresponding delay only when it is turned on, so that the corresponding delay can receive the second trigger signal; The delay unit is used to acquire the second trigger signal, delay the second trigger signal for the target duration, and then use the delayed second trigger signal as the third trigger signal to send the third trigger signal to the output module.

6. The data acquisition system of claim 3, wherein, It also includes a configuration module, which is used to configure the target duration.

7. The data acquisition system of claim 6, wherein, The configuration module configures the target duration based on at least one of the first trigger signal, the exposure duration of a single image acquisition by the first camera, and the requirement for the number of additional image acquisitions.

8. The data acquisition system of any one of claims 3 to 7, wherein, The target triggering unit includes at least one trigger, and the delay module includes a timer; the output terminal of the trigger is electrically connected to the input terminal of the corresponding timer. The trigger is used to acquire the second trigger signal and, in response to the second trigger signal, to trigger the corresponding timer to restart timing; The timer is used to output the third trigger signal to the output module when the timer reaches the target duration.

9. The data acquisition system of claim 1, wherein, It also includes a configuration module; The configuration module is used to acquire external environmental information of the vehicle and determine additional exposure information of the first camera based on the external environmental information; The additional exposure information is used to at least indicate whether the first camera needs to perform the additional image acquisition. The triggering module only triggers the first camera to perform the additional image acquisition when the scanning angle of the lidar is not within the field of view of the first camera, and only when the additional exposure information indicates that the first camera needs to perform the additional image acquisition.

Images