A data synchronization acquisition method, control module, system and storage medium

Abstract

The present invention discloses a data synchronization acquisition method, control module, system and storage medium, wherein the method comprises: obtaining the rotation direction of a laser radar; generating an adjustment signal for controlling a rotatable rolling shutter camera to rotate to a preset state according to the rotation direction of the laser radar; wherein the output result of the rotatable rolling shutter camera in the preset state is consistent with the output result of the laser radar in the stretching direction of the moving object; obtaining the rotation speed of the laser radar and the shooting parameters of the rotatable rolling shutter camera; setting the shooting frame rate and shooting time of the rotatable rolling shutter camera according to the rotation speed and the shooting parameters; generating a shooting signal for controlling the shooting of the rotatable rolling shutter camera according to the shooting frame rate and the shooting time. The present invention realizes the function of synchronizing the pixel points measured by the camera with the laser radar point cloud acquisition, which can greatly reduce the difficulty and calculation amount of the information fusion between the camera and the laser radar, and improve the reliability of the results.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a data synchronization acquisition method, a control module, a system and a storage medium. Background Art

[0002] At present, most autonomous driving vehicles use cameras and lidars for target perception. There is a certain time difference when collecting the pixel points of the same frame image of the camera and the point cloud of the lidar. Therefore, when the target to be measured moves at high speed, the contour of the object will be deformed, and the time of each point of the two sensors cannot be effectively synchronized.

[0003] In practical applications, rolling shutter exposure cameras are widely used due to factors such as cost and sensitivity. Figure 1 is a schematic diagram of the rolling shutter exposure time in the prior art, specifically as follows Figure 1 As shown, the rolling shutter exposure camera adopts an independent exposure method for each row, where t V is the single frame image time, t exp is the exposure time, t H is the single line output time, and t H It is also the exposure staggered time of each row.

[0004] Figure 2 It is a schematic diagram of the stretching of the image of a horizontally moving object by a camera and a laser radar in the prior art. Figure 2 (a) is a horizontally moving object. The shape of the object is a square, and the solid arrow above it is used to indicate the direction of the object's movement.

[0005] Figure 2 (b) Figure 2 (a) is a schematic diagram of the camera output results when the square object moves horizontally at high speed. The solid arrows above it are used to indicate the single receiving direction of the camera, and the dashed arrows are used to indicate the scanning order between adjacent solid arrows. The camera scans multiple times in the scanning order shown by the arrows to obtain Figure 2 (b) is the image shown. When a properly installed camera shoots a high-speed horizontally moving object, the shape of the object in the camera output result is a parallelogram. Figure 2 (a) will produce oblique stretching, and the stretching amplitude is related to the relative speed and distance of the object. The camera itself cannot measure the relative speed and distance of the object and must rely on other sensors.

[0006] Figure 2 (c) Figure 2(a) is a schematic diagram of the laser radar output results when the square object shown in (a) moves horizontally at high speed. The solid arrow on the left is used to indicate the single receiving direction of the laser radar, and the dotted arrow is used to indicate the scanning order between adjacent solid arrows. The laser radar scans multiple times in the scanning order shown by the arrows to obtain Figure 2 (c) is the image shown. Figure 2 (a) and Figure 2 As shown in (c), the laser radar generally scans 360°. As a result, when the laser radar obtains the point cloud of a high-speed horizontally moving object, the shape of the object in the laser radar output result is a rectangle. The output result is different from Figure 2 (a) Compared with the above, lateral stretching will occur, and the stretching amplitude is related to the relative speed of the object and the distance.

[0007] The images of horizontally moving objects taken by the camera and the lidar have inconsistent object deformation. The shape of the object needs to be restored separately before information fusion can be performed. The restoration requires accurate object speed and distance information as a guide, which requires a lot of calculations and the results are uncontrollable.

[0008] Therefore, it is necessary to provide a technical solution to reduce the amount and difficulty of deformation restoration calculations for cameras and lidars, and to realize the synchronization of pixel points measured by the camera and point cloud acquisition by the lidar. Summary of the invention

[0009] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the first aspect of the present invention proposes a data synchronization acquisition method, comprising:

[0010] Get the rotation direction of the laser radar;

[0011] Generate an adjustment signal for controlling the rotatable rolling shutter camera to rotate to a preset state according to the rotation direction; wherein, in the preset state, the stretching direction of the output result of the rotatable rolling shutter camera on the moving object is consistent with the stretching direction of the output result of the laser radar on the moving object;

[0012] Get the rotation speed of the laser radar and the shooting parameters of the rotatable rolling shutter camera;

[0013] According to the rotation speed and the shooting parameters, setting the shooting frame rate and shooting time of the rotatable rolling shutter camera;

[0014] A shooting signal for controlling the shooting of the rotatable rolling camera is generated according to the shooting frame rate and the shooting time; wherein the rotatable rolling camera can respond to the shooting signal to shoot according to the shooting frame rate and the shooting time, and obtain a target shooting result; the target shooting result has a consistent object deformation with the scanning result of the laser radar.

[0015] Further, generating an adjustment signal for controlling the rotatable rolling shutter camera to rotate to a preset state according to the rotation direction includes:

[0016] When the laser radar rotates clockwise, a first adjustment signal is generated for controlling the rotatable rolling shutter camera to rotate to a first preset state; wherein the first preset state can be obtained by rotating the rotatable rolling shutter camera in the initial state to the left by 90°; the output result of the rotatable rolling shutter camera in the initial state is consistent with the display direction of the output result of the laser radar;

[0017] When the laser radar rotates counterclockwise, a second adjustment signal is generated to control the rotatable rolling shutter camera to rotate to a second preset state; the second preset state can be obtained by rotating the rotatable rolling shutter camera in the initial state to the right by 90°.

[0018] Further, the shooting parameters include an effective field angle, a total number of rows, and an effective number of rows; and setting a shooting frame rate and a shooting time of the rotatable rolling shutter camera according to the rotation speed and the shooting parameters includes:

[0019] Calculate the time it takes for the laser radar to sweep the effective field of view of the camera according to the effective field of view angle and the rotation speed;

[0020] Calculate the camera frame rate according to the total number of rows, the valid rows, and the time for the laser radar to sweep the effective field of view of the camera;

[0021] Obtaining the exposure time of the next frame output by the automatic exposure algorithm based on the camera frame rate;

[0022] Based on the exposure time of all the rows and the next frame, the shooting time that can make the laser radar scanning line coincide with the midpoint of the exposure time of the first row of the camera is calculated.

[0023] Furthermore, based on the exposure time of all the rows and the next frame, the shooting time that can make the laser radar scan line coincide with the midpoint of the exposure time of the first row of the camera is calculated, and the shooting time is calculated using the following formula:

[0024] t shot =-((V total -VFP)t H -t exp / 2)

[0025] Among them, t shot For shooting time;

[0026] V total for all rows of the camera;

[0027] VFP is the field synchronization front shoulder;

[0028] tH Output time for a single line;

[0029] t exp Exposure time for the next frame.

[0030] A second aspect of the present invention provides a data synchronization acquisition control module, comprising:

[0031] A first acquisition module is used to acquire the rotation direction of the laser radar;

[0032] A camera adjustment module, used to generate an adjustment signal for controlling the rotatable rolling shutter camera to rotate to a preset state according to the rotation direction; wherein in the preset state, the stretching direction of the output result of the rotatable rolling shutter camera on the moving object is consistent with the stretching direction of the output result of the laser radar on the moving object;

[0033] The second acquisition module is used to obtain the rotation speed of the laser radar and the shooting parameters of the rotatable rolling shutter camera;

[0034] A shooting parameter setting module, used to set the shooting frame rate and shooting time of the rotatable rolling shutter camera according to the rotation speed and the shooting parameters;

[0035] A shooting control module is used to generate a shooting signal for controlling the shooting of the rotatable rolling shutter camera according to the shooting frame rate and the shooting time; wherein the rotatable rolling shutter camera can respond to the shooting signal to shoot according to the shooting frame rate and the shooting time, and obtain a target shooting result; the target shooting result has a consistent object deformation with the scanning result of the laser radar.

[0036] Furthermore, the camera adjustment module includes:

[0037] A first adjustment module is used to generate a first adjustment signal for controlling the rotatable rolling shutter camera to rotate to a first preset state when the laser radar rotates clockwise; wherein the first preset state can be obtained by rotating the rotatable rolling shutter camera in the initial state to the left by 90°; and the output result of the rotatable rolling shutter camera in the initial state is consistent with the display direction of the output result of the laser radar;

[0038] The second adjustment module is used to generate a second adjustment signal for controlling the rotatable rolling shutter camera to rotate to a second preset state when the laser radar rotates counterclockwise; the second preset state can be obtained by rotating the rotatable rolling shutter camera in the initial state to the right by 90°.

[0039] Further, the shooting parameters include effective field angle, all rows and effective rows; the shooting parameter setting module includes:

[0040] An effective field of view angle calculation module, used to calculate the time it takes for the laser radar to sweep the effective field of view angle of the camera according to the effective field of view angle and the rotation speed;

[0041] A camera frame rate calculation module, used to calculate the camera frame rate according to the total number of rows, the valid rows and the time for the laser radar to scan the effective field of view of the camera;

[0042] A next frame exposure time acquisition module, used to obtain the next frame exposure time output by the automatic exposure algorithm based on the camera frame rate;

[0043] The shooting time calculation module is used to calculate the shooting time that can make the laser radar scanning line coincide with the midpoint of the exposure time of the first row of the camera based on all the rows and the exposure time of the next frame.

[0044] Furthermore, the shooting time calculation module uses the following formula to calculate the shooting time:

[0045] t shot =-((V total -VFP)t H -t exp / 2)

[0046] Among them, t shot For shooting time;

[0047] V total for all rows of the camera;

[0048] VFP is the field synchronization front shoulder;

[0049] t H Output time for a single line;

[0050] t exp Exposure time for the next frame.

[0051] A third aspect of the present invention provides a data synchronization acquisition system, comprising a laser radar, a rotatable rolling shutter camera, and the data synchronization acquisition control module described in the second aspect of the present invention;

[0052] The number of the rotatable rolling shutter cameras is one or more, and the multiple rotatable rolling shutter cameras are distributed along the circumference of the laser radar. The laser radar is connected to the data synchronization acquisition control module, and the rotatable rolling shutter camera is connected to the data synchronization acquisition control module.

[0053] The fourth aspect of the present invention proposes an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the data synchronization acquisition method proposed in the first aspect of the present invention.

[0054] The fifth aspect of the present invention proposes a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the data synchronization acquisition method proposed in the first aspect of the present invention.

[0055] The data synchronization acquisition method, control module, system and storage medium provided by the present invention apply a 90° rotated camera installation method, so that the stretching direction of the camera shooting result relative to the moving object is consistent with the stretching direction of the laser radar scanning result relative to the moving object; by setting the camera parameters, the camera and the laser radar have exactly the same frame within the same field of view, and the camera and laser radar pixels and point clouds can match; by setting the shooting timing, the rolling camera and the laser radar reception can achieve high-precision time synchronization close to the pixel level; based on the comprehensive application of the above three improvements, the motion characteristics, object position and time information of the camera and the laser radar are unified, and the function of synchronizing the pixel points measured by the camera with the laser radar point cloud acquisition is realized, which can greatly reduce the difficulty and calculation amount of the camera and laser radar information fusion and improve the reliability of the results.

[0056] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 It is a schematic diagram of rolling shutter exposure time in the prior art;

[0059] Figure 2 It is a schematic diagram of stretching the image of a horizontally moving object by a camera and a laser radar in the prior art;

[0060] Figure 3 is a flow chart of a data synchronization acquisition method provided by an embodiment of the present invention;

[0061] Figure 4 is a schematic diagram of camera rotation provided by an embodiment of the present invention;

[0062] Figure 5 is a flowchart of step S104 provided in an embodiment of the present invention;

[0063] Figure 6 It is a structural block diagram of a data synchronization acquisition control module provided by an embodiment of the present invention;

[0064] Figure 7 It is a schematic diagram of the installation of a data synchronization acquisition system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Examples of the embodiments are shown in the drawings, where the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions.

[0066] Example

[0067] In order to reduce the amount of calculation and difficulty of deformation restoration between cameras and lidars, and to realize the function of synchronizing the pixel points measured by the camera with the point cloud acquisition of the lidar, an embodiment of the present invention provides a data synchronization acquisition method, which can be used to reduce the amount of subsequent information fusion calculations and improve the reliability of the results.

[0068] Figure 3 It is a flow chart of a data synchronization collection method provided by an embodiment of the present invention. This specification provides method operation steps as described in the embodiment or flow chart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment) according to the method shown in the embodiment or the accompanying drawings. Specifically, Figure 3 As shown, the method may include the following steps:

[0069] S101: Obtain the rotation direction of the laser radar;

[0070] Specifically, the rotation direction is an inherent property of the mechanical laser radar, and the rotation direction of the laser radar can be clockwise or counterclockwise.

[0071] S102: generating an adjustment signal for controlling the rotatable rolling shutter camera to rotate to a preset state according to the rotation direction of the laser radar;

[0072] Among them, the stretching direction of the moving object by the output result of the rotatable rolling shutter camera in the preset state is consistent with the stretching direction of the moving object by the output result of the laser radar.

[0073] It should be noted that the rotation direction of the laser radar is only used to determine the adjustment method (including rotation direction and rotation angle) of rotating the rolling camera from the initial state to the preset state. The camera does not rotate during operation and is not affected by the rotation direction of the laser radar.

[0074] In some embodiments, generating an adjustment signal for controlling the rotatable rolling shutter camera to rotate to a preset state according to the rotation direction of the laser radar includes:

[0075] When the laser radar rotates clockwise, a first adjustment signal is generated for controlling the rotatable rolling shutter camera to rotate to a first preset state; the first preset state can be obtained by rotating the rotatable rolling shutter camera in the initial state to the left by 90°; the output result of the rotatable rolling shutter camera in the initial state is consistent with the display direction of the output result of the laser radar; Figure 4 is a schematic diagram of camera rotation provided by an embodiment of the present invention, wherein: Figure 4 (a) shows the rotatable rolling shutter camera in the initial state. Figure 4 (b) shows the rotatable rolling shutter camera after rotation. Figure 4 (a) with Figure 4 The arc with arrows between (b) is used to indicate the rotation direction of the rotatable rolling shutter camera.

[0076] When the laser radar rotates counterclockwise, a second adjustment signal is generated for controlling the rotatable rolling shutter camera to rotate to a second preset state; the second preset state can be obtained by rotating the rotatable rolling shutter camera in the initial state to the right by 90°;

[0077] When the laser radar rotates counterclockwise, a second rotation signal is generated; the second rotation signal is used to control the rotatable rolling shutter camera to rotate 90° to the right from the initial state;

[0078] Among them, the camera image output by the rotatable rolling shutter camera in the initial state is consistent with the display direction of the scanning result of the laser radar.

[0079] S103: Obtain the rotation speed m of the laser radar and the shooting parameters of the rotatable rolling shutter camera; wherein the shooting parameters include the effective field of view angle p, the total line V total and effective lines;

[0080] Among them, all rows V total It refers to the corresponding line between two field synchronization pulses, and the effective line V refers to the corresponding line of the display timing segment.

[0081] Specifically, the unit of the effective field of view angle p is "° (degrees)", and the size of the effective field of view angle p is usually 90°, 120°. According to actual needs, the value of the effective field of view angle p can also be set to other values, for example, 70°, 110°, etc., but this embodiment is not limited to this.

[0082] Specifically, the unit of the laser radar rotation speed is "r / s (circles per second)", and the laser radar rotation speed m is usually 10r / s. According to actual needs, the value of the effective field of view angle p can also be other values, for example, 70°, 110°, etc., but this embodiment is not limited to this.

[0083] S104: Setting the shooting frame rate Fs and shooting time t of the rotatable rolling shutter camera according to the rotation speed m and shooting parameters shot ;

[0084] By modifying the camera parameter settings, the camera and lidar can have exactly the same frame within the same field of view, and the camera and lidar pixels and point clouds can match. Figure 5 is a flowchart of step S104 provided in an embodiment of the present invention, and the specific example is as follows Figure 5 As shown, in some embodiments, step S104 includes the following steps:

[0085] S1041: Calculate the time t for the laser radar to sweep the camera's effective field of view angle according to the effective field of view angle p and the rotation speed m sweep ;

[0086] Specifically, the time it takes for the laser radar to scan the camera's effective field of view is calculated as t sweep =p / (360m)

[0087] S1042: According to all rows V total , effective line V and the time t that the laser radar sweeps across the camera’s effective field of view sweep , calculate the camera frame rate Fs;

[0088] Specifically, the camera frame rate is calculated as Fs = V / (V total / t sweep ).

[0089] S1043: Obtain the next frame exposure time t output by the automatic exposure algorithm based on the camera frame rate Fs exp ;

[0090] S1044: Based on all rows V total , next frame exposure time t exp , calculate the shooting time t that can make the laser radar scan line coincide with the midpoint of the camera's first line exposure time shot .

[0091] In some embodiments, the shooting time t is calculated using the following formula: shot :

[0092] t shot =-((V total -VFP)t H -t exp / 2)

[0093] Among them, t shot For shooting time;

[0094] V total For all rows of the camera;

[0095] VFP is the field synchronization front shoulder;

[0096] t H Output time for a single line;

[0097] t exp Exposure time for the next frame.

[0098] The negative sign in the shooting time calculation formula means that the calculated shooting time needs to be advanced.

[0099] By calculating the set shooting timing according to the above method, the rolling shutter camera and the lidar receiver can achieve high-precision time synchronization close to the pixel level.

[0100] The data synchronization acquisition method provided in the embodiment of the present invention unifies the motion characteristics, object position and time information of the rotatable camera and the laser radar, which can greatly reduce the calculation amount and restoration difficulty of the deformation restoration of the camera and the laser radar, and realizes the function of synchronizing the pixel points measured by the camera with the laser radar point cloud acquisition, which is conducive to reducing the subsequent information fusion calculation amount and improving the reliability of the results.

[0101] S105: generating a shooting signal for controlling the rotatable rolling shutter camera to shoot according to the shooting frame rate and the shooting time;

[0102] Among them, the rotatable rolling shutter camera can respond to the shooting signal to shoot according to the shooting frame rate and shooting time, and obtain the target shooting result; the target shooting result has the same object deformation as the scanning result of the laser radar.

[0103] Figure 6 is a structural block diagram of a data synchronization acquisition control module provided by an embodiment of the present invention, specifically as follows Figure 6 As shown, a data synchronization acquisition control module provided by an embodiment of the present invention includes the following modules:

[0104] A first acquisition module 201 is used to acquire the rotation direction of the laser radar;

[0105] The camera adjustment module 202 is used to generate an adjustment signal for controlling the rotatable rolling shutter camera to rotate to a preset state according to the rotation direction of the laser radar;

[0106] Among them, the stretching direction of the moving object by the output result of the rotatable rolling shutter camera in the preset state is consistent with the stretching direction of the moving object by the output result of the laser radar.

[0107] The second acquisition module 203 is used to acquire the rotation speed of the laser radar and the shooting parameters of the rotatable rolling shutter camera;

[0108] The shooting parameter setting module 204 is used to set the shooting frame rate and shooting time of the rotatable rolling shutter camera according to the rotation speed and shooting parameters;

[0109] A shooting control module 205, used to generate a shooting signal for controlling the rotatable rolling shutter camera to shoot according to the shooting frame rate and the shooting time;

[0110] Among them, the rotatable rolling shutter camera can respond to the shooting signal to shoot according to the shooting frame rate and shooting time, and obtain the target shooting result; the target shooting result has the same object deformation as the scanning result of the laser radar.

[0111] In some embodiments, the camera adjustment module 202 includes the following modules:

[0112] A first adjustment module is used to generate a first adjustment signal for controlling the rotatable rolling shutter camera to rotate to a first preset state when the laser radar rotates clockwise; wherein the first preset state can be obtained by rotating the rotatable rolling shutter camera in an initial state to the left by 90°; and the output result of the rotatable rolling shutter camera in the initial state is consistent with the display direction of the output result of the laser radar;

[0113] The second adjustment module is used to generate a second adjustment signal for controlling the rotatable rolling shutter camera to rotate to a second preset state when the laser radar rotates counterclockwise; the second preset state can be obtained by rotating the rotatable rolling shutter camera in the initial state 90° to the right.

[0114] In some embodiments, the shooting parameters include effective field angle, total row V totaland effective line V; the shooting parameter setting module 204 includes:

[0115] The effective field of view angle calculation module is used to calculate the time t that the laser radar sweeps the effective field of view angle of the camera based on the effective field of view angle p and the rotation speed m sweep =p / (360m);

[0116] The camera frame rate calculation module is used to calculate the frame rate of all rows according to the V total , effective line V and the time t that the laser radar sweeps across the camera’s effective field of view sweep , calculate the camera frame rate Fs = V / (V total / t sweep );

[0117] The next frame exposure time acquisition module is used to obtain the next frame exposure time t output by the automatic exposure algorithm based on the camera frame rate Fs exp ;

[0118] Shooting time calculation module, used to calculate the shooting time based on all rows V total , next frame exposure time t exp , calculate the shooting time t that can make the laser radar scan line coincide with the midpoint of the camera's first line exposure time shot The shooting time calculation module uses the following formula to calculate the shooting time t shot :

[0119] t shot =-((V total -VFP)t H -t exp / 2)

[0120] Among them, t shot For shooting time;

[0121] V total For all rows of the camera;

[0122] VFP is the field synchronization front shoulder;

[0123] t H Output time for a single line;

[0124] t exp Exposure time for the next frame.

[0125] Figure 7 is a schematic diagram of an installation of a data synchronization acquisition system provided by an embodiment of the present invention, specifically as follows Figure 7 As shown, the data synchronization acquisition system includes a laser radar, a rotatable rolling shutter camera and a data synchronization acquisition control module disclosed in the above embodiment;

[0126] In some embodiments, the number of rotatable rolling shutter cameras is one, and the rotatable rolling shutter camera is arranged along the circumference of the laser radar. The laser radar and the data synchronization acquisition device are synchronously connected through the PPS synchronization signal, and the rotatable rolling shutter camera is connected to the data synchronization acquisition control module.

[0127] In some embodiments, there are multiple rotatable rolling shutter cameras, which are distributed along the circumference of the laser radar. The laser radar is synchronously connected to the data synchronization acquisition device through the PPS synchronization signal, and the rotatable rolling shutter camera is connected to the data synchronization acquisition control module. The data synchronization acquisition system can be used for the combination of cameras and laser radars with different resolutions and different field of view angles. For example, Figure 7 The data synchronization acquisition system shown has three rotatable rolling shutter cameras with different effective field of view angles, camera 1, camera 2 and camera 3. The effective field of view angle of camera 1 is p1°, the effective field of view angle of camera 2 is p2°, and the effective field of view angle of camera 3 is p3°, p1>p2>p3.

[0128] Specifically, the data synchronization acquisition control module can be an FPGA chip or an MCU chip.

[0129] Specifically, the rotatable rolling shutter camera includes a rotating mechanism and a rolling shutter camera, and the rolling shutter camera is rotatably connected to the rotating mechanism.

[0130] Theoretically, it is also feasible to manually operate the rotating mechanism to rotate the rolling shutter camera to a preset state. Of course, it is also feasible not to set the rotating mechanism when manually adjusting the camera from the initial state to the preset state. However, the above scheme is not convenient enough to operate, and the installation accuracy is difficult to guarantee. For the sake of convenience and accuracy, the automatic control adjustment method is preferred in actual applications, that is, the adjustment method of controlling the rotating mechanism to automatically rotate to the preset state through the data synchronization acquisition control module.

[0131] When the automatic control adjustment method is adopted, the rotating mechanism can be electrically connected to the data synchronization acquisition control module, and the rotating mechanism has the function of driving the rolling shutter camera to rotate in a predetermined direction and at a predetermined angle according to an adjustment signal.

[0132] It should be noted that the rotating mechanism can be implemented in various forms, which are not limited in this embodiment.

[0133] An embodiment of the present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement a data synchronization acquisition method as in the method embodiment.

[0134] An embodiment of the present invention also provides a storage medium, which can be set in a server to store at least one instruction, at least one program, code set or instruction set related to the data synchronization collection method in the method embodiment. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the data synchronization collection method provided by the above-mentioned method embodiment.

[0135] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0136] It can be seen from the embodiments of the data synchronization acquisition method, device, system and storage medium provided by the present invention that a 90° rotated camera installation method is applied in the embodiments of the present invention, so that the stretching direction of the camera shooting result relative to the moving object is consistent with the stretching direction of the laser radar scanning result relative to the moving object; by setting the camera parameters, the camera and the laser radar have exactly the same frame within the same field of view, and the camera and laser radar pixels and point clouds can match; by setting the shooting timing, the rolling camera and the laser radar reception can achieve high-precision time synchronization close to the pixel level; based on the comprehensive application of the above three improvements, the motion characteristics, object position and time information of the camera and the laser radar are unified, and the function of synchronizing the pixel points measured by the camera with the laser radar point cloud acquisition is realized, which can greatly reduce the difficulty and calculation amount of the camera and laser radar information fusion and improve the reliability of the results.

[0137] It should be noted that the sequence of the embodiments of the present invention described above is for description only and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0138] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and server embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0139] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0140] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data synchronization acquisition method, characterized in that: include: Get the rotation direction of the laser radar; Generate an adjustment signal for controlling the rotatable rolling shutter camera to rotate to a preset state according to the rotation direction; wherein, in the preset state, the stretching direction of the output result of the rotatable rolling shutter camera on the moving object is consistent with the stretching direction of the output result of the laser radar on the moving object; Get the rotation speed of the laser radar and the shooting parameters of the rotatable rolling shutter camera; According to the rotation speed and the shooting parameters, setting the shooting frame rate and shooting time of the rotatable rolling shutter camera; A shooting signal for controlling the rotatable rolling shutter camera to shoot is generated according to the shooting frame rate and the shooting time; wherein the rotatable rolling shutter camera can shoot according to the shooting frame rate and the shooting time in response to the shooting signal, and obtain a target shooting result; the target shooting result has a consistent object deformation with the scanning result of the laser radar; The shooting parameters include an effective field of view angle, all rows and effective rows; the shooting frame rate and shooting time of the rotatable rolling shutter camera are set according to the rotation speed and the shooting parameters, including: calculating the time for the laser radar to sweep the effective field of view angle of the camera according to the effective field of view angle and the rotation speed; calculating the camera frame rate according to all rows, the effective rows and the time for the laser radar to sweep the effective field of view angle of the camera; obtaining the exposure time of the next frame output by the automatic exposure algorithm based on the camera frame rate; and using the following formula, based on all rows and the exposure time of the next frame, calculating the shooting time that can make the laser radar scanning line coincide with the midpoint of the exposure time of the first row of the camera: t shot =-((V total -VFP)t H -t exp / 2); where t shot is the shooting time, V total is the total number of rows of the camera, VFP is the field synchronization front shoulder, t H is the single line output time, t exp Exposure time for the next frame.

2. The data synchronization acquisition method according to claim 1, characterized in that: The step of generating an adjustment signal for controlling the rotatable rolling shutter camera to rotate to a preset state according to the rotation direction includes: When the laser radar rotates clockwise, a first adjustment signal is generated for controlling the rotatable rolling shutter camera to rotate to a first preset state; wherein the first preset state can be obtained by rotating the rotatable rolling shutter camera in the initial state to the left by 90°; the output result of the rotatable rolling shutter camera in the initial state is consistent with the display direction of the output result of the laser radar; When the laser radar rotates counterclockwise, a second adjustment signal is generated to control the rotatable rolling shutter camera to rotate to a second preset state; the second preset state can be obtained by rotating the rotatable rolling shutter camera in the initial state to the right by 90°.

3. A data synchronization acquisition control module, characterized in that: include: A first acquisition module is used to acquire the rotation direction of the laser radar; A camera adjustment module, used to generate an adjustment signal for controlling the rotatable rolling shutter camera to rotate to a preset state according to the rotation direction; wherein in the preset state, the stretching direction of the output result of the rotatable rolling shutter camera on the moving object is consistent with the stretching direction of the output result of the laser radar on the moving object; The second acquisition module is used to obtain the rotation speed of the laser radar and the shooting parameters of the rotatable rolling shutter camera; A shooting parameter setting module, used to set the shooting frame rate and shooting time of the rotatable rolling shutter camera according to the rotation speed and the shooting parameters; A shooting control module, used for generating a shooting signal for controlling the rotatable rolling shutter camera to shoot according to the shooting frame rate and the shooting time; wherein the rotatable rolling shutter camera can shoot according to the shooting frame rate and the shooting time in response to the shooting signal, and obtain a target shooting result; the target shooting result has a consistent object deformation with the scanning result of the laser radar; The shooting parameters include an effective field of view angle, all rows and effective rows; the shooting frame rate and shooting time of the rotatable rolling shutter camera are set according to the rotation speed and the shooting parameters, including: calculating the time for the laser radar to sweep the effective field of view angle of the camera according to the effective field of view angle and the rotation speed; calculating the camera frame rate according to all rows, the effective rows and the time for the laser radar to sweep the effective field of view angle of the camera; obtaining the exposure time of the next frame output by the automatic exposure algorithm based on the camera frame rate; and using the following formula, based on all rows and the exposure time of the next frame, calculating the shooting time that can make the laser radar scanning line coincide with the midpoint of the exposure time of the first row of the camera: t shot =-((V total -VFP)t H -t exp / 2); where t shot is the shooting time, V total is the total number of rows of the camera, VFP is the field synchronization front shoulder, t H is the single line output time, t exp Exposure time for the next frame.

4. The data synchronization acquisition control module according to claim 3, characterized in that: The camera adjustment module comprises: A first adjustment module is used to generate a first adjustment signal for controlling the rotatable rolling shutter camera to rotate to a first preset state when the laser radar rotates clockwise; wherein the first preset state can be obtained by rotating the rotatable rolling shutter camera in the initial state to the left by 90°; and the output result of the rotatable rolling shutter camera in the initial state is consistent with the display direction of the output result of the laser radar; The second adjustment module is used to generate a second adjustment signal for controlling the rotatable rolling shutter camera to rotate to a second preset state when the laser radar rotates counterclockwise; the second preset state can be obtained by rotating the rotatable rolling shutter camera in the initial state to the right by 90°.

5. A data synchronization acquisition system, characterized in that: It includes a laser radar, a rotatable rolling shutter camera and a data synchronization acquisition control module as described in any one of claims 3-4; The number of the rotatable rolling shutter cameras is one or more, and the multiple rotatable rolling shutter cameras are distributed along the circumference of the laser radar. The laser radar is connected to the data synchronization acquisition control module, and the rotatable rolling shutter camera is connected to the data synchronization acquisition control module.

6. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the data synchronization acquisition method as described in any one of claims 1-2.

Images