Monitoring method of laser radar system and laser radar system

Abstract

The invention relates to a monitoring method for a laser radar system, and the method is characterized in that the system comprises a laser radar, a scanner, a first reflector and a camera, and the method comprises the steps: obtaining reflection echoes, obtaining a first target object detected by the laser radar based on the reflection echoes, the reflection echo comprises the echo of the first target object, which is received after the detection signal transmitted by the laser radar is deflected by the scanner to the first target object; image information collected by the camera is obtained, a second target object recognized by the camera is obtained based on the image information, and the image information comprises image information obtained after reflected light of the second target object is deflected by the scanner and then enters the camera through the first reflector; and under the condition that the first target object and the second target object are not the same target object, the deflection angle of the scanner is adjusted based on reflection echoes and image information, so that the laser radar and the camera monitor the same target at the same time.

Description

Technical Field

[0001] This application relates to the field of wind-measuring lidar, and in particular to a monitoring method for a lidar system and a lidar system. Background Technology

[0002] Doppler wind lidar is a non-contact telemetry lidar system based on the coherent Doppler velocities principle. Figure 1 This is a schematic diagram of a Doppler wind-measuring lidar in one embodiment. It is additionally equipped with a scanner whose beam direction can be freely adjusted, allowing for specific scanning of a particular airspace. A three-dimensional scanning wind-measuring lidar is a lidar system based on the above principles. Figure 2 This is a schematic diagram of a 3D scanning optical antenna in one embodiment. With the continuous expansion and diversification of applications for 3D scanning wind-measuring lidar, the demand for real-time visual monitoring of the measured target is becoming increasingly prominent. The integration of visualization functions helps operators intuitively grasp the actual situation of the target area, further improving the reliability of equipment operation and the accuracy of data interpretation.

[0003] Existing visualization solutions can only monitor the working environment around the lidar, and cannot keep up with the radar's detection direction. Operators need to first indirectly determine whether the target object's trajectory has been successfully tracked based on the environmental information collected by the equipment, and then obtain specific target data through complex calculations. The indirect judgment method is not only cumbersome, but may also affect the monitoring accuracy and efficiency due to data interpretation deviations, making it difficult to meet the precise target visualization and tracking requirements in practical applications. Summary of the Invention

[0004] Therefore, it is necessary to provide a monitoring method for a lidar system to address the aforementioned technical problems.

[0005] Firstly, this application provides a monitoring method for a lidar system. The method includes:

[0006] The reflected echo is obtained, and the first target object detected by the lidar is obtained based on the reflected echo. The reflected echo includes the echo of the first target object received after the detection signal emitted by the lidar is deflected by the scanner and directed toward the first target object.

[0007] The image information captured by the camera is obtained, and the second target object identified by the camera is obtained based on the image information. The image information includes the image information of the reflected light from the second target object being deflected by the scanner and then incident on the camera through the first reflecting mirror.

[0008] If the first target object and the second target object are not the same target object, the deflection angle of the scanner is adjusted based on the reflected echo and the image information so that the lidar and the camera can monitor the same target simultaneously.

[0009] In one embodiment, the method for adjusting the camera includes:

[0010] The camera receives a first control command and adjusts the distance between the lens and the photosensitive element to a preset angle based on the first control command. The camera includes a lens and a photosensitive element.

[0011] The angle of the scanner is adjusted so that the image of the star captured by the camera through the scanner is clear, thereby completing the camera adjustment.

[0012] In one embodiment, the scanner adjustment method includes:

[0013] The scanner receives a second control command and adjusts the second and third reflectors based on the second control command. The scanner includes the second and third reflectors.

[0014] The scanner is adjusted by acquiring an image captured by a camera that includes a window mirror, wherein the window mirror occupies a preset proportion of the image area and is located in the center of the image.

[0015] In one embodiment, determining that the first target object and the second target object are not the same target object includes:

[0016] The image is corrected based on a preset color correction matrix. The construction process of the preset color correction matrix includes: acquiring a color image under stable lighting, obtaining a set of color correction matrices through an iterative method, and obtaining the first-ranked color correction matrix by calculating the average variance of each color correction matrix in the set of color correction matrices, which is used as the preset color correction matrix.

[0017] The judgment is based on the first and second target objects after color correction.

[0018] In one embodiment, the transmission path of the laser beam is obtained, and the positions of the first reflector and the camera are determined based on the transmission path of the laser beam, wherein the positions of the first reflector and the camera do not overlap with the transmission path of the laser beam.

[0019] In one embodiment, the scanner adjustment method further includes:

[0020] Adjust the positions of the second and third reflectors so that, when the second and third reflectors are at different angles, the entire imaging optical path of the camera passes through the second and third reflectors.

[0021] Secondly, this application also provides a lidar system, the system comprising:

[0022] The scanner is a hollow two-axis mechanical rotating structure used to rotate along the longitudinal and transverse axes to change the elevation and azimuth angles of the laser beam emitted by the lidar.

[0023] A first reflecting mirror is used to help the camera capture the target object;

[0024] A camera, wherein the camera is positioned on the outgoing light path of the first reflecting mirror;

[0025] A lidar, wherein the laser beam emitted by the lidar is emitted through the hollow structure of the scanner;

[0026] A telescope located in the optical path of the laser beam.

[0027] In one embodiment, the scanner further includes a second reflector and a third reflector;

[0028] The second reflector is located on the initial incident path of the light reflected from the target, and the third reflector is disposed on the outgoing light path of the second reflector to reflect the light back to the first reflector.

[0029] In one embodiment, the first reflector and the camera are mounted in a separate window on the side wall of the scanner.

[0030] In one embodiment, the first reflector includes a convex reflector.

[0031] The monitoring method of the above-mentioned lidar system has at least the following beneficial effects:

[0032] The embodiments provided in this disclosure can dynamically adjust the scanner deflection angle through dual feedback of reflected echo and image information, ensuring that the imaging system and the lidar are always aligned with the same target and maintain tracking. Furthermore, the camera can intuitively determine whether the target object has been tracked.

[0033] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments or conventional technologies of this disclosure, the accompanying drawings used in the description of the embodiments or conventional technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of a Doppler wind-measuring lidar in one embodiment;

[0036] Figure 2 This is a schematic diagram of a three-dimensional scanning optical antenna in one embodiment;

[0037] Figure 3 This is a schematic diagram of a lidar system in one embodiment;

[0038] Figure 4 This is a flowchart illustrating a monitoring method for a lidar system in one embodiment;

[0039] Figure 5 This is a structural diagram of one embodiment including a follow-up camera;

[0040] Figure 6 This is a schematic diagram illustrating the adjustment of the first reflecting mirror in one embodiment;

[0041] Figure 7 This is a schematic diagram of the camera imaging optical path when the reflector is at different angles in one embodiment;

[0042] Figure 8 This is a schematic diagram of a lidar system in one embodiment;

[0043] Figure 9 This is a schematic diagram of the imaging optical path design for an embodiment that does not obstruct the laser beam of a lidar.

[0044] Figure 10 This is a schematic diagram comparing the imaging light rays using a plane mirror and a convex mirror in one embodiment. Detailed Implementation

[0045] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0046] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims. The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any specific order.

[0047] This disclosure provides a monitoring method for a lidar system, which can be applied to, for example... Figure 3 The application environment shown. The system includes a LiDAR, a scanner, a first reflector, and a camera. The LiDAR can be... Figure 1 The Doppler wind-measuring lidar places a camera and a first reflector at an appropriate angle between the lidar and the scanner. The camera and the first reflector are fixedly installed, while the scanner can perform a deflection function.

[0048] In some embodiments of this disclosure, such as Figure 4 As shown, a monitoring method for a lidar system is provided. In one specific embodiment, the method may include the following steps:

[0049] S402: Obtain the reflected echo, and obtain the first target object detected by the lidar based on the reflected echo. The reflected echo includes the echo of the first target object received after the detection signal emitted by the lidar is deflected by the scanner and directed toward the first target object.

[0050] The laser inside the lidar emits a laser beam, which is the detection signal. The laser beam is emitted through the hollow structure of the scanner. The scanner can be a two-axis mechanical rotating structure. The longitudinal axis adjusts the elevation angle to control the vertical scanning of the laser beam, and the lateral axis adjusts the azimuth angle to control the horizontal scanning of the laser beam. By driving the longitudinal and lateral axes of the scanner to rotate, the elevation and azimuth angles of the detection signal are adjusted. After the detection signal illuminates the first target object, it generates a reflected echo. The detector receives the reflected echo, and the signal processing unit obtains the first target object detected by the lidar.

[0051] S404: Obtain the image information captured by the camera, and obtain the second target object identified by the camera based on the image information. The image information includes the image information of the reflected light from the second target object being deflected by the scanner and then incident on the camera through the first reflector.

[0052] The scanner may include a second reflector and a third reflector. The reflected light from the second target object first enters the second reflector. After being reflected by the second reflector, the light changes its propagation direction and enters the third reflector. The third reflector, as a secondary reflection element, further adjusts the direction of the light path, causing the light to propagate towards the first reflector. When the scanner rotates, the first reflector can be finely adjusted in angle by a motor drive to compensate for the change in the field of view caused by the scanner's rotation. The parallel light reflected by the first reflector enters the camera lens perpendicularly, and the lens converges the parallel light onto the photosensitive element.

[0053] S406: When the first target object and the second target object are not the same target object, adjust the deflection angle of the scanner based on the reflected echo and the image information so that the lidar and the camera can monitor the same target simultaneously.

[0054] If the first target object and the second target object are not the same target object, the pixel deviation of the positions of the first target object and the second target object can be calculated and converted into the angle deviation that the scanner needs to adjust so that the LiDAR and the camera are aligned with the same target.

[0055] In the monitoring method of the above-mentioned lidar system, the scanner deflection angle can be dynamically adjusted through the dual feedback of reflected echo and image information, which can ensure that the imaging system and lidar are always aligned with the same target and maintain tracking. Moreover, the camera can be used to intuitively determine whether the target object has been tracked.

[0056] Figure 5 This is a structural diagram of one embodiment including a follow-up camera. A first motor can be used to control the camera, and a second motor can be used to adjust the angle of a first reflector. An industrial control computer is connected to the first motor and the second motor, and sends commands to control the operation of the first motor and the second motor.

[0057] In some embodiments of this disclosure, the method for adjusting the camera includes:

[0058] The camera receives a first control command and adjusts the distance between the lens and the photosensitive element to a preset angle based on the first control command. The camera includes a lens and a photosensitive element.

[0059] The angle of the scanner is adjusted so that the image of the star captured by the camera through the scanner is clear, thereby completing the camera adjustment.

[0060] Stars are extremely far from Earth, and the light they emit can be considered parallel light when it reaches Earth. When a camera captures a clear image of a star, the distance between the lens and the image sensor is exactly at infinity for focusing, ensuring the clarity of the image of distant targets.

[0061] The industrial control computer sends a first control command, containing preset distance parameters between the lens and the image sensor. A first motor drives the lens to move along the optical axis, precisely adjusting the distance between the lens and the image sensor to the preset value. The lens's focus is set to infinity, ensuring that parallel-incident starlight converges onto the image sensor, maintaining image clarity during long-term operation. The industrial control computer will periodically and automatically refocus according to the set parameters, guaranteeing long-distance detection capabilities over extended periods.

[0062] In some embodiments of this disclosure, the method for adjusting the scanner includes:

[0063] The scanner receives a second control command and adjusts the second and third reflectors based on the second control command. The scanner includes the second and third reflectors.

[0064] The scanner is adjusted by acquiring an image captured by a camera that includes a window mirror, wherein the window mirror occupies a preset proportion of the image area and is located in the center of the image.

[0065] The camera captures images including window mirrors. Figure 6 This is a schematic diagram of the adjustment of the first reflector in one embodiment. The window mirror can be a circular light-transmitting element. Image 6-1 shows the first reflector at the problematic angle. Based on the deviation calculation results, the industrial control computer identifies the proportion of the captured window mirror in the entire image and its center position. It then sends an adjustment command to the second motor to maximize the proportion of the window mirror in the total area and ensure that the center of the window mirror is located at the center of the image. The adjustment is then complete. Image 6-2 shows the first reflector at the normal angle in one embodiment. The industrial control computer will automatically calibrate the pointing according to the set parameters periodically to prevent long-term vibration and temperature changes from causing the installation position to shift and to prevent misalignment between the pointing and the laser beam.

[0066] In some embodiments of this disclosure, determining that the first target object and the second target object are not the same target object includes:

[0067] The image is corrected based on a preset color correction matrix. The construction process of the preset color correction matrix includes: acquiring a color image under stable lighting, obtaining a set of color correction matrices through an iterative method, and obtaining the first-ranked color correction matrix by calculating the average variance of each color correction matrix in the set of color correction matrices, which is used as the preset color correction matrix.

[0068] The judgment is based on the first and second target objects after color correction.

[0069] Standard color images under stable lighting conditions are captured to obtain ideal RGB values. Then, a set of color correction matrices is obtained through an iterative method. Finally, the optimal color correction matrix is ​​selected by calculating the average variance of each color restoration matrix, and the original image is corrected using this matrix.

[0070] In some embodiments of this disclosure, the transmission path of the laser beam is obtained, and the positions of the first reflector and the camera are determined based on the transmission path of the laser beam, wherein the positions of the first reflector and the camera do not overlap with the transmission path of the laser beam.

[0071] The camera is a device to assist in monitoring atmospheric conditions within the detection range of the lidar. The installation of the camera must not affect the detection performance of the lidar itself; that is, the installation of the camera and the fine-tuning reflector must not coincide with the laser beam.

[0072] In some embodiments of this disclosure, the scanner adjustment method further includes:

[0073] Adjust the positions of the second and third reflectors so that, when the second and third reflectors are at different angles, the entire imaging optical path of the camera passes through the second and third reflectors.

[0074] Figure 7 This is a schematic diagram of the camera imaging optical path when the reflector is at different angles in one embodiment. 7-1 is a schematic diagram of the camera on the short side of the first reflector, and 7-2 is a schematic diagram of the camera on the long side of the first reflector.

[0075] The scanner is a hollow two-axis mechanical rotating mechanism that can rotate along the longitudinal and transverse axes to allow the light beam to pass through and to change the emission elevation and azimuth angles, achieving multiple angle adjustments. This ensures that the two-axis reflectors are positioned at different angles, and the camera's imaging optical path can completely pass through the two reflectors and the window mirror.

[0076] This disclosure also provides a lidar system, the system comprising:

[0077] The scanner is a hollow two-axis mechanical rotating structure used to rotate along the longitudinal and transverse axes to change the elevation and azimuth angles of the laser beam emitted by the lidar.

[0078] A first reflecting mirror is used to help the camera capture the target object;

[0079] A camera, wherein the camera is positioned on the outgoing light path of the first reflecting mirror;

[0080] A lidar, wherein the laser beam emitted by the lidar is emitted through the hollow structure of the scanner;

[0081] A telescope located in the optical path of the laser beam.

[0082] Figure 8 This is a schematic diagram of a lidar system in one embodiment. The scanner includes a second reflector and a third reflector. The image of the target is reflected by the second reflector and then incident on the third reflector; after being reflected by the third reflector, it is incident on the first reflector. The first reflector does not interfere with the laser beam emitted by the laser. By finely adjusting the first reflector, the image of the target surface can be incident on the camera and converged onto the photosensitive element through the camera lens. The circular mirror ensures that no blind spots are generated as the scanner rotates and makes full use of the reflector aperture. The camera can transmit data to the radar industrial control computer for data processing through various communication methods.

[0083] In some embodiments of this disclosure, the first reflector and the camera are mounted in a separate window on the side wall of the scanner.

[0084] Figure 9 This is a schematic diagram of the imaging optical path design in one embodiment that does not obstruct the laser beam of the lidar. The installation of the camera and the first reflector cannot coincide with the laser beam; a design with a window in the side wall can be adopted.

[0085] In some embodiments of this disclosure, the first reflector includes a convex reflector.

[0086] A convex reflector can be used as the first reflector to ensure that the detection range at long distances is consistent with the laser beam. If a simpler planar reflector is used, the monitored range will be limited by the camera's focal length, the camera's photosensitive surface, the scanner's length, and the scanner window diameter, and the field of view will be smaller than that of the laser beam.

[0087] A convex mirror has an outwardly convex arc surface. When light shines on the mirror, it undergoes divergent reflection. Parallel light rays, after being reflected by the convex mirror, have their backward extensions converge at a virtual focal point behind the mirror. This allows the camera to capture incident light rays at a wider angle than a plane mirror. When a camera shoots through a convex mirror, the mirror's divergent effect on light significantly increases the camera's observable range.

[0088] Convex mirrors expand the spatial range that a camera can capture through their optical reflection properties, resulting in a wider field of view and coverage of a larger area.

[0089] Figure 10 This is a schematic diagram comparing the imaging rays using a plane mirror and a convex mirror in one embodiment. 10-1 shows the imaging ray diagram using a plane mirror, and 10-2 shows the imaging ray diagram using a convex mirror.

[0090] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0091] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] The embodiments described above are merely illustrative of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent disclosure. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these all fall within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the appended claims.

Claims

1. A method of monitoring a lidar system, the method comprising: The system comprises a laser radar, a scanner, a first mirror, a camera, and the method comprises: acquiring a reflection echo, obtaining a first target object detected by the laser radar based on the reflection echo, the reflection echo comprising a reflection echo of a detection signal emitted by the laser radar after being deflected by the scanner towards the first target object; acquiring image information collected by the camera, obtaining a second target object recognized by the camera based on the image information, the image information comprising image information of reflected light of the second target object after being deflected by the scanner and being incident on the camera through the first mirror; in a case where the first target object and the second target object are not the same target object, adjusting a deflection angle of the scanner based on the reflection echo and the image information, so that the laser radar and the camera simultaneously monitor the same target.

2. The method of claim 1, wherein, The adjustment method of the camera comprises: receiving a first control instruction, adjusting a distance between a lens and a photosensitive element to a preset angle based on the first control instruction, the camera comprising the lens and the photosensitive element; adjusting an angle of the scanner so that a star image collected by the camera through the scanner is clearly imaged, thereby completing the adjustment of the camera.

3. The method of claim 1, wherein, The adjustment method of the scanner comprises: receiving a second control instruction, adjusting a second mirror and a third mirror based on the second control instruction, the scanner comprising the second mirror and the third mirror; acquiring an image containing a window mirror photographed by the camera, in a case where a proportion of the window mirror in an image area meets a preset proportion and the window mirror is located in a central region of the image, thereby completing the adjustment of the scanner.

4. The method of claim 1, wherein, Judging that the first target object and the second target object are not the same target object comprises: correcting an image based on a preset color correction matrix, a construction process of the preset color correction matrix comprising: acquiring a color image under stable illumination, obtaining a color correction matrix set through an iterative method, obtaining a color correction matrix ranked first through calculation of average variances of each color correction matrix in the color correction matrix set, and taking the color correction matrix as the preset color correction matrix; judging based on the first target object and the second target object after color correction.

5. The method of claim 1, wherein, Acquiring a transmission path of a laser beam, determining positions of the first mirror and the camera based on the transmission path of the laser beam, the positions of the first mirror and the camera not overlapping with the transmission path of the laser beam.

6. The method of claim 3, wherein, The adjustment method of the scanner further comprises: adjusting positions of the second mirror and the third mirror, so that in a case where the second mirror and the third mirror are at different included angles, imaging light paths of the camera all pass through the second mirror and the third mirror.

7. A lidar system, comprising: The system is applied to the monitoring method of any one of claims 1-6, and the system comprises: a scanner, which is a hollow two-axis mechanical rotating structure, used to rotate along a longitudinal axis and a transverse axis to change a pitch angle and an azimuth angle of a laser beam emitted by a laser radar; a first mirror, used to help a camera to photograph a target object; a camera, arranged on an exit light path of the first mirror; A laser radar, a laser beam emitted by the laser radar is emitted through the hollow structure of the scanner; A telescope is located on the light path of the laser beam.

8. The system of claim 7, wherein, The scanner further comprises a second mirror and a third mirror. The second mirror is located on the initial incident path of the reflected light of the target to be measured, and the third mirror is arranged on the light path of the second mirror and used for reflecting the light to the first mirror.

9. The system of claim 8, wherein, The first mirror and the camera are mounted at independent windows formed on the side wall of the scanner.

10. The system of claim 7, wherein, The first mirror comprises a convex mirror.

Images