Unmanned aerial vehicle optical radar detection system and method and unmanned aerial vehicle

Abstract

The invention discloses an unmanned aerial vehicle optical radar detection system and method and an unmanned aerial vehicle, and belongs to the field of unmanned aerial vehicles. Comprising a detection subsystem, a turntable subsystem, an image processing subsystem and a display control subsystem. Wherein the rotary table subsystem comprises a rotary table, and the detection subsystem is loaded in the rotary table, so that when the rotary table rotates, the detection subsystem can obtain image information of different orientations and different pitching angles. In this way, images in multiple directions and angles can be obtained without installing multiple cameras, and the cost of the unmanned aerial vehicle is greatly reduced.

Description

Technical Field

[0001] The present application relates to the field of drone technology, and in particular, to a drone optical radar detection system and method and a drone. Background Art

[0002] The drone industry has experienced explosive growth in recent years. According to data, by 2023, the scale of China's civilian drone industry had exceeded 120 billion yuan, and is expected to reach approximately 160 billion yuan by 2024, demonstrating strong growth momentum. Furthermore, China holds a significant share of the global military drone market, further driving the industry's development.

[0003] The rapid development and widespread application of drone technology is a key driver of the industry's continued expansion. Drones are not only widely used in traditional fields such as aerial photography, agriculture, and environmental monitoring, but also demonstrate significant potential in emerging sectors such as logistics and transportation, urban air mobility, and emergency rescue. With the booming low-altitude economy, the application scenarios of drones in cities will become more diverse, further unleashing market potential.

[0004] However, the low-altitude environment is more complex, including not only flying birds but also some floating objects. The cameras in the existing drone designs can only detect targets in a certain direction. If you want to detect targets in multiple directions, you need multiple cameras, which is costly. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, the present application provides a drone optical radar detection system and method and a drone to solve the problem that the camera in the existing drone design can only detect a certain direction. If you want to detect targets in multiple directions, you need multiple cameras, which is costly.

[0006] The technical solution adopted by this application to solve its technical problems is:

[0007] In a first aspect, a UAV optical radar detection system is provided, comprising:

[0008] Detection subsystem, used to obtain image information and distance information;

[0009] A turntable subsystem, comprising a turntable, wherein the detection subsystem is mounted on the turntable, and when the turntable rotates, the detection subsystem can obtain image information at different azimuths and different elevation angles;

[0010] An image processing subsystem, configured to identify image information acquired by the detection subsystem;

[0011] The display and control subsystem is used to display the recognition results and control the rotation of the turntable subsystem.

[0012] Furthermore, the system includes a weekly scanning mode and a tracking mode;

[0013] In the circumferential scanning mode, the turntable rotates 360° in azimuth at a preset speed, and the pitch can be stepped up or down by a preset angle per rotation;

[0014] In the tracking mode, the rotation of the turntable is controlled according to the position of the target, which includes azimuth, pitch angle and distance.

[0015] Furthermore, controlling the rotation of the turntable according to the position of the target includes:

[0016] Get the target image and current frame image;

[0017] A pre-trained network model is used to extract the attention vector of the target image, as well as the semantic vector of each sub-region of the current frame image and the fusion of the target image and the eigenvalues corresponding to each semantic vector;

[0018] Calculating the similarity between each semantic vector and the attention vector; multiplying the similarity by the feature value and taking the sub-region with the highest product as the target sub-region;

[0019] The position of the target sub-region is acquired, and the rotation of the turntable is controlled according to the position of the target sub-region.

[0020] Furthermore, it also includes:

[0021] In the tracking mode, acquiring an updated image, wherein the updated image and the initial image are images of the same target;

[0022] The score mark is used as a first query vector, and a first key vector and a first value vector are obtained according to the update image;

[0023] Inputting the first query vector, the first key vector, and the first value vector into a first attention module to obtain a second query vector; and simultaneously obtaining a second key vector and a second value vector based on the initial image;

[0024] Inputting the second query vector, the second key vector, and the second value vector into a second attention module and passing them through a multilayer perceptron and an activation function to obtain a prediction score for the updated image;

[0025] When the prediction score exceeds a threshold, the updated image is used as a target image.

[0026] Furthermore, it also includes:

[0027] In the tracking mode, if the target is lost, the loss duration is calculated starting from the time the target is lost.

[0028] When the loss duration is less than a preset duration, the predicted position of the target is determined based on a target prediction curve, and the turntable is controlled to rotate according to the predicted position.

[0029] Furthermore, determining the predicted position of the target based on the target prediction curve includes:

[0030] Obtaining the true curve of the target before it is lost, and obtaining the predicted curve of the target using three prediction methods, wherein the three prediction methods are linear prediction, quadratic prediction, and hybrid prediction. The linear prediction is performed when the target is in pure uniform motion, the quadratic prediction is performed when the target is in pure accelerated motion, and the hybrid prediction is performed when the target is in both uniform and accelerated motion.

[0031] Calculating the distance between each predicted curve and the true curve at the lost position, where the lost position is the position at which the target is lost;

[0032] The prediction curve with the smallest distance is taken as the target prediction curve.

[0033] Furthermore, it also includes:

[0034] Upon receiving a cyclic scanning mode start instruction from a host computer, the servo communication board of the turntable transmits a signal generated by the servo communication board to the thermal imager oscillating mirror to control the thermal imager oscillating mirror to start oscillating scanning. The thermal imager oscillating mirror is used to compensate the image acquired by the thermal imager in the cyclic scanning mode;

[0035] Obtaining an integral pulse generated when the thermal imager's oscillating mirror starts to swing, and using the integral pulse as a synchronization signal;

[0036] The synchronization signal is divided into three paths, the first path is transmitted to the thermal imager as a synchronous integral signal to control the integral exposure of the thermal imager and synchronously output the image frame; the second path is transmitted to the turntable servo communication board to synchronously read the turntable / pitch code disk data and transmit it to the image processor in real time; the third path is transmitted to the image processor to synchronously detect the target's miss distance, and the azimuth / pitch code disk data collected and transmitted by the turntable servo communication board is synchronously written on the first line of the current frame image as the target positioning data calculation parameter. At the same time, the image processor calculates the target's orientation data according to the mathematical model using the target miss distance data and the turntable's azimuth / pitch code disk data;

[0037] The image processor numbers the continuous image frames output by the thermal imager starting with the zero-crossing signal transmitted by the turntable azimuth encoder, and stitches the images according to the sequence number to form a panoramic image, which is transmitted to the host computer through the network for display.

[0038] Furthermore, it also includes:

[0039] Upon receiving the tracking mode start command from the host computer, the turntable servo communication board generates a synchronization signal that is simultaneously transmitted to the thermal imager, visible light camera, and laser rangefinder;

[0040] The synchronization signal is divided into three paths. The first path is transmitted to the thermal imager as a synchronous exposure signal to control the thermal imager's integral exposure and synchronously output the image frame; the second path is transmitted to the turntable communication board to synchronously read the turntable azimuth / pitch code disk data and transmit it to the image processor in real time; the third path is transmitted to the image processor to synchronously detect the target's miss distance, and synchronously write the azimuth / pitch code disk data collected and transmitted by the turntable servo communication board on the first line of the current frame image as a target positioning data calculation parameter; at the same time, the image processor calculates the target positioning data or target orientation data according to the mathematical model using the target miss distance data and the turntable's azimuth / pitch code disk data, laser rangefinder data, and turntable attitude sensor data.

[0041] In a second aspect, a UAV optical radar detection method is provided, which is applied to the UAV optical radar detection system described in any of the above items.

[0042] In a third aspect, a drone is provided, comprising a drone optical radar detection system as described in any of the above items.

[0043] Beneficial effects:

[0044] The technical solution of this application provides a UAV optical radar detection system and method, as well as a UAV, comprising a detection subsystem, a turntable subsystem, an image processing subsystem, and a display and control subsystem. The turntable subsystem includes a turntable, within which the detection subsystem is mounted. As the turntable rotates, the detection subsystem can acquire image information from different directions and pitch angles. This eliminates the need to install multiple cameras, enabling the acquisition of images from multiple directions and angles, significantly reducing the cost of the UAV. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0046] Figure 1 This is a schematic diagram of the structure of a UAV optical radar detection system provided in an embodiment of the present application;

[0047] Figure 2 This is a schematic diagram of synchronization control in a weekly scanning working mode provided by an embodiment of the present application;

[0048] Figure 3This is a schematic diagram of synchronization control in a tracking mode provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application are described in detail below with reference to the accompanying drawings and examples. Obviously, the described embodiments are only some of the embodiments of this application, rather than all of them. Based on the embodiments in this application, all other implementation methods obtained by ordinary technicians in this field without making any creative work are within the scope of protection of this application.

[0050] Reference Figure 1 The embodiment of the present application provides a UAV optical radar detection system 10, comprising:

[0051] The detection subsystem 11 is used to acquire image and distance information. Preferably, the detection subsystem includes an infrared thermal imager, a visible light camera, and an optional laser rangefinder. The infrared thermal imager is primarily used at night or when visibility is low, while the visible light camera is primarily used during the day when visibility is good. The laser rangefinder is primarily used to measure the distance to the target.

[0052] The infrared thermal imager assembly consists of an infrared camera, a swinging scanning mirror, a visual axis adjustment and locking device, a protective housing, etc. The functions and performance of the infrared camera assembly are as follows:

[0053] a. Detector: Long-wave uncooled infrared 1280×1024; b. Operating band: 8μm–14μm; c. NETD ≤ 50mk; d. Focal plane pixel count: 1280×1024; e. Pixel size: 12μm; f. Frame rate: 50Hz; g. Fixed-focus lens with a focal length of 80mm. h. Focus memory function, automatically adjusting focus based on the current field of view; i. Shutter correction and background correction functions, with adjustable automatic and manual correction intervals. j. Video signal output: MIPI.

[0054] Visible light camera composition and performance

[0055] The visible light camera assembly consists of a visible light camera (consisting of a visible light lens and a camera), a visual axis adjustment and locking device, a protective housing, etc. The functions and performance of the visible light camera assembly are as follows:

[0056] ①Visible light camera

[0057] a. Operating wavelength: 0.4μm-1.1μm; b. Focal plane pixel count: 1 / 1.8" CMOS 1920×1080; c. Pixel size: 2.9μm; d. Frame rate: ≥30 frames; e. Variable zoom lens with a focal length of 6-560mm; f. Focusing modes: Autofocus / single focus / manual focus / semi-auto focus; g. Optical fog penetration: Supported; h. Electronic image stabilization: Supported; i. Video signal: HD-SDI, Ethernet port; j. Power supply: DC12V±10%; k. Power consumption: 2.5W static (5.5W maximum).

[0058] ②Optical axis adjustment locking device

[0059] a. Ability to level the optical axis; b. Ability to adjust the azimuth and pitch angle of the optical axis; c. Ability to maintain the position after locking and a certain degree of vibration resistance.

[0060] Laser rangefinder composition and performance

[0061] The laser rangefinder is mainly composed of three systems: laser emission system, laser receiving system, and signal processing system. Main performance parameters:

[0062] ① Operating wavelength: 1.54μm; ② Beam divergence: ≥0.6mrad; ③ Laser optical axis stability: ≤0.05mrad; ④ Parallelism error of the optical axis to the installation datum: ≤0.3mrad; ⑤ Accuracy: ≥98%; ⑥ Ranging frequency: 5Hz, 10Hz; ⑦ Ranging range: ≥2km for a 350mm radial UAV target with a diffuse reflectance of not less than 0.2 (visibility not less than 10km, relative humidity not greater than 70%); ⑧ Ranging accuracy: 3m (RMS); ⑨ Ranging cycle: single ranging, one-minute continuous ranging;

[0063] The turntable subsystem 12 includes a turntable, in which the detection subsystem is loaded. When the turntable rotates, the detection subsystem can obtain image information at different orientations and different pitch angles. Preferably, the turntable is loaded with a detection subsystem, an electronic control component and an image processor to drive the rotation of the detection subsystem to realize the search of the airspace and the tracking of the target.

[0064] The electronic control component consists of a servo control communication board, a power supply board, etc. The functions of the module are described as follows:

[0065] ①Power board

[0066] The host power input uses DC 28V, which is then converted by the power board to the rest of the power used in the host subsystem; the module supplies the voltage and distributes it to the output. This module controls, manages and protects all power outputs.

[0067] ②Servo control communication board

[0068] aThis module parses the commands of the system display and control computer through the Ethernet interface and distributes them to other controlled modules of the system (including infrared thermal imager, visible light camera, laser rangefinder, image processor, etc.), and at the same time packages the feedback information of the controlled modules (including current status information, parameter information, fault information and various measurement information) and sends it to the display and control computer.

[0069] bThis module is responsible for generating the system synchronization pulse from the system clock source and all the synchronization pulse signals of the system.

[0070] c Complete the synchronous control of the system.

[0071] That is, this module is responsible for using the pulse signal output by the turntable servo control board as a synchronization pulse signal to start the swing mirror in the circular scanning mode, and sending it as an external synchronization signal to the thermal imager to start the thermal imager exposure. This synchronization pulse is used to read the code disk data latched by the turntable, thereby completing the synchronization of the turntable azimuth / pitch angle, thermal imager exposure, and the swing mirror.

[0072] Turntable composition and main indicators

[0073] ①Composition:

[0074] The turntable consists of an azimuth mechanism (azimuth encoder, azimuth motor, transmission mechanism), a pitch mechanism (pitch encoder, pitch motor, transmission mechanism), a turntable servo control communication module, and a slip ring. The functions of each module are described as follows:

[0075] a. Positioning mechanism

[0076] The azimuth mechanism provides the turntable's azimuth surface, is the actuator for the turntable's azimuth rotation, and provides information about the turntable's azimuth angle. The slip ring connects the power supply and signals of the power modules above the azimuth rotation surface with the circuits of the modules below the azimuth rotation surface.

[0077] b. Pitch mechanism

[0078] The pitch mechanism provides the turntable pitch surface, is the executive component of the turntable pitch rotation, and provides the turntable pitch angle information.

[0079] c.Servo control communication module

[0080] The servo control communication module is the control component for the azimuth and pitch servo drives and serves as the central control module for the host subsystem. This module utilizes a fully digital dual closed-loop control system consisting of an inner speed loop and an outer position loop to drive the turntable, achieving position and speed control, as well as stable control of the turntable.

[0081] d. Gyroscope

[0082] The gyroscope is installed in the detector chassis and serves as a measurement device for the attitude data of the photoelectric detector (infrared, visible light, laser). It provides real-time attitude input data for the turntable servo control board to control the azimuth and pitch mechanisms to ensure the stability of the detector's optical axis and the accuracy of the optical axis angle.

[0083] ②Main indicators

[0084] a. Rotary table axis servo

[0085] Rotation range: Pitch pointing range: -20°~80°; Azimuth pointing range: 360°, azimuth continuous rotation;

[0086] Angle measurement accuracy: ≤0.01°

[0087] Servo angular velocity: angular velocity ≥ 180° / s, angular acceleration ≥ 60° / s2

[0088] Mechanical leveling mechanism, the turntable can be adjusted horizontally by manual adjustment. After adjustment, the turntable level is ≤0.002°;

[0089] The image processing subsystem 13 is used to identify the image information acquired by the detection subsystem. Preferably, the image processing subsystem includes: an image processor, an image processing algorithm and software.

[0090] The display and control subsystem 14 is used to display the recognition results and control the rotation of the turntable subsystem. Preferably, the display and control subsystem includes a display and control computer, a management and control platform software, and a power box.

[0091] Each subsystem functions independently and is highly modular. The detection subsystem utilizes a U-shaped frame for structural connection to the turntable. Flanges and bases are used for structural connection, while the turntable subsystem and display and control subsystem are electrically connected using a network cable.

[0092] The system includes a weekly scanning mode and a tracking mode;

[0093] In the circumferential scanning mode, the turntable rotates 360° in azimuth at a preset speed, and the pitch can be stepped up or down by a preset angle per rotation;

[0094] In the tracking mode, the rotation of the turntable is controlled according to the position of the target, which includes azimuth, pitch angle and distance.

[0095] As a preferred implementation method of the embodiment of the present application, the rotation of the turntable is controlled according to the position of the target, including: acquiring a target image and a current frame image; using a pre-trained network model to extract the focus vector of the target image, and extracting the semantic vector after the fusion of each sub-region of the current frame image and the target image and the eigenvalue corresponding to each semantic vector; calculating the similarity between each semantic vector and the focus vector; multiplying the similarity with the eigenvalue and using the sub-region with the highest product as the target sub-region; acquiring the position of the target sub-region, and controlling the rotation of the turntable according to the position of the target sub-region.

[0096] That is, in the tracking mode, the rotation of the turntable is controlled according to the current frame image and the target image. According to the tracking algorithm of the present application, the target can be determined from the current frame image and accurately tracked.

[0097] Among them, the pre-trained network includes an input layer, a hidden layer and an output layer. The input layer includes two input units, one for inputting the target image and the other for inputting the current frame image. The hidden layer includes three convolutional layers, and the output resolution of each convolutional layer gradually becomes smaller and the number of channels gradually increases. The output layer is used to output the sub-region of the current frame image that includes the target. Unlike traditional neural networks, the present application not only extracts the respective features of the target image and the current image, but also fuses the features of the sub-regions of the target image and the current image. If the sub-region includes the target, the fused features are the same as the features of the target image. If the sub-region does not include the target, the fused features are quite different from the features of the target image, and can be quickly distinguished.

[0098] To validate the proposed framework, we conducted experiments on two simple localization heads: the Corner head and the Token head. The Corner head draws inspiration from STARK, but unlike STARK, it does not integrate the decoder and encoder in the head. Instead, it includes only a few convolutional layers to locate corner points and extract them from the frame. We also experimented with a query-based localization head similar to DETR, adding a learnable regression token in the final stage and directly regressing the coordinates through an FFN. This method also achieved good results, demonstrating the excellent generalization capabilities of the aforementioned framework.

[0099] The method also includes: obtaining an updated image in the tracking mode, wherein the updated image and the initial image are images of the same target; using the score label as a first query vector, and simultaneously obtaining a first key vector and a first value vector based on the updated image; inputting the first query vector, the first key vector, and the first value vector into a first attention module to obtain a second query vector; simultaneously obtaining a second key vector and a second value vector based on the initial image; inputting the second query vector, the second key vector, and the second value vector into a second attention module and performing a multi-layer perceptron and an activation function to obtain a predicted score for the updated image; and when the predicted score exceeds a threshold, using the updated image as the target image. Because the target may be deformed or the target may appear different due to viewing angles, if the target image remains unchanged, tracking accuracy will be reduced. Therefore, it is necessary to update the target image in real time. If the updated target image is not the correct target, tracking accuracy will be further reduced. Therefore, when tracking, if the target is matched from the current frame image, the image in the current frame is used as the updated image. This can effectively improve accuracy.

[0100] In addition, in the tracking mode, if the target is lost, the loss duration is obtained by counting from the time the target is lost; when the loss duration is less than the preset duration, the predicted position of the target is determined based on the target prediction curve, and the turntable is controlled to rotate according to the predicted position.

[0101] If the target is lost due to occlusion, it needs to be recaptured. There are two situations:

[0102] 1) In a short period of time, such as 3 seconds, the target recapture area is the tracking gate obtained based on memory prediction, and the target is recognized using the prior knowledge remembered immediately before the target is lost.

[0103] 2) If the target is lost for a long time, such as more than 3 seconds, the recapture area is enlarged and the target is searched and identified again.

[0104] In the system, the target tracking results are directly output to control the turntable to make corresponding actions in order to achieve target tracking. In order to achieve stable target tracking, the output of the target tracking results should be filtered to achieve a stable and smooth target tracking process. Usually, the smoothing filtering method is used to achieve this. Here, the Kalman filter method is chosen to achieve smooth filtering of the target tracking output results. In addition, the system requires that the servo control "blind tracking" time is greater than 3s, that is, when the target is obscured and does not appear in the image for some reason, the target tracking can be maintained. At this time, the Kalman filter algorithm can also be used to estimate the target's motion trajectory. When the target is not tracked, the turntable is controlled to maintain the tracking state so that the target can be captured again in the field of view and continue tracking after the obstruction is removed.

[0105] Furthermore, determining the predicted position of the target based on the target prediction curve includes:

[0106] Obtain a true curve before the target is lost, and obtain three prediction methods to obtain a prediction curve of the target, the three prediction methods are linear prediction, quadratic prediction and hybrid prediction, the linear prediction is a prediction when the target is in pure uniform motion, the quadratic prediction is a prediction when the target is in pure accelerated motion, and the hybrid prediction is a prediction when the target is in both uniform motion (a certain direction, such as vertical direction) and accelerated motion (another direction, such as horizontal direction); wherein the hybrid prediction is in the form of a function that assigns a weight to the functions of the linear prediction and the quadratic prediction, respectively, wherein the weight can be obtained according to fitting, or determined according to the angular acceleration when the target is lost.

[0107] Calculating the distance between each predicted curve and the true curve at the lost position, where the lost position is the position at which the target is lost;

[0108] The prediction curve with the smallest distance is taken as the target prediction curve.

[0109] Operating mode design is a key factor in optimizing the performance of a LiDAR detection system. The appropriate operating mode not only facilitates operation but also maximizes system performance. Operating mode design is based on a specific mission analysis. The ultimate requirements of a LiDAR detection system are target search, identification, tracking, and positioning. The system is designed with the following basic operating modes, which can be switched between at any time.

[0110] (1) Automatic tracking mode

[0111] The system performs a 360° omnidirectional scanning search at a certain speed. Every time it rotates, the pitch can be stepped up or down by a fixed angle (settable) to perform follow-up search and detect and identify moving targets in the search area.

[0112] Once the target is captured (the gate locks onto the target), the "Search to Track" mode can be activated. In automatic tracking mode, the system controls the turntable to automatically follow the target, stabilizing the target at the center of the optical axis of the field of view. Based on the image data and turntable data, the system calculates and outputs the target's precise azimuth, pitch angle, and distance (with a laser rangefinder) in real time.

[0113] If the target is obscured or lost due to other reasons during the tracking process, the system will pre-track the target for 3 seconds based on its previous motion trajectory, and recapture and track the target during the pre-tracking process.

[0114] (2) External guidance automatic tracking mode

[0115] After the radar or other equipment detects the target, the system sends the target parameters (azimuth / elevation angle and distance) to guide the optoelectronic system to aim the detection equipment at the target, and detects and captures the target that enters the detector's field of view. After the target is captured, it automatically switches to automatic tracking mode, and measures and outputs the target's position data (number, azimuth, elevation angle, distance and time) during the target tracking process.

[0116] Synchronization is an important part of this measurement system. To ensure the accuracy of the system measurement and ease of use, this system uses an internal synchronization signal. For external synchronization signals, this system only receives time codes to synchronize the system's time signal.

[0117] The information synchronization of this system is divided into two working modes: one is the synchronization mode in the tracking working mode; the other is the synchronization mode in the weekly scanning working mode. The two working modes can be switched. The two working modes are described as follows:

[0118] (1) Synchronization method in tracking working mode

[0119] When the system is in tracking mode, the turntable communication processing board automatically generates a 50Hz synchronization signal, and the system works based on this signal.

[0120] (2) Synchronization method in weekly scanning mode

[0121] In the cyclic scanning mode, the system uses the synchronization signal generated by the turntable communication board to start the thermal imager's oscillating mirror. The oscillating mirror generates an exposure pulse signal as a synchronization signal. First, it is transmitted to the thermal imager as a synchronous integration signal to control the thermal imager's integration. Second, it is transmitted to the turntable communication board to synchronously read the turntable's azimuth and pitch code disk data. Third, it is transmitted to the image processor to control the target's deviation from the image center (miss distance). The azimuth and pitch code disk data collected and transmitted by the turntable communication board are synchronously written to the first line of the current frame image as the target positioning data calculation parameter. The system operates based on this signal as a benchmark.

[0122] System synchronization signal distribution

[0123] The host communication processing board uses a 100 MHz clock as its local clock source and synthesizes it into a 50 Hz system operating synchronous clock. The communication processing board transmits this 50 Hz system operating synchronous clock to the servo control board, image processor, infrared thermal imager, visible light camera, and laser rangefinder. This allows all of these devices to operate based on a unified synchronous signal.

[0124] The servo control board uses a 100MHz clock as its local clock source, synthesizing it into a 200Hz signal. The phase of this 200Hz signal is then corrected using the system's 50Hz synchronous clock source. This 200Hz signal serves as the synchronous readout for the azimuth and pitch encoders, and serves as the reference for the servo control board's velocity loop control. The servo control board's position loop is also referenced to the system's 50Hz synchronous signal.

[0125] Synchronous processing of system real-time data

[0126] The system performs synchronous data processing in a synchronized manner. The system calculates the target's current positioning information based on information such as the target miss distance from the video image, the turntable's azimuth / pitch angles, and the turntable's horizontal tilt angle. The entire system operates synchronously at 50Hz. In this synchronous mode, the system queues the collected data, including the target miss distance from the video image, the turntable's azimuth / pitch angles, and the host's horizontal tilt angle, aligning the data at the same moment in time. The system then completes the target positioning data processing according to a fixed sequence. This synchronous operation ensures the accuracy of the system's external data output.

[0127] System synchronization in weekly scanning mode

[0128] During the high-speed rotation of the turntable, in order to achieve clear imaging and 360° panoramic image stitching, each frame of the image requires precise timing synchronization and alignment of the thermal imager, scanning mirror, and turntable.

[0129] The host computer sets the system to circular scanning mode, sets the turntable circular scanning speed, the thermal imager's integration time, and the imaging frame rate, and issues a circular scanning command to control the turntable to scan at the set speed. The command is sent to the servo communication processing board, which calculates the synchronization period based on the set parameters and outputs a synchronization signal based on this period. This synchronization signal is simultaneously sent to the thermal imager and the swinging scanning mirror, achieving synchronous operation of the thermal imager, swinging scanning mirror, and turntable. The specific working process is as follows:

[0130] (1) In the weekly scanning mode, if Figure 2 As shown:

[0131] The host computer issues a sweep command, which transmits a synchronization signal generated by the turntable servo communication board to the thermal imager's oscillating mirror, initiating the mirror's sweep. The mirror generates an integral signal, which serves as a synchronization signal. The first signal is transmitted to the thermal imager as a synchronous integral signal to control the camera's integral exposure and synchronize image frame output. The second signal is transmitted to the turntable servo communication board to synchronously read the turntable's azimuth / pitch encoder data and transmit it to the image processor in real time. The third signal is transmitted to the image processor to synchronously detect the target's deviation (miss distance) in the image and write the azimuth / pitch encoder data collected and transmitted by the turntable servo communication board to the first line of the current frame image as a parameter for target positioning data calculation. Simultaneously, the image processor calculates the target's orientation (azimuth / pitch angle) using a mathematical model based on the target miss distance data and the turntable's azimuth / pitch encoder data.

[0132] The image processor numbers the continuous image frames output by the thermal imager, starting with the zero-crossing signal transmitted by the turntable azimuth encoder. It then stitches the images together according to the sequence number to form a panoramic image, which is then transmitted to the host computer via the network for display. The target orientation data measured in each frame is simultaneously transmitted to the host computer via the serial port for display and recording.

[0133] (2) In tracking mode, if Figure 3 As shown:

[0134] The host computer issues tracking commands, using synchronization signals generated by the turntable servo communication board to simultaneously transmit them to the thermal imager, visible light camera, and laser rangefinder. The first signal is transmitted to the thermal imager as a synchronous exposure signal to control the thermal imager's integral exposure and synchronize image frame output. The second signal is transmitted to the turntable communication board to synchronously read the turntable's azimuth / pitch encoder data and transmit it to the image processor in real time. The third signal is transmitted to the image processor to synchronously detect the target image's center deviation (miss distance) within the image. The azimuth / pitch encoder data collected and transmitted by the turntable servo communication board is then written to the first line of the current frame image as a parameter for target positioning data calculation. Simultaneously, the image processor uses the target miss distance data, the turntable's azimuth / pitch encoder data, the laser rangefinder data, and the turntable's attitude sensor data according to a mathematical model to calculate the target's positioning data (when distance is available) or target orientation data (when distance is not available).

[0135] Based on the same inventive concept, the present application also provides a UAV optical radar detection system method, which is applied to the UAV optical radar detection system described in any of the above items.

[0136] Based on the same inventive concept, the present application also provides a drone, comprising a drone optical radar detection system as described in any of the above items.

[0137] It should be noted that, in the description of this application, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0138] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

Claims

1. An optical radar detection system for unmanned aerial vehicles, characterized in that: include: Detection subsystem, used to obtain image information and distance information; A turntable subsystem, comprising a turntable, wherein the detection subsystem is mounted on the turntable, and when the turntable rotates, the detection subsystem can obtain image information at different azimuths and different elevation angles; An image processing subsystem, configured to identify image information acquired by the detection subsystem; The display and control subsystem is used to display the recognition results and control the rotation of the turntable subsystem.

2. The system according to claim 1, wherein: The system includes a weekly scanning mode and a tracking mode; In the circumferential scanning mode, the turntable rotates 360° in azimuth at a preset speed, and the pitch can be stepped up or down by a preset angle per rotation; In the tracking mode, the rotation of the turntable is controlled according to the position of the target, which includes azimuth, pitch angle and distance.

3. The system according to claim 2, characterized in that: The controlling the rotation of the turntable according to the position of the target comprises: Get the target image and current frame image; A pre-trained network model is used to extract the attention vector of the target image, as well as the semantic vector of each sub-region of the current frame image and the fusion of the target image and the eigenvalues corresponding to each semantic vector; Calculating the similarity between each semantic vector and the attention vector; multiplying the similarity by the feature value and taking the sub-region with the highest product as the target sub-region; The position of the target sub-region is acquired, and the rotation of the turntable is controlled according to the position of the target sub-region.

4. The system according to claim 3, characterized in that Also includes: In the tracking mode, acquiring an updated image, wherein the updated image and the initial image are images of the same target; The score mark is used as a first query vector, and a first key vector and a first value vector are obtained according to the update image; Inputting the first query vector, the first key vector, and the first value vector into a first attention module to obtain a second query vector; and simultaneously obtaining a second key vector and a second value vector based on the initial image; Inputting the second query vector, the second key vector, and the second value vector into a second attention module and passing them through a multilayer perceptron and an activation function to obtain a prediction score for the updated image; When the prediction score exceeds a threshold, the updated image is used as a target image.

5. The system according to claim 2, wherein: Also includes: In the tracking mode, if the target is lost, the loss duration is calculated starting from the time the target is lost. When the loss duration is less than a preset duration, the predicted position of the target is determined based on a target prediction curve, and the turntable is controlled to rotate according to the predicted position.

6. The system according to claim 5, characterized in that: Determining the predicted position of the target based on the target prediction curve includes: Obtaining the true curve of the target before it is lost, and obtaining the predicted curve of the target using three prediction methods, wherein the three prediction methods are linear prediction, quadratic prediction, and hybrid prediction. The linear prediction is performed when the target is in pure uniform motion, the quadratic prediction is performed when the target is in pure accelerated motion, and the hybrid prediction is performed when the target is in both uniform and accelerated motion. Calculating the distance between each predicted curve and the true curve at the lost position, where the lost position is the position at which the target is lost; The prediction curve with the smallest distance is taken as the target prediction curve.

7. The system according to claim 2, wherein: Also includes: Upon receiving a cyclic scanning mode start instruction from a host computer, the servo communication board of the turntable transmits a signal generated by the servo communication board to the thermal imager oscillating mirror to control the thermal imager oscillating mirror to start oscillating scanning. The thermal imager oscillating mirror is used to compensate the image acquired by the thermal imager in the cyclic scanning mode; Obtaining an integral pulse generated when the thermal imager's oscillating mirror starts to swing, and using the integral pulse as a synchronization signal; The synchronization signal is divided into three paths, the first path is transmitted to the thermal imager as a synchronization integration signal to control the thermal imager integral exposure and synchronously output image frames; The second channel is transmitted to the turntable servo communication board to synchronously read the turntable / pitch encoder data and transmit it to the image processor in real time; The third channel is transmitted to the image processor to synchronously detect the target's miss distance, and the azimuth / pitch code disk data collected and transmitted by the turntable servo communication board is synchronously written to the first line of the current frame image as the target positioning data calculation parameter. At the same time, the image processor calculates the target's orientation data based on the target miss distance data and the turntable's azimuth / pitch code disk data according to the mathematical model; The image processor numbers the continuous image frames output by the thermal imager starting with the zero-crossing signal transmitted by the turntable azimuth encoder, and stitches the images according to the sequence number to form a panoramic image, which is transmitted to the host computer through the network for display.

8. The system according to claim 2, wherein: Also includes: Upon receiving the tracking mode start command from the host computer, the turntable servo communication board generates a synchronization signal that is simultaneously transmitted to the thermal imager, visible light camera, and laser rangefinder; The synchronization signal is divided into three channels. The first channel is transmitted to the thermal imager as a synchronization exposure signal to control the thermal imager's integral exposure and synchronously output image frames. The second channel is transmitted to the turntable communication board to synchronously read the turntable azimuth / pitch code disk data and transmit it to the image processor in real time; The third channel is transmitted to the image processor to synchronously detect the target's miss distance, and the azimuth / pitch code disk data collected and transmitted by the turntable servo communication board is synchronously written in the first line of the current frame image as the target positioning data calculation parameter; at the same time, the image processor calculates the target positioning data or target orientation data according to the mathematical model using the target miss distance data and the turntable's azimuth / pitch code disk data, laser rangefinder data, and turntable attitude sensor data.

9. A method for detecting unmanned aerial vehicle optical radar, characterized in that: Applicable to the system according to any one of claims 1 to 8.

10. A drone, characterized by: A system comprising any one of claims 1 to 8.

Images