An optoelectronic observation device for the control of a drone

Abstract

This invention discloses an optoelectronic observation device for controlling unmanned aerial vehicles (UAVs). The device includes an embedded information processing module and multiple front-end acquisition units and a servo turntable connected in communication to form an integrated hardware structure. The embedded information processing module runs a shared software platform, which includes control software modules for each front-end acquisition unit, a UAV target recognition and tracking software module, a servo turntable control software module, a comprehensive processing software module, and an unsupervised self-learning software module. The unsupervised self-learning software module iteratively optimizes the detection and recognition model when the device is idle and automatically deploys it to the UAV target recognition and tracking software module. This application effectively solves the problems of low target recognition accuracy, slow tracking response, poor software-hardware coordination, and low intelligence level in existing UAV control tasks through the deep integration of a shared software platform, integrated hardware, and an unsupervised self-learning detection and recognition model, thereby improving the reliability, efficiency, and intelligence level of UAV control tasks.

Description

Technical Field

[0001] This invention relates to the field of optoelectronic observation equipment technology, specifically to an optoelectronic observation device for controlling unmanned aerial vehicles (UAVs), and more particularly to an optoelectronic observation device for controlling UAVs that integrates hardware integration, software platform sharing, and an unsupervised self-learning detection and recognition model. Background Technology

[0002] As the threat posed by illegally flying drones to key security areas such as airports, oil fields, and prisons intensifies, drone detection and control technologies have emerged. In drone detection and control systems, optoelectronic observation equipment plays a crucial and indispensable role, bearing the core responsibility of target identification and confirmation. Under normal circumstances, optoelectronic observation equipment receives guidance information from radar or radio detection equipment. By adjusting the equipment, it acquires real-time video and related data information of suspicious areas, thereby enabling the identification and confirmation of suspicious targets.

[0003] However, existing UAV photoelectric observation equipment still has many shortcomings and is difficult to meet the high-precision control requirements of key areas: (1) The signal processing efficiency is low, the target recognition accuracy is not high, and it is easily affected by external environmental interference, which leads to misjudgment and affects the timeliness of control decisions. Moreover, most of the existing detection and recognition models are supervised learning, which requires manual labeling of a large number of samples. The detection and recognition models are already solidified before the equipment leaves the factory. If an update is needed, a new detection and recognition model needs to be reloaded, which cannot adapt to different scene changes independently; (2) The software and hardware are mostly designed separately, lacking a unified software sharing platform. Each hardware module has its own software platform, poor coordination, data cannot be shared efficiently, functional expansion is cumbersome, and the hardware integration is low. The equipment cost is high, the size is large, and the operation and maintenance are inconvenient, which further limits the control efficiency and intelligence level of the equipment.

[0004] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to address the shortcomings of the existing technology by providing an optoelectronic observation device for controlling drones. This device effectively solves the technical problems of low target recognition accuracy, slow tracking response, poor software and hardware coordination, and low intelligence level in complex environments. Through the deep integration of software platform, hardware integration, and unsupervised self-learning model, the reliability, efficiency, and intelligence level of drone control tasks are significantly improved, and the device can autonomously adapt to different scene changes.

[0006] To address the aforementioned technical problems, this invention discloses an optoelectronic observation device for controlling unmanned aerial vehicles (UAVs), comprising an embedded information processing module and multiple front-end acquisition units and a servo turntable that are directly connected to the embedded information processing module to form a hardware integrated structure.

[0007] The embedded information processing module serves as the core control unit of the integrated hardware structure and runs on a shared software platform.

[0008] The software uses a shared platform and adopts a modular design, including:

[0009] Each front-end acquisition unit control software module is used to control the corresponding front-end acquisition unit to acquire and preprocess video image data;

[0010] The UAV target recognition and tracking software module is used to acquire preprocessed video image data, call the built-in detection and recognition model to perform target detection, tracking and recognition on the video image data, and output the target's three-dimensional coordinate information;

[0011] The servo turntable control software module is used to control the servo turntable to drive the multiple front-end acquisition units to rotate and track the target based on the target's three-dimensional coordinate information;

[0012] The integrated processing software module is used to interact with the drone management platform, issue control commands to the aforementioned control software modules, and collect status information from each module.

[0013] The system also includes an unsupervised self-learning software module, which iteratively optimizes the detection and recognition model when the device is idle, and deploys the optimized detection and recognition model to the UAV target recognition and tracking software module to improve the accuracy of subsequent UAV target recognition and adaptability in complex environments.

[0014] In one embodiment, the plurality of front-end acquisition units include an infrared thermal imager, a visible light camera, and a laser rangefinder. Each of the plurality of front-end acquisition units transmits its own acquired raw video image data to the embedded information processing module in real time, providing data support for subsequent video preprocessing, target detection, tracking, and recognition.

[0015] In one embodiment, the integrated hardware structure further includes a power supply unit for supplying power to the integrated hardware structure.

[0016] In one embodiment, the servo turntable includes an azimuth rotation mechanism and a pitch mechanism disposed on top of the azimuth rotation mechanism, and the plurality of front-end acquisition units and the embedded information processing module are carried on the pitch mechanism.

[0017] In one embodiment, the azimuth rotation mechanism includes an azimuth frame, an azimuth driver, an azimuth motor, an azimuth encoder, and an azimuth gyroscope. The pitch mechanism includes a pitch frame, a pitch driver, a pitch motor, a pitch encoder, and a pitch gyroscope. The plurality of front-end acquisition units and the embedded information processing module are mounted on the pitch frame. The azimuth encoder and the pitch encoder are used to acquire the rotation angle information of the servo turntable in real time. The azimuth gyroscope and the pitch gyroscope are used to acquire the angular velocity information of the servo turntable in real time, and feed the rotation angle information and angular velocity information back to the servo turntable control software module. The servo turntable control software module compares and calibrates the feedback information with the received control commands, generates corresponding azimuth control signals and pitch control signals, and sends them to the azimuth driver and the pitch driver, respectively. The azimuth driver is connected to the azimuth motor and is used to drive the azimuth frame to rotate around the azimuth axis. The pitch driver is connected to the pitch motor and is used to drive the pitch frame to rotate around the pitch axis, thereby forming a closed-loop control of the servo turntable.

[0018] The specific structure of the servo turntable is not the main improvement point of this application; existing servo turntable structures can be used.

[0019] In one embodiment, the shared software platform adopts a standardized interface design, supporting multi-version compatibility and functional plugin expansion. New recognition algorithms or drone control functions can be flexibly added according to actual needs. This eliminates the need for overall software reconstruction and hardware structure modification, reducing equipment upgrade and maintenance costs. Simultaneously, the shared software platform enables data sharing among various hardware modules, breaking down the barriers between software and hardware design, improving the collaborative efficiency of each module, and ensuring deep collaboration between the unsupervised self-learning model, target recognition algorithm, and hardware modules such as photoelectric detection and turntable drive. This achieves integrated management and control of observation, recognition, tracking, and transmission.

[0020] In one embodiment, the iterative optimization of the detection and recognition model includes:

[0021] Simultaneously, data is automatically collected from historical and real-time video data without the need for manual sample labeling, and multi-dimensional features are automatically extracted. The multi-dimensional features include the drone's motion features, visible light optical features, and infrared features, wherein the motion features include at least the drone's shape outline, flight speed, and flight trajectory.

[0022] Unsupervised clustering is performed on the extracted multidimensional features to autonomously distinguish UAV targets from other targets. By iteratively optimizing the threshold, a new detection and recognition model is generated.

[0023] The new detection and recognition model is compared with the old detection and recognition model. If the new detection and recognition model is better than the old one, it is automatically deployed to the UAV target recognition and tracking software module. If it is worse than the old one, the data collection process is restarted.

[0024] The process of iteratively optimizing the detection and recognition model is carried out when the UAV photoelectric observation equipment is powered on but not performing its duty tasks. This does not affect the equipment's normal duty tasks and can make full use of the hardware resources of the UAV photoelectric observation equipment.

[0025] In one embodiment, the integrated processing software module includes a communication service component, an optoelectronic management component, a data storage and service component, an information fusion component, and a threat assessment component.

[0026] In one embodiment, the communication service component is used to interact with the management and control drone platform, receive guidance and control commands, and send the reported target information, video data, and device status information to the management and control platform; the photoelectric management component is used to distribute the control commands received by the communication service component to the corresponding software modules and collect the status information returned by each software module, summarize it, and then report it to the communication service component; the data storage and service component is used to receive the video data collected by the infrared thermal imager and the visible light camera, encode and compress it, store it, and push the video data to the management and control drone platform through the communication service component.

[0027] In one embodiment, the information fusion component is used to fuse target data from visible light video, target data from infrared video, and guidance data to form comprehensive intelligence, and report it to the UAV management platform; the threat assessment component is used to assess the degree of threat posed by the detected target to the defensive position.

[0028] Beneficial effects:

[0029] 1. This invention effectively solves the technical problems of low target recognition accuracy, slow tracking response, poor software-hardware coordination, and low intelligence level in existing systems by deeply integrating a shared software platform, integrated hardware, and an unsupervised self-learning detection and recognition model. This significantly improves the reliability, efficiency, and intelligence level of UAV management tasks. Specifically, the shared software platform and integrated hardware provide a low-latency, multi-source synchronous, and stable execution environment; the unsupervised self-learning closed loop utilizes this environment to continuously correct the target feature detection and recognition model; the corrected detection and recognition model is then fed back to the recognition, tracking, and servo control links; thus, in complex scenarios, it achieves stable tracking, rapid re-acquisition, and continuous task execution effects that are difficult to predict with traditional discrete systems or static detection and recognition model systems.

[0030] 2. This invention employs an embedded information processing module and target recognition algorithm, which improves signal processing efficiency and target recognition accuracy, effectively filters out interfering targets, and reduces the probability of misjudgment. At the same time, in conjunction with a high-precision servo turntable, it achieves fast and accurate tracking of UAV targets, with response speed and turning accuracy superior to existing equipment.

[0031] 3. This invention integrates a shared software platform with a unified hardware design, integrating all hardware modules into the same housing to form a compact, integrated structure. This reduces equipment size, facilitates installation and maintenance, and lowers costs. The software enables unified management, data sharing, and functional expansion of all hardware modules, breaking down the barriers between separate hardware modules, reducing information transmission latency, improving the collaborative efficiency of each module, lowering equipment upgrade and maintenance costs, and achieving integrated management of observation, identification, tracking, and transmission.

[0032] 4. This invention integrates an unsupervised self-learning detection and recognition model, which eliminates the need for manual sample labeling. It can autonomously collect and analyze target data in different scenarios, autonomously optimize the parameters of the detection and recognition model, and achieve dynamic upgrades in recognition capabilities. This effectively solves the problems of poor adaptability and the need for manual maintenance in existing supervised learning detection and recognition models, significantly improves the intelligence level of the equipment and its adaptability to complex environments, and ensures that the equipment can maintain high recognition accuracy in different scenarios.

[0033] 5. Deep collaboration between software platform, hardware integration, and unsupervised self-learning detection and recognition models enables full automation of video data from acquisition, preprocessing, recognition, tracking to transmission, reducing manual intervention, improving management efficiency, and ensuring that real-time optimization of the detection and recognition model does not affect the normal operation of the equipment, further enhancing the practicality and reliability of the equipment. Attached Figure Description

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0035] Figure 1 A schematic diagram of an optoelectronic observation device for controlling unmanned aerial vehicles provided in the first embodiment of the present invention.

[0036] Figure 2 for Figure 1 The diagram shown is a block diagram of the optoelectronic observation equipment used to control drones.

[0037] Figure 3 for Figure 1 The diagram shows the workflow of the optoelectronic observation equipment used to control drones.

[0038] Figure 4 for Figure 1The diagram shown is an information flow diagram of the optoelectronic observation equipment used to control drones.

[0039] Figure 5 for Figure 1 The image shown is a schematic diagram of an electro-optical observation device used to control drones.

[0040] Figure 6 The flowchart illustrates the process of optimizing the detection and recognition model using an unsupervised self-learning software module provided in the first embodiment of the present invention.

[0041] The attached figures are labeled as follows: 1. Infrared thermal imager; 2. Embedded information processing module; 3. Visible light camera; 4. Laser rangefinder; 5. Servo turntable; 51. Azimuth rotation mechanism; 52. Pitch mechanism; 6. Power supply unit. Detailed Implementation

[0042] Example 1

[0043] In related technologies, the software and hardware of UAV optoelectronic observation equipment are designed separately, with each hardware module having its own software. This results in poor coordination, triggering a series of chain reactions that ultimately severely impact the real-time performance, reliability, accuracy, intelligence level, and environmental adaptability of UAV control missions. Specifically, it further leads to the following problems:

[0044] 1. Significantly increased system response latency leads to "seeing but not keeping up": Due to the separate design of software and hardware, after the visible light / infrared camera acquires the target image, it needs to undergo preprocessing, such as format conversion, by its own software before being transmitted to the embedded information processing module through a non-standard interface. After processing, the embedded information processing module transmits control commands to the servo turntable software through another non-standard interface. The servo turntable software then drives the motor. Each stage's "built-in software" has its own data format and communication protocol. Data needs to be repeatedly packaged, unpacked, and converted between these "built-in software" components. For high-speed, maneuverable small drones, this millisecond-level latency accumulation causes the servo turntable to be perpetually "chasing" the target's trail, unable to achieve high-precision locking, and the target is easily lost.

[0045] 2. Inability to achieve global intelligence: Advanced intelligent algorithms need to acquire and process real-time data from all modules simultaneously. However, in an architecture with poor coordination, it is difficult to unify and align the temporal and spatial benchmarks of the data from each module. The models inside the device are trained based on the data collected in the early stage and cannot be adaptively optimized according to the scenario in which the device is deployed. Moreover, they lack self-learning capabilities.

[0046] Therefore, this embodiment provides an optoelectronic observation device for controlling drones. In use, it is installed on the ground to control aerial drones. See also... Figure 1 , Figure 2 and Figure 5 The optoelectronic observation equipment includes three front-end acquisition units, an embedded information processing module 2, a servo turntable 5, and a power supply unit 6. The three front-end acquisition units are an infrared thermal imager 1, a visible light camera 3, and a laser rangefinder 4. The three front-end acquisition units are mounted on the servo turntable 5 to rotate along the azimuth and pitch axes and track the target under the drive of the servo turntable 5.

[0047] See Figure 4 Both the infrared thermal imager 1 and the visible light camera 3 are used to acquire scene and target data within the field of view. The laser rangefinder 4 is used to obtain target distance information. The servo turntable 5 is used to execute position control commands sent by the embedded information processing module 2 to achieve turning or tracking. The embedded information processing module 2 is used to interact with the control and management drone platform, mainly including receiving control commands and guidance data from the control and management drone platform, and reporting target 3D coordinate information and equipment status information. Moreover, as the central brain of the device, the embedded information processing module 2 is also used to preprocess the raw video images acquired by the infrared thermal imager 1 and the visible light camera 3, control the infrared thermal imager 1 and the visible light camera 3, control the laser rangefinder 4, control the servo turntable 5, perform unsupervised self-learning of target recognition, tracking, and detection recognition models, and collect status information of each module.

[0048] The power supply unit 6 is connected to the infrared thermal imager 1, the embedded information processing module 2, the visible light camera 3, the laser rangefinder 4, and the servo turntable 5, respectively, to provide the required operating current to each module.

[0049] The infrared thermal imager 1, the visible light camera 3, the laser rangefinder 4, and the servo turntable 5 are all directly connected to the embedded information processing module 2 to form an integrated hardware structure.

[0050] Embedded information processing module 2 serves as the core control unit of the integrated hardware structure and runs on a shared software platform. In other words, embedded information processing module 2 is the core hub for the collaboration between optoelectronic software and hardware. The shared software platform adopts a modular design and includes: an infrared thermal imager control software module, a visible light camera control software module, and a laser rangefinder control software module, each used to control its corresponding hardware module to acquire raw video image data and perform preprocessing; a UAV target recognition and tracking software module, used to acquire preprocessed video image data, call the built-in detection and recognition model to perform target detection, tracking, and recognition on the video image data, and output the target's three-dimensional coordinate information; a servo turntable control software module, used to control the servo turntable 5 to track the target based on the target's three-dimensional coordinate information; a comprehensive processing software module, used to interact with the UAV management platform, control the aforementioned modules, and collect the status information of each module; and an unsupervised self-learning software module, used to iteratively optimize the detection and recognition model when the device is idle, and deploy the optimized detection and recognition model to the UAV target recognition and tracking software module to improve the subsequent accuracy of UAV target recognition and adaptability in complex environments.

[0051] Each software module is adapted to its corresponding hardware module to achieve unified scheduling and functional coordination of hardware resources.

[0052] The infrared thermal imager control software module, the visible light camera control software module, and the laser rangefinder control software module are used to control the corresponding hardware modules to acquire raw video image data and perform preprocessing. Specifically, the infrared thermal imager control software module corresponds to infrared thermal imager 1. The visible light camera control software module corresponds to visible light camera 3. The laser rangefinder control software module corresponds to laser rangefinder 4.

[0053] In this application, "equipment idle" means when the equipment is powered on but not performing any tasks.

[0054] This application significantly improves the reliability, efficiency, and intelligence of drone management tasks through the deep integration of a shared software platform, integrated hardware, and unsupervised self-learning detection and recognition models.

[0055] Infrared thermal imager 1, visible light camera 3, and laser rangefinder 4, as front-end acquisition units of the integrated hardware architecture, all transmit their respective raw video image data to the embedded information processing module 2 in real time.

[0056] See Figure 4The shared software platform adopts a standardized interface design, supports multi-version compatibility and functional plugin expansion, and can flexibly add new recognition algorithms or drone control functions according to actual needs. The platform includes modules for infrared thermal imager control, visible light camera control, laser rangefinder control, servo turntable control, drone target recognition and tracking, integrated processing, and unsupervised self-learning. If new control functions are needed in the future, only the corresponding software module plugins need to be added; no changes to the hardware structure are required, reducing equipment upgrade and maintenance costs.

[0057] The servo turntable in this embodiment includes an azimuth rotation mechanism 51 and a pitch mechanism 52 disposed on top of the azimuth rotation mechanism. The azimuth rotation mechanism 51 includes an azimuth frame, an azimuth driver, an azimuth motor, an azimuth encoder, and an azimuth gyroscope. The pitch mechanism 52 includes a pitch frame, a pitch driver, a pitch motor, a pitch encoder, and a pitch gyroscope. The plurality of front-end acquisition units and the embedded information processing module 2 are supported on the pitch frame. The azimuth encoder and the pitch encoder are used to acquire the rotation angle information of the servo turntable 5 in real time. The azimuth gyroscope and the pitch gyroscope are used to acquire the angular velocity information of the servo turntable 5 in real time, and feed the rotation angle information and angular velocity information back to the servo turntable control software module. The servo turntable control software module compares and calibrates the feedback information with the received control commands, generates corresponding azimuth control signals and pitch control signals, and sends them to the azimuth driver and the pitch driver, respectively. The azimuth driver is connected to the azimuth motor and is used to drive the azimuth frame to rotate around the azimuth axis. The pitch driver is connected to the pitch motor and is used to drive the pitch frame to rotate around the pitch axis, thereby forming a closed-loop control of the servo turntable 5.

[0058] Specifically, the infrared thermal imager 1 includes at least an infrared lens and an infrared detector, and the visible light camera 3 includes at least a visible light lens and a visible light detector. The infrared thermal imager control software module primarily controls the infrared lens and receives status feedback from it, acquiring and preprocessing the raw video images collected by the infrared detector. The visible light camera control software module primarily controls the visible light lens, receives status feedback from it, acquires and preprocesses the raw video images collected by the visible light detector. The laser rangefinder control software module receives target distance information and status information reported by the laser rangefinder 4. The servo turntable control software module issues steering commands to the servo turntable 5 to track the target, receives rotation angle and angular velocity information feedback from the servo turntable 5, and achieves closed-loop control. The UAV target recognition and tracking software module acquires preprocessed video data, uses a built-in detection and recognition model to detect, track, and recognize UAV targets, and outputs the target's three-dimensional coordinate information. The unsupervised self-learning software module iteratively optimizes the detection and recognition model, optimizing iteratively when the device is idle, and deploys the optimized detection and recognition model to the UAV target recognition and tracking software module. The integrated processing software module interacts with the UAV management platform, issues control commands to each module, and collects status information. The modules here include an infrared thermal imager 1, a visible light camera 3, a laser rangefinder 4, an embedded information processing module 2, and a servo turntable 5.

[0059] Figure 3This embodiment illustrates the workflow of the photoelectric observation equipment for controlling unmanned aerial vehicles (UAVs). The workflow mainly consists of system self-check, unsupervised self-learning, target search, target confirmation, and target processing. The system self-check specifically involves: after powering on, the system checks for normal operation. If normal, it reports self-check completion and proceeds to the unsupervised self-learning phase; if abnormal, it reports a fault, performs system maintenance, powers off, and then powers on again. The unsupervised self-learning phase involves: checking for tasks. If tasks are present, it proceeds to the target search phase; if no tasks are present, it begins self-learning, iteratively updating the detection and recognition model. Simultaneously, it checks for tasks in real-time; if tasks are present, it immediately proceeds to the target search phase. The target search phase is as follows: First, a mode selection is made, which includes manual search, automatic search, and external guidance mode. In manual search, if a target is found, the system proceeds to the target confirmation phase to determine if it is a task target. If it is, the system proceeds to the target processing phase; otherwise, it returns to the target search phase to continue the manual search. In automatic search mode, if a target is found, the system proceeds to the target confirmation phase to determine if it is a task target. If it is, the system proceeds to the target processing phase; otherwise, it returns to the target search phase to continue the automatic search. In external guidance mode, if a target is found, the system proceeds to the target confirmation phase to determine if it matches the guidance. If it matches, the system proceeds to the target processing phase; otherwise, it continues to receive external guidance information and continues searching for targets. In the target processing phase, target tracking is performed, video recording for evidence is initiated, and target information is reported, thus completing the task.

[0060] See Figure 4The information interaction between the optoelectronic observation equipment for controlling drones and the drone control platform provided in this embodiment is as follows: The drone control platform sends guidance and control commands to the integrated processing software module of the embedded information processing module 2 of the optoelectronic observation equipment, and simultaneously receives target information, video data, and equipment status information sent by the integrated processing software module of the embedded information processing module 2 of the optoelectronic observation equipment. The integrated processing software module internally includes a communication service component, an optoelectronic management component, a data storage and service component, an information fusion component, and a threat assessment component: The communication service component is used to interact with the drone control platform, receive the issued guidance and control commands, and send the reported target information, video data, and equipment status information to the control platform; the optoelectronic management component is used to distribute the control commands received by the communication service component to the corresponding software modules and collect the status information returned by each software module, summarize it, and then report it to the communication service component; the data storage and service component is used to receive video data collected by the infrared thermal imager and the visible light camera, encode and compress it, store it, and push the video data to the drone control platform through the communication service component. The information fusion component is used to fuse target data from visible light video, infrared video, and guidance data to form comprehensive intelligence, which is then reported to the drone management platform; the threat assessment component is used to assess the degree of threat posed by detected targets to the defensive position. For example... Figure 4 As shown in the information flow path, the above components work together to enable the issuance of instructions, data reporting, and video transmission between the photoelectric observation equipment and the control platform.

[0061] See Figure 4The infrared identification channel information flow of the photoelectric observation equipment for controlling drones provided by the present invention is as follows: the original video collected by the infrared lens and detector of the infrared thermal imager 1 is sent to the infrared image preprocessing component in the infrared thermal imager control software module of the embedded information processing module 2. The infrared image preprocessing component performs preprocessing operations such as image enhancement and sharpening on the original video to form a visual image. At the same time, the infrared thermal imager 1 is controlled by the infrared integrated processing component of the infrared thermal imager control software module and sends its own status information to the component. The pre-processed video is sent to the target recognition and tracking software module on the same platform. The target recognition and tracking software module encodes and compresses the pre-processed video and then transfers it to the data storage and service components in the embedded information processing module 2. On the other hand, it performs infrared target detection on the pre-processed video. If no target information is obtained, the detection continues; if target information is obtained, target tracking and target recognition are performed. At the same time, the target is tracked and located, the miss distance is calculated, and a miss distance command is sent to the servo turntable control software module in the embedded information processing module 2. The servo turntable control software module controls the azimuth driver and pitch driver of the servo turntable 5 according to the miss distance command. The azimuth driver of the servo turntable 5 receives the control command from the servo turntable control software module and drives the azimuth motor to rotate. At the same time, the azimuth encoder and azimuth gyroscope feed back the real-time acquired azimuth angle and azimuth angular velocity to the servo turntable control software module in the embedded information processing module 2. The servo turntable control software module realizes azimuth tracking in real time. The pitch driver of servo turntable 5 receives control commands from the servo turntable control software module of embedded information processing module 2, driving the pitch motor to rotate. At the same time, the pitch encoder and pitch gyroscope feed back the collected pitch angle and pitch angular velocity to the servo turntable control software module of embedded information processing module 2. The servo turntable control software module realizes pitch tracking in real time. When tracking the target in real time, the tracking and positioning component of the target recognition and tracking software module sends a ranging command to the laser rangefinder control software module. The laser rangefinder control software module sends a command to the transmitting component of laser rangefinder 4 and receives the return value from the receiving component of laser rangefinder 4. By calculating the distance information between the target and the photoelectric observation device, the three-dimensional coordinate information of the target is parsed out. The three-dimensional coordinate information of the target is sent to the integrated processing software module for information fusion and threat assessment. The photoelectric management component of the integrated management software module reports the target information to the control and management UAV platform through the communication service component. Specifically, the integrated processing software module receives target type information identified by a visible light camera or infrared thermal imager, target azimuth and elevation angle information determined by the servo module, and target distance information obtained by the ranging module. It then fuses the above information to form complete target information that includes target category, direction, and distance. Based on the complete target information, it assesses the threat level of the target to determine whether it is a target that requires priority handling.When a target is identified as a high-threat target, it can be reported immediately or prioritized for handling.

[0062] See Figure 4 The visible light recognition channel information flow of the photoelectric observation device for controlling drones provided by this invention is basically the same as that of the infrared recognition channel, the difference being the original video source and processing module, as detailed below:

[0063] The visible light camera 3's visible light lens and detector capture raw video, which is then sent to the visible light image preprocessing component in the visible light camera control software module of the embedded information processing module 2. The visible light image preprocessing component performs preprocessing operations such as image enhancement and sharpening on the raw video to form a visual image. Simultaneously, the visible light camera 3 is controlled by the visible light integrated processing component of the visible light camera control software module and sends its own status information to this component.

[0064] The pre-processed video is sent to the target recognition and tracking software module on the same platform. The subsequent encoding and compression, target detection, tracking and positioning, servo closed-loop tracking, laser ranging, information fusion and reporting processes are exactly the same as those of the infrared recognition channel, and will not be described in detail here.

[0065] See Figure 4The infrared and visible light dual-channel identification information flow of the optoelectronic observation equipment for controlling drones provided by the present invention is as follows: the infrared thermal imager 1 and the visible light camera 3 simultaneously send their respective collected raw videos to their respective processing software in the embedded information processing module 2. After the visible light image preprocessing component of the visible light camera software in the embedded information processing module 2 completes the visible light video preprocessing, it is handed over to the target recognition and tracking software on the same platform for encoding and compression, and then transferred to the data storage and service in the embedded information processing module 2. At the same time, visible light target detection is performed. Meanwhile, after the infrared image preprocessing component of the infrared software in the embedded information processing module 2 completes the infrared video preprocessing, it is handed over to the target recognition and tracking software on the same platform for encoding and compression, and then transferred to the data storage and service in the embedded information processing module 2. At the same time, infrared target detection is performed. If target information is obtained, target tracking and target recognition are performed. Simultaneously, the target is tracked and located, and a miss distance command is sent to the servo control software in the embedded information processing module 2. The azimuth driver of the servo turntable 5 receives the control command from the servo control software of the embedded information processing module 2 and drives the azimuth motor to rotate. At the same time, the azimuth encoder and azimuth gyroscope feed back the collected azimuth angle and azimuth angular velocity to the servo control software of the embedded information processing module 2. The servo control software realizes azimuth tracking in real time. The pitch driver of the servo turntable 5 receives control commands from the servo control software of the embedded information processing module 2, driving the pitch motor to rotate. At the same time, the pitch encoder and pitch gyroscope feed back the collected pitch angle and pitch angular velocity to the servo control software of the embedded information processing module 2. The servo control software realizes pitch tracking in real time. When tracking the target in real time, the tracking and positioning component sends a ranging command to the laser rangefinder control software module of the embedded information processing module 2. The laser rangefinder control software module sends a command to the transmitting module of the laser rangefinder 4, and at the same time receives the return value from the receiving component of the laser rangefinder 4. By calculating, the distance information between the target and the photoelectric observation device is obtained, realizing the collection of the target's three-dimensional coordinate information. This information is sent to the integrated processing software of the embedded information processing module 2 for information fusion and threat assessment, and the target information is reported to the control and management UAV platform through the photoelectric management of the integrated processing software module of the embedded information processing module 2.

[0066] Figure 6 This diagram illustrates the workflow of the unsupervised self-learning software module iteratively optimizing the detection and recognition model in this embodiment. See also... Figure 6The iterative optimization of the detection and recognition model includes: automatically collecting data from both historical and real-time video data; automatically extracting multi-dimensional features of the target data without manual sample labeling. These multi-dimensional features include the drone's motion characteristics, visible light optical characteristics, and infrared characteristics, with motion features including at least the drone's outline, flight speed, and flight trajectory; performing unsupervised clustering on the extracted multi-dimensional features to autonomously distinguish drone targets from other targets; gradually improving the target discrimination accuracy by iteratively optimizing the threshold, thus generating a new detection and recognition model; comparing the new detection and recognition model with the old one; if the new model is superior, it is autonomously deployed to the drone target recognition and tracking software module to achieve dynamic upgrades in recognition capabilities; if the new model is inferior, data collection is restarted. The unsupervised self-learning software module runs when the photoelectric observation equipment is powered on but not performing tasks, ensuring that the equipment can perform normal tasks without affecting its operation, while fully utilizing the hardware resources of the photoelectric observation equipment. The specific form of autonomously deploying the new detection and recognition model to the drone target recognition and tracking software module can be by directly updating the detection and recognition model in the existing software module.

[0067] This invention provides a concept and method for an optoelectronic observation device for controlling unmanned aerial vehicles (UAVs). Many methods and approaches exist for implementing this technical solution; the above are merely preferred embodiments. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A photoelectric observation device for controlling unmanned aerial vehicles (UAVs), characterized in that, It includes an embedded information processing module (2) and multiple front-end acquisition units and servo turntables (5) that are directly connected to the embedded information processing module (2) to form a hardware integrated structure. The embedded information processing module (2) serves as the core control unit of the integrated hardware structure and runs on a shared software platform; The software uses a shared platform and adopts a modular design, including: Each front-end acquisition unit control software module is used to control the corresponding front-end acquisition unit to acquire and preprocess video image data; The UAV target recognition and tracking software module is used to acquire preprocessed video image data, call the built-in detection and recognition model to perform target detection, tracking and recognition on the video image data, and output the target's three-dimensional coordinate information; The servo turntable control software module is used to control the servo turntable (5) to drive the multiple front-end acquisition units to rotate and track the target according to the target's three-dimensional coordinate information; The integrated processing software module is used to interact with the drone management platform, issue control commands to the aforementioned control software modules, and collect status information from each module. The system also includes an unsupervised self-learning software module, which iteratively optimizes the detection and recognition model when the device is idle, and deploys the optimized detection and recognition model to the UAV target recognition and tracking software module to improve the accuracy of subsequent UAV target recognition and adaptability in complex environments.

2. The photoelectric observation device for controlling unmanned aerial vehicles according to claim 1, characterized in that, The multiple front-end acquisition units include an infrared thermal imager (1), a visible light camera (3), and a laser rangefinder (4). Each of the multiple front-end acquisition units transmits its own acquired raw video image data to the embedded information processing module (2) in real time.

3. The photoelectric observation device for controlling unmanned aerial vehicles according to claim 1, characterized in that, The integrated hardware structure also includes a power supply unit (6), which is used to supply power to the integrated hardware structure.

4. The photoelectric observation device for controlling unmanned aerial vehicles according to claim 1, characterized in that, The servo turntable (5) includes an azimuth rotation mechanism (51) and a pitch mechanism (52) disposed on the top of the azimuth rotation mechanism. The multiple front-end acquisition units and the embedded information processing module (2) are carried on the pitch mechanism (52).

5. The photoelectric observation device for controlling unmanned aerial vehicles according to claim 4, characterized in that, The azimuth rotation mechanism (51) includes an azimuth frame, an azimuth driver, an azimuth motor, an azimuth encoder, and an azimuth gyroscope. The pitch mechanism (52) includes a pitch frame, a pitch driver, a pitch motor, a pitch encoder, and a pitch gyroscope. The multiple front-end acquisition units and the embedded information processing module (2) are carried on the pitch frame. The azimuth encoder and the pitch encoder are used to collect the rotation angle information of the servo turntable (5) in real time. The azimuth gyroscope and the pitch gyroscope are used to collect the angular velocity information of the servo turntable (5) in real time, and feed the rotation angle information and angular velocity information back to the servo turntable control software module. The servo turntable control software module compares and calibrates the feedback information with the received control command, generates the corresponding azimuth control signal and pitch control signal, and sends them to the azimuth driver and the pitch driver respectively. The azimuth driver is connected to the azimuth motor and is used to drive the azimuth frame to rotate around the azimuth axis. The pitch driver is connected to the pitch motor and is used to drive the pitch frame to rotate around the pitch axis, thereby forming a closed-loop control of the servo turntable (5).

6. The photoelectric observation device for controlling unmanned aerial vehicles according to claim 1, characterized in that, The software platform adopts a standardized interface design, supports multi-version compatibility and functional plugin expansion, and can flexibly add new recognition algorithms or drone control functions according to actual needs.

7. The photoelectric observation device for controlling unmanned aerial vehicles according to claim 1, characterized in that, The iterative optimization of the detection and recognition model includes: Simultaneously, data is automatically collected from historical and real-time video data without the need for manual sample labeling, and multi-dimensional features are automatically extracted. The multi-dimensional features include the drone's motion features, visible light optical features, and infrared features, wherein the motion features include at least the drone's shape outline, flight speed, and flight trajectory. Unsupervised clustering is performed on the extracted multidimensional features to autonomously distinguish UAV targets from other targets. By iteratively optimizing the threshold, a new detection and recognition model is generated. The new detection and recognition model is compared with the old detection and recognition model. If the new detection and recognition model is better than the old one, it is automatically deployed to the UAV target recognition and tracking software module. If it is worse than the old one, the data collection process is restarted.

8. The photoelectric observation device for controlling unmanned aerial vehicles according to claim 2, characterized in that, The integrated processing software module includes a communication service component, an optoelectronic management component, a data storage and service component, an information fusion component, and a threat assessment component.

9. The photoelectric observation device for controlling unmanned aerial vehicles according to claim 8, characterized in that, The communication service component is used to interact with the control drone platform, receive guidance and control commands, and send the reported target information, video data, and device status information to the control platform. The photoelectric management component is used to distribute the control commands received by the communication service component to the corresponding software modules and collect the status information returned by each software module. After summarizing, it is reported by the communication service component. The data storage and service component is used to receive video data collected by the infrared thermal imager and the visible light camera, encode and compress it, store it, and push the video data to the control drone platform through the communication service component.

10. The photoelectric observation device for controlling unmanned aerial vehicles according to claim 8, characterized in that, The information fusion component is used to fuse target data from visible light video, target data from infrared video, and guidance data to form comprehensive intelligence, which is then reported to the drone management platform. The threat assessment component is used to assess the degree of threat posed by detected targets to the defensive position.

Images