Photoelectric system integrated management method and system

Abstract

The invention discloses a photoelectric system integrated management method and system. The system comprises a photoelectric subsystem and an integrated processing subsystem. The photoelectric subsystem adopts a receiving aperture comprehensive configuration; the comprehensive processing subsystem comprises a system management module, an intelligent processing module and a navigation module; the photoelectric subsystem comprises a laser transmitting module with an independent transmitting aperture and a common-aperture receiving assembly; the common-caliber receiving assembly comprises an infrared imaging module, a laser receiving module and a light spot tracking module; and data interaction between the system management module and the intelligent processing module is realized through driving of a RapidIO interface, a PCIE interface and an RS422 interface. The photoelectric system integrated management system has a highly integrated hardware architecture, real-time software driving management and a real-time low-delay data transmission architecture. Wide application prospects are realized.

Description

Technical Field

[0001] The present invention relates to the field of optoelectronic technology, and particularly to a method and system for comprehensive management of an optoelectronic system. Background Art

[0002] Optoelectronic systems are widely used in the fields of military, aviation, aerospace, etc. for tasks such as target search, recognition, tracking, ranging, positioning, and illumination. Traditional optoelectronic systems include an infrared imaging module, a laser ranging module, a spot tracking module, etc. These modules work independently through their respective processors and control units, and data interaction and collaborative work between systems rely on complex interfaces and communication protocols.

[0003] On the one hand, the above-mentioned traditional optoelectronic systems usually adopt a distributed architecture, and data is exchanged between each module through serial communication, Ethernet, or other communication protocols, resulting in a large data transmission delay and affecting the real-time performance of the system. On the other hand, the system management module is responsible for coordinating the work of each module. However, due to the independent operation of each module, it is difficult for the system management module to efficiently coordinate the work of each module, resulting in a low overall collaborative work efficiency of the system. In addition, there are also defects such as poor scalability, high power consumption, and large volume. For example, the hardware and software architectures of traditional systems are relatively fixed and difficult to be flexibly expanded and upgraded according to task requirements. Traditional systems usually adopt multiple independent processors and control units, which are difficult to meet the requirements of lightweight and low power consumption for modern platforms such as unmanned aerial vehicles. Summary of the Invention

[0004] The purpose of the present invention is to propose a method and system for comprehensive management of an optoelectronic system to solve the problems of communication delay, low collaborative work efficiency, poor scalability, high power consumption, and large volume in the prior art.

[0005] The present invention provides an optoelectronic system comprehensive management system, including an optoelectronic subsystem and a comprehensive processing subsystem; the optoelectronic subsystem adopts a receiving aperture synthesis configuration; the comprehensive processing subsystem includes a system management module, an intelligent processing module, and a navigation module; The optoelectronic subsystem includes a laser emission module with an independent emission aperture and a common aperture receiving assembly; the common aperture receiving assembly includes an infrared imaging module, a laser receiving module, and a spot tracking module; Data interaction between the system management module and the intelligent processing module is driven and implemented through RapidIO interfaces, PCIE interfaces, and RS422 interfaces; The system management module is used to implement one or more of system management, sensor management, data communication, sensor interface management, servo control, and spot tracking algorithms; the intelligent processing module is used to implement one or more of image processing, AI target recognition, map matching, video compression, image storage, and image transmission; the navigation module is cross-linked with the aircraft inertial navigation and satellite communication antenna and is used to generate aircraft position, aircraft attitude, sensor attitude, and time information data for the system management module and the intelligent processing module to call.

[0006] The optoelectronic sub-system is provided with an independent laser emission aperture of not less than 50 mm and an independent visible light receiving aperture of 30 mm.

[0007] Furthermore, the common aperture receiving component of the optoelectronic system integrated management system of the present invention includes a Cassegrain antenna with an optical synthetic aperture in various combination modes; the combination modes include any combination of two or more of short-wave & medium-wave & long-wave infrared, visible light, and laser; light of each wavelength is separated by a dichroic and beam splitter and then enters different detector targets for signal modulation and signal filtering.

[0008] The optoelectronic sub-system is provided with a common aperture receiving component of not less than 150 mm, and a beam splitter and dichroic filter are used between the infrared imaging module, the laser receiving module, and the spot tracking module of the common aperture receiving component to ensure separate optical transmission paths for each wavelength.

[0009] Furthermore, the integrated processing sub-system of the optoelectronic system integrated management system of the present invention realizes video acquisition and preprocessing through the FPGA of the intelligent processing module, transmits video data to the intelligent processing module through the PCIe interface, distributes compressed video by using an Ethernet switch, the intelligent processing module supports collaborative processing of high-computing algorithms by at least two intelligent processing chips, and video data transmission between the intelligent processing chips of the intelligent processing module and the FPGA of the intelligent processing module is realized through PCIE interface driving.

[0010] Furthermore, the video processing process in the integrated information processing sub-system of the optoelectronic system integrated management system of the present invention includes: The infrared light signal is transmitted to the FPGA through an optoelectronic transceiver based on the SRIO protocol, and the FPGA analyzes and preprocesses the data according to the SRIO protocol and then transmits it to the intelligent processing chip through the PCIE2.0 4X interface for processing; The white light AVT signal enters the FPGA through the AVT protocol and a video receiving chip, and the FPGA analyzes and preprocesses the data according to the AVT protocol and then transmits it to the intelligent processing chip through the PCIE2.0 4X interface for processing; Dedicated video transmission channel: The infrared video processed by the intelligent processing chip is transmitted to another intelligent processing chip via MIPI; The compressed video processed by the intelligent processing chip is distributed to 6 Gigabit Ethernet outputs through an Ethernet switch.

[0011] Furthermore, the system management module of the optoelectronic system integrated management system of the present invention includes a dual-core CPU unit, an FPGA unit capable of realizing multi-source data fusion, a 289A bus module, and a storage unit; The first processor of the dual-core CPU unit is responsible for management tasks and algorithm tasks, and the second processor is responsible for communication tasks. The FPGA unit is connected to the CPU unit through an LBC interface, and the FPGA unit is externally connected to a storage unit; the FPGA unit extends multiple RS422 interfaces to communicate with the intelligent processing module, and converges through the LBC bus to the dual-core CPU unit to implement system management, sensor management, data communication, sensor interface management, servo control, and spot tracking algorithm of the system management module; the dual-core CPU unit controls the 289A bus module through a PCIE bus to communicate with the mission computer.

[0012] Furthermore, the navigation module of the optoelectronic system integrated management system of the present invention includes a navigation board and a navigation sub-card, a navigation sub-card 2, and a self-destruct memory card mounted on the navigation board. The navigation sub-card and the navigation sub-card 2 are cross-linked with the aircraft inertial navigation and satellite communication antenna to generate data such as the position of the aircraft, the attitude of the aircraft, the attitude of the sensor, and time information for use by the main control function and the image function.

[0013] Furthermore, the CPU of the system management module of the optoelectronic system integrated management system of the present invention runs the domestic operating system DeltaSVM with partitioned channels, The operating system DeltaSVM includes a secure virtual machine kernel and a core-level board support package, runs in the core state of the processor, and realizes isolation and scheduling of system resources through a hardware privilege management unit; The operating system DeltaSVM also includes a runtime library, configured to be callable in both the system state and the user state, and realizes dynamic memory allocation through a memory management unit; The operating system DeltaSVM also includes a partition-level board support package and an aircraft software operating environment component, runs in the user state, and restricts direct access to core state resources through a privilege isolation mechanism; The operating system DeltaSVM also includes a processor architecture that supports multi-level privilege state switching, including the core state, the system state, and the user state; The secure virtual machine kernel manages real-time task scheduling through a hardware interrupt controller; The core board support package interacts with physical peripheral drivers through register mapping; The runtime library provides a standardized interface across privilege levels for user-space applications to call system-level functions.

[0014] Furthermore, the processor of the intelligent processing module of the optoelectronic system integrated management system of the present invention runs the soft real-time operating system openEuler-22.03. The soft real-time operating system openEuler-22.03 interacts with the hardware through a layered architecture, and the core depends on the kernel driver, the hardware abstraction interface, and the system call to achieve seamless connection from physical devices to user programs.

[0015] Furthermore, the outer envelope of the optoelectronic subsystem of the optoelectronic system integrated management system of the present invention is an optical window assembly; the optical window assembly is composed of a multi-spectral composite material with high light transmittance and an impact-resistant support structure. The edge of the optical window is coated with a lightweight metal / composite material, which not only ensures the overall structural strength but also reduces the reflectance through surface treatment technology, improving concealment and environmental adaptability.

[0016] Furthermore, the integrated processing subsystem of the optoelectronic system integrated management system of the present invention is installed in a fixed frame and does not rotate with the servo system, and the optoelectronic payloads are all placed on the pitch axis system.

[0017] An optoelectronic system integrated management method based on the optoelectronic system integrated management system described in any one of the above, specifically includes the following steps: S1: Construct a highly integrated hardware architecture, including constructing an optoelectronic subsystem using a receiving aperture synthesis configuration and setting an integrated processing subsystem, and the setting of the integrated processing subsystem includes an integrated system management module, an intelligent processing module, and a navigation module; S2: Implement real-time software driver management, including deploying the operating system DeltaSVM with partitioned channels in the system management module and deploying the soft real-time operating system openEuler - 22.03 in the intelligent processing module; S3: Establish a real-time low-latency data communication architecture, implement video capture and preprocessing through an FPGA, and transmit video data to the intelligent processing module using a PCIe interface; use an Ethernet switch to distribute compressed video, supporting the transmission of multiple 100-Mbps Ethernet signals; S4: Modular design, where data is exchanged between modules through standard interfaces, supporting the collaborative processing of high-computation algorithms by dual intelligent processing modules; S5: Low-power management, using low-power processors and dynamic power management strategies to reduce the overall power consumption of the system.

[0018] The beneficial effects of the optoelectronic system integrated management method and system of the present invention are as follows: First, through the receiving aperture synthesis configuration and multi-module hardware integration design, the present invention significantly optimizes the defects of the distributed architecture of traditional optoelectronic systems. The optoelectronic sub-system adopts a common aperture receiving component and an independent transmitting aperture design, and combines dichroic / spectroscopic technology to achieve efficient separation and parallel processing of multi-spectral (short-wave / mid-wave / long-wave infrared, visible light, laser) signals, solving the problems of resource redundancy and low cooperation efficiency caused by the independent operation of multiple modules in traditional systems. The integrated processing sub-system realizes real-time data interaction between the system management module and the intelligent processing module through high-speed interfaces such as RapidIO, PCIe, and RS422. Combining a dual-core CPU unit (task management type and communication type), an FPGA multi-source data fusion architecture, and the collaborative operation of two intelligent processing chips, the algorithm processing delay is reduced to the millisecond level, greatly improving the response speed of the overall system. Moreover, the hardware architecture adopts a design that separates the fixed frame from the pitch axis system, reducing the interference of servo rotation on the core processing unit and further ensuring real-time performance and stability.

[0019] Secondly, through standard interfaces and modular design, the present invention also realizes highly flexible expansion of the functions of the optoelectronic system. The common aperture receiving component supports any combination of multi-band optical synthetic apertures (such as Cassegrain antennas) and can adapt to the spectral coverage range required for different tasks (such as short-wave infrared + laser or visible light + long-wave infrared). The intelligent processing module constructs a distributed video processing channel based on the PCIe 2.0 4X interface and an Ethernet switch, supports parallel compression, transmission, and AI target recognition of multiple high-resolution infrared / white light videos, and at the same time, through the computing power cooperation of two intelligent processing chips, meets the real-time computing requirements of complex algorithms (such as AI target recognition and map matching). The system management module uses the domestic Tao partition operating system DeltaSVM. Through a three-level permission isolation mechanism of the core state, system state, and user state and a dynamic hardware resource scheduling mechanism, it ensures the strong real-time performance of key tasks (such as servo control and spot tracking), and at the same time is compatible with multi-source sensor interfaces (such as 289A bus and RS422), significantly improving the task adaptability and upgradeability of the system in scenarios such as reconnaissance and UAV navigation.

[0020] In addition, the optoelectronic system integrated management system of the present invention also has the advantages of low-power management and concealment optimization. On the one hand, through lightweight materials and dynamic power management strategies, the present invention effectively reduces the overall power consumption and physical volume of the system. The optical window assembly uses a multi-spectral composite material with a high light transmittance (≥60% in the visible light band) and an impact-resistant support structure, combined with edge lightweight metal / composite material cladding technology, to reduce the weight while ensuring the structural strength. The reflectance of the optical window is ≤1.5% (visible light band) and ≤0.8% (laser / infrared band), significantly improving the concealment and anti-detection ability in specific environments. On the other hand, the system internally uses a low-power processor and an intelligent power management module, and realizes power reduction through dynamic task load allocation (such as CPU dual-core time-sharing processing of management and communication tasks). In addition, the deep cross-linking of the navigation module with the aircraft inertial navigation / satellite communication antenna, combined with the self-destructing memory card design, further enhances the reliability and data security of the system in high-risk tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is the architecture diagram of the optoelectronic sub-system of the optoelectronic system integrated management system of the present invention; Figure 2 is the schematic diagram of the main data flow of the integrated information processing sub-system of the present invention; Figure 3 is the architecture diagram of the 2DeltaSVM of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] In order to have a clearer understanding of the technical features, objectives, and effects of the present invention, the specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention. Those not specified in the specific embodiments are carried out according to conventional conditions or conditions provided by the manufacturer.

[0023] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present disclosure. The terms used in the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS 1:

[0024] An optoelectronic system integrated management system includes an optoelectronic sub-system and an integrated processing sub-system; the optoelectronic sub-system adopts a receiving aperture integrated configuration; the integrated processing sub-system includes a system management module, an intelligent processing module, and a navigation module; The optoelectronic sub-system includes a laser emission module with an independent emission aperture and a co-aperture receiving assembly; the co-aperture receiving assembly includes an infrared imaging module, a laser receiving module, and a spot tracking module; The data interaction between the system management module and the intelligent processing module is driven and realized through RapidIO interfaces, PCIE interfaces, and RS422 interfaces; The system management module is used to implement one or more of system management, sensor management, data communication, sensor interface management, servo control, and spot tracking algorithms; the intelligent processing module is used to implement one or more of image processing, AI target recognition, map matching, video compression, image storage, and image transmission; the navigation module is cross-linked with the aircraft inertial navigation and satellite communication antenna, and is used to generate aircraft position, aircraft attitude, sensor attitude, and time information data for the system management module and the intelligent processing module to call.

[0025] In other embodiments, the co-aperture receiving assembly includes a Cassegrain antenna with an optically synthesized aperture in various combination modes; the combination modes include any combination of two or more of short-wave & medium-wave & long-wave infrared, visible light, and laser; light of each wavelength is separated by a dichroic and beam splitter and then enters different detector targets for signal modulation and signal filtering.

[0026] In other embodiments, the integrated processing subsystem realizes video acquisition and preprocessing through the FPGA of the intelligent processing module, transmits video data to the intelligent processing module through the PCIe interface, distributes compressed video using an Ethernet switch, the intelligent processing module supports collaborative processing of high-computing power algorithms by at least two intelligent processing chips, and the video data transmission between the intelligent processing chips of the intelligent processing module and the FPGA of the intelligent processing module is driven and realized through the PCIE interface.

[0027] In other embodiments, the video processing process in the integrated information processing subsystem includes: The infrared optical signal is transmitted to the FPGA of the intelligent processing module through an optoelectronic transceiver based on the SRIO protocol, and the FPGA of the intelligent processing module parses and preprocesses the data according to the SRIO protocol and then transmits it to the intelligent processing chip through the PCIE2.0 4X interface for processing; The white light AVT signal enters the FPGA of the intelligent processing module through the AVT protocol and a video receiving chip, and the FPGA of the intelligent processing module parses and preprocesses the data according to the AVT protocol and then transmits it to the intelligent processing chip through the PCIE2.0 4X interface for processing; Dedicated video transmission channel: The infrared video after being processed by the intelligent processing chip is transmitted to another intelligent processing chip through MIPI; The compressed video processed by the intelligent processing chip is distributed to 6 Gigabit Ethernet outputs through an Ethernet switch.

[0028] In other embodiments, the system management module includes a dual-core CPU unit, an FPGA unit capable of realizing multi-source data fusion, a 289A bus module, and a storage unit; The first processor of the dual-core CPU unit is responsible for management tasks and algorithm tasks, and the second processor is responsible for communication tasks. The FPGA unit is connected to the CPU unit through an LBC interface, and the FPGA unit is externally connected to a storage unit; the FPGA unit expands multiple RS422 interfaces to communicate with the intelligent processing module and converges to the dual-core CPU unit through the LBC bus to realize system management, sensor management, data communication, sensor interface management, servo control, and spot tracking algorithm of the system management module; the dual-core CPU unit controls the 289A bus module to communicate with the mission computer through the PCIE bus.

[0029] In other embodiments, the navigation module includes a navigation board and a navigation sub-card, a navigation sub-card 2, and a self-destruct memory card mounted on the navigation board.

[0030] In other embodiments, the CPU of the system management module runs the domestic operating system DeltaSVM with partitioned channels. The operating system DeltaSVM includes a secure virtual machine kernel and a core-level board support package, runs in the kernel mode of the processor, and realizes isolation and scheduling of system resources through a hardware privilege management unit; The operating system DeltaSVM further includes a runtime library, configured to be callable both in the system state and the user state, and realizes dynamic memory allocation through a memory management unit; The operating system DeltaSVM further includes a partition-level board support package and aircraft software running environment components, runs in the user state, and restricts direct access to kernel state resources through a privilege isolation mechanism; The operating system DeltaSVM further includes a processor architecture that supports multi-level privilege state switching, including kernel state, system state, and user state; The secure virtual machine kernel manages real-time task scheduling through a hardware interrupt controller; The core-level board support package interacts with physical peripheral drivers through register mapping; The runtime library provides a standardized interface across privilege states for user-state applications to call system-level functions.

[0031] In other embodiments, the processor of the intelligent processing module runs the soft real-time operating system openEuler-22.03.

[0032] In other embodiments, the integrated processing subsystem is installed within a fixed frame and does not rotate with the servo system, and all the optoelectronic payloads are placed on the pitch axis system. Specific Embodiment 2:

[0033] An optoelectronic system integrated management method based on the optoelectronic system integrated management system described in any of the above embodiments, the method specifically includes the following steps: S1: Construct a highly integrated hardware architecture, including constructing an optoelectronic subsystem using a receiving aperture integrated configuration and setting up an integrated processing subsystem, and the setting up of the integrated processing subsystem includes an integrated system management module, an intelligent processing module, and a navigation module; S2: Implement real-time software-driven management, including deploying the DeltaSVM operating system for the partitioned road in the system management module, and deploying the openEuler - 22.03 soft real-time operating system in the intelligent processing module; S3: Establish a real-time low-latency data communication architecture, implement video acquisition and preprocessing through FPGA, and transmit video data to the intelligent processing module using a PCIe interface; use an Ethernet switch to distribute compressed video, supporting the transmission of multiple 100-Mbps Ethernet signals; S4: Modular design, where data is exchanged between modules through standard interfaces, supporting the collaborative processing of high-computing-power algorithms by dual intelligent processing modules; S5: Low-power management, using low-power processors and dynamic power management strategies to reduce the overall power consumption of the system. Example 1:

[0034] An optoelectronic system integrated management system, including an optoelectronic subsystem and an integrated processing subsystem; the optoelectronic subsystem adopts a receiving aperture integrated configuration; the integrated processing subsystem includes a system management module, an intelligent processing module, and a navigation module; The optoelectronic subsystem includes a laser emission module with an independent emission aperture and a common-aperture receiving component; the common-aperture receiving component includes an infrared imaging module, a laser receiving module, and a spot tracking module; Data interaction between the system management module and the intelligent processing module is driven and implemented through RapidIO interfaces, PCIE interfaces, and RS422 interfaces; The system management module is used to implement one or more of system management, sensor management, data communication, sensor interface management, servo control, and spot tracking algorithms; the intelligent processing module is used to implement one or more of image processing, AI target recognition, map matching, video compression, image storage, and image transmission.

[0035] The navigation module is cross-linked with the aircraft inertial navigation and satellite communication antenna, and is used to generate aircraft position, aircraft attitude, sensor attitude, and time information data for the system management module and the intelligent processing module to call. The navigation module includes a navigation board and navigation sub-cards, navigation sub-card 2, and a self-destruct memory card mounted on the navigation board.

[0036] The common aperture receiving component includes a Cassegrain antenna with an optically synthesized aperture in various combination modes; the combination modes include any two or more combinations of short-wave & medium-wave & long-wave infrared, visible light, and laser; the light of each wavelength is separated by a dichroic and beam splitter mirror and then enters different detector targets for signal modulation and signal filtering.

[0037] The optoelectronic sub-system is provided with an independent laser emission aperture of not less than 50 mm and an independent visible light receiving aperture of 30 mm. The laser emission module is provided with an independent emission aperture physically isolated from the common aperture receiving component. The independent emission aperture adopts a laser generator, a full-wave antenna, a pump control circuit, a power supply circuit, and a temperature control circuit, and has the ability of temperature control warning and dynamic and static output adjustment.

[0038] The optoelectronic sub-system is provided with a common aperture receiving component of not less than 150 mm. The infrared imaging module, laser receiving module, and spot tracking module of the common aperture receiving component adopt a beam splitter and dichroic filter to ensure separate optical transmission paths for each wavelength.

[0039] In Embodiment 1, as Figure 1 shown, in the optoelectronic sub-system of the optoelectronic system integrated management system, the common aperture receiving component includes a Cassegrain antenna with a combined aperture for laser reception and short-wave infrared; the light in the laser and near-infrared bands is separated by dichroism to obtain a laser signal and an infrared signal; the laser signal passes through a second-order optical modulation system and enters the laser receiving module and the spot tracking module respectively to achieve laser ranging and spot tracking; the infrared signal enters the infrared imaging module to achieve infrared detection; the infrared signal first passes through an infrared optical lens group and then passes through a focusing / non-uniformity correction component (which consists of a servo mechanism, a servo control circuit, an internal correction mechanism, an internal correction control circuit, and a temperature compensation unit) and then enters the infrared core for imaging.

[0040] The integrated processing sub-system realizes video acquisition and preprocessing through the FPGA of the intelligent processing module, transmits video data to the intelligent processing module through a PCIe interface, distributes compressed video using an Ethernet switch, and the intelligent processing module supports cooperative processing of high-computing power algorithms by at least two intelligent processing chips. The video data transmission between the intelligent processing chips of the intelligent processing module and the FPGA of the intelligent processing module is realized through PCIE interface driving.

[0041] In Embodiment 1, the intelligent processing module consists of an intelligent board. This board is based on a domestic high-reliability, high-performance, high-computing power, and low-power intelligent module. Utilizing the capabilities of the processor, AI accelerator, VPU, etc. integrated in this module, various image processing, AI target recognition, map matching, and video compression functions based on machine learning are realized. A large-capacity storage disk of 1TB is connected under this module, enabling the storage of electronic maps and compressed video data. And it is sent to 6-channel 100M Ethernet outputs through an Ethernet switching chip. Meanwhile, the intelligent board realizes the conversion, preprocessing of video, and the exchange of video processing data based on FPGA.

[0042] In Embodiment 1, the video processing function consists of three parts, which are distributed on the intelligent processing module: The first part: Video acquisition, preprocessing, and distribution: Adopt video acquisition based on the FPGA of the intelligent processing module and a distribution mechanism based on the PCIe high-speed interface; The second part: AI video processing and compression storage: Rely on the powerful encoding and decoding capabilities of the intelligent processing chip, the excellent computing power of up to 20T, and a large-capacity self-destructive storage disk of 2T to achieve; The third part: Compressed video distribution: A responsible for the distribution of Ethernet compressed video by an 8-port Ethernet switch.

[0043] As Figure 2 shown, the infrared light signal is transmitted to the FPGA of the intelligent processing module through an optoelectronic transceiver based on the SRIO protocol. The FPGA of the intelligent processing module parses the data according to the SRIO protocol and preprocesses it, and then transmits it to the intelligent processing chip through the PCIE2.0 4X interface for target recognition, map matching, video compression, image storage, image transmission, etc. processing, as Figure 2 the data flow directions of ① and ② in.

[0044] The white light AVT signal enters the FPGA of the intelligent processing module through the AVT protocol and a video receiving chip. The FPGA of the intelligent processing module parses the data according to the AVT protocol and preprocesses it, and then transmits it to the intelligent processing chip through the PCIE2.0 4X interface for target recognition, map matching, video compression, image storage, image transmission, etc. processing, as Figure 2 the data flow directions of ③ and ④ in.

[0045] Dedicated video transmission channel: The infrared video processed by the intelligent processing chip is transmitted to another intelligent processing chip through MIPI; as Figure 2 the data flow direction of ⑤ in.

[0046] The compressed video processed by the intelligent processing chip is distributed to 6-channel 100M Ethernet outputs through an Ethernet switch, as Figure 2 the data flow direction of ⑬ in.

[0047] The log data processing process in the integrated information processing subsystem of the optoelectronic system integrated management system includes: The system management module interacts with the FPGA through the SRIO interface, and the FPGA can then distribute it to the intelligent processing module through the PCIE2.0 4X interface. The intelligent processing module stores the log data in the SSD solid-state drive, as shown in Figure 2 the data flow directions of ⑥, ⑦, and ⑭ in

[0048] The measurement and control data processing process in the integrated information processing subsystem of the optoelectronic system integrated management system includes: The system management module interacts with the intelligent processing module through the RS422 interface. The intelligent processing module implements the control commands issued by the system management module and reports the status to the system management module, as shown in Figure 2 the data flow directions of ⑩, ⑪, and ⑫ in

[0049] The laser emission module includes a 1550nm eye-safe wavelength laser (905nm or 650nm visible light optional), integrating pulse ranging and target indication functions, supporting ranging with an accuracy of ±1m within 10km and dynamic beam calibration. It provides an indication mark for the optical sensor (visible light / infrared camera) of the pod, supports power grading adjustment, low-power warning at close range, and high-power damage at long range. In this Embodiment 1, the infrared imaging module is composed of a high-sensitivity cooled infrared focal plane detector, a wide-band chalcogenide glass / germanium crystal optical lens, a micro Stirling cooler (cooled type), a real-time image processing chip, and a temperature compensation circuit. It captures the target thermal radiation through the 8-14μm long-wave infrared band, combines the adaptive non-uniformity correction (NUC) algorithm and the digital detail enhancement (DDE) technology, and outputs a thermal imaging picture with a resolution of 640×512@30Hz or higher.

[0050] The laser receiving module is composed of a high-response-speed InGaAs / silicon-based avalanche photodiode (APD), a narrow-band filter (such as 1550nm / 1064nm wavelength), a low-noise preamplifier, and a digital processing circuit. Through the adaptive threshold detection technology, it realizes the precise capture of nanosecond-level laser pulse signals (repetition frequency ≥20kHz), with a dynamic range of 120dB, and can effectively extract the laser echo reflected or actively irradiated by the target under strong ambient light interference, and the ranging accuracy is better than ±0.1m.

[0051] The spot tracking module consists of a high-frame-rate CMOS / CCD imaging sensor, a narrowband tunable filter, an FPGA parallel processing architecture, and an adaptive closed-loop servo mechanism. Through the sub-pixel-level spot centroid extraction algorithm and Kalman filter prediction technology, it realizes the real-time locking of spots with a diameter ≥ 0.1 mrad, with a tracking accuracy of ±5 μrad, a frame rate ≥ 200 Hz, and can stably track under the conditions of high-speed maneuvering of the target (angular velocity ≥ 50° / s) or strong background light interference (such as direct sunlight).

[0052] The system management module includes a dual-core CPU unit, an FPGA unit capable of realizing multi-source data fusion, a 289A bus module, and a storage unit; The first processor of the dual-core CPU unit is responsible for management tasks and algorithm tasks, and the second processor is responsible for communication tasks. The FPGA unit is connected to the CPU unit through an LBC interface, and the FPGA unit is externally connected to a storage unit; the FPGA unit expands multiple RS422 interfaces to communicate with the intelligent processing module, and converges to the dual-core CPU unit through the LBC bus to realize system management, sensor management, data communication, sensor interface management, servo control, and spot tracking algorithm of the system management module; the dual-core CPU unit controls the 289A bus module to communicate with the mission computer through the PCIE bus.

[0053] By deploying the Tao partition operating system and Euler operating system on the system management module and the intelligent processing module respectively, the real-time performance and collaborative working efficiency of the system are ensured.

[0054] The CPU of the system management module runs the domestic Tao partition operating system DeltaSVM, and its supporting software integrated development environment is LambdaTOOL-SVM.

[0055] As Figure 3 shown, the operating system DeltaSVM mainly includes a secure virtual machine kernel (SVMK), a core-level board support package (KBSP), a runtime library (RTL), a partition-level board support package (VBSP), and an aircraft software runtime environment component (VRTE). Among them, the secure virtual machine kernel and the core-level board support package run in the core state (privileged state), the runtime library can be used in both the system state and the user state, and the rest of the parts, including partition applications, run in the user state.

[0056] The operating system DeltaSVM includes a secure virtual machine kernel and a core-level board support package, runs in the core state of the processor, and realizes the isolation and scheduling of system resources through the hardware privilege management unit; The operating system DeltaSVM also includes a runtime library, configured to be callable in both the system state and the user state, and realizes dynamic memory allocation through the memory management unit; The operating system DeltaSVM also includes a partition-level board support package and aircraft software operating environment components, runs in user mode, and restricts direct access to kernel-mode resources through a privilege isolation mechanism; The operating system DeltaSVM also includes a processor architecture that supports multi-level privilege state switching, including kernel mode, system mode, and user mode; The security virtual machine kernel manages real-time task scheduling through a hardware interrupt controller; The core-level board support package interacts with physical peripheral drivers through register mapping; The runtime library provides a standardized interface across privilege states for user-mode applications to call system-level functions.

[0057] The operating system DeltaSVM also has the following characteristics: Partition space isolation: Ensure that the data space within a partition is not damaged by applications in other partitions, and at the same time ensure that the operating system core data space is not damaged by partition applications; Partition time isolation: Support the time-schedule scheduling method to ensure that partition applications are run within their corresponding running time windows and are not preempted by other partitions for the corresponding running time windows; Interrupt resource isolation: Support the distribution of interrupts according to configuration to ensure the isolation of interrupt resources; Switchable time schedule during operation: Support switching between multiple statically configured time schedules during operation to facilitate user support for switching multiple application software operation modes.

[0058] Provide inter-partition communication: Provide an inter-partition communication mechanism for shared memory, service interrupts, and message publishing.

[0059] Provide rate-group task scheduling service: Provide rate-group task scheduling capabilities within a partition, which can schedule registered tasks according to the specified rate-group period, and at the same time provide a detection and handling mechanism for rate-group task timeouts; Provide system expansion: Provide multiple expansion interfaces to facilitate users to make corresponding expansions to the operating system to meet complex application requirements.

[0060] Provide configurable privileged system calls: Through privileged system calls, the management ability of the system can be provided in a partition, and at the same time, a static configuration mechanism is adopted to prevent the unintentional or illegal use of privileged system calls.

[0061] Health management and fault handling: Ensure that the normal operation of applications in other partitions is not affected by a fault in a partition application, and provide a configurable multi-level fault handling mechanism to facilitate users to handle partition faults.

[0062] Two-level watchdog, supporting the system hardware watchdog and providing a partition-level software watchdog. The system-level watchdog is unique in the system and monitors the system's operating status; the partition-level watchdog can be owned by each partition and monitors the operating status of the specified partition.

[0063] Full-static configuration of resources: All system resources are configured statically, supporting configuration data validity checks and configuration generation to ensure the determinacy of resources during application operation.

[0064] Dual-clock scheduling penalty: Supports triggering by the main frame of the schedule table and the arrival of the time window, using different clock trigger sources to reduce the time drift of the main frame.

[0065] The processor of the intelligent processing module runs the soft real-time operating system openEuler-22.03. The soft real-time operating system openEuler-22.03 interacts with the hardware through a layered architecture, and the core depends on the kernel driver, hardware abstraction interface, and system calls to achieve seamless connection from physical devices to user programs.

[0066] The soft real-time operating system openEuler-22.03 has soft real-time characteristics, mainly including the following: Low latency: By optimizing kernel scheduling and interrupt handling, ensure that tasks are completed within the specified time.

[0067] Determinacy: Provide more predictable response times, suitable for scenarios that require stable performance.

[0068] Resource isolation: Adopt CPU isolation and memory control technologies to reduce interference between tasks.

[0069] The main functions of the soft real-time operating system openEuler-22.03 include: Real-time scheduling: Support real-time scheduling policies such as SCHED_FIFO and SCHED_RR.

[0070] Kernel optimization: Optimize task switching, interrupt handling, and lock mechanisms to improve real-time performance.

[0071] Tool support: Provide real-time performance analysis tools to help developers debug and optimize.

[0072] The integrated processing subsystem is installed in a fixed frame and does not rotate with the servo system. All the optoelectronic payloads are placed on the pitch axis system.

[0073] The optical and electrical subsystem outer envelope of the optical and electrical system integrated management system is an optical window assembly; the optical window assembly is composed of a multi-spectral composite material with high light transmittance and an impact-resistant support structure. The edge of the optical window is coated with a lightweight metal / composite material, which not only ensures the overall structural strength but also reduces the reflectance through surface treatment technology, improving the concealment and environmental adaptability.

[0074] Embodiment 2: An optical and electrical system integrated management method based on the optical and electrical system integrated management system described in Embodiment 1, the method specifically includes the following steps: S1: Construct a highly integrated hardware architecture, including constructing the optical and electrical subsystem by adopting a receiving aperture integrated configuration and setting up an integrated processing subsystem, and the setting up of the integrated processing subsystem includes an integrated system management module, an intelligent processing module, and a navigation module; S2: Implement real-time software-driven management, including deploying the operating system DeltaSVM of the partitioned road in the system management module and deploying the soft real-time operating system openEuler - 22.03 in the intelligent processing module; S3: Establish a real-time low-latency data communication architecture, realize video acquisition and preprocessing through FPGA, and transmit video data to the intelligent processing module by using the PCIe interface; use an Ethernet switch to distribute compressed video, supporting the transmission of multiple 100M Ethernet signals; S4: Modular design, where data is exchanged between modules through standard interfaces, supporting the collaborative processing of high-computing power algorithms by dual intelligent processing modules; S5: Low-power management, adopting a low-power processor and a dynamic power management strategy to reduce the overall power consumption of the system. Embodiment 3:

[0075] Apply the optical and electrical system integrated management system described in the above Embodiment 1 to the unmanned aerial vehicle ground reconnaissance and target indication system.

[0076] ‌Application scenario:‌ A certain type of reconnaissance and strike integrated unmanned aerial vehicle performs a patrol mission and needs to conduct real-time detection, identification, and laser indication of ground moving targets (such as vehicles) to guide missiles.

[0077] ‌System implementation functions:‌ 1.‌Target search and identification‌ ‌Infrared imaging module‌: Capture the vehicle's thermal radiation through the 8-14μm long-wave infrared band, and combine with DDE technology to output a 640×512@60Hz high-definition thermal imaging picture, and can also identify the target contour at night or in a smoky environment.

[0078] AI Target Recognition: The intelligent processing module deploys the YOLOv8 model to analyze infrared images in real time, classify and recognize vehicle types (trucks / armored vehicles) with a confidence level ≥ 95%.

[0079] 2. Laser Ranging and Indication Laser Emission Module: It uses a 1550nm eye-safe wavelength to emit 20kHz pulsed laser, achieving a ranging accuracy of ±0.5m within an 8km distance, and simultaneously generating a dynamic beam to mark the target position.

[0080] Spot Tracking Module: It locks the laser spot through a 200Hz frame rate CMOS sensor, combines Kalman filtering to predict the vehicle's movement trajectory, with a tracking accuracy of ±5μrad, ensuring that the missile seeker continuously captures the target.

[0081] 3. Multi-module Cooperative Control System Management Module: It coordinates laser emission, infrared imaging, and servo mechanisms through the system bus, adjusts the focal length of the optical lens group in real time (response time < 10ms) to adapt to the altitude changes of the UAV (500 - 3000m).

[0082] Navigation Board: It fuses inertial navigation and satellite communication data, outputs the attitude of the carrier aircraft (accuracy 0.1°) and the target geographical coordinates, and guides the missile to bind fire control parameters.

[0083] Technical Advantages: Reduced Communication Delay: The RapidIO and PCIE interfaces enable data interaction delay between modules < 1ms, which is 10 times faster than traditional serial communication and Ethernet.

[0084] Lightweight Design: The co-aperture receiving component integrates multi-spectral sensors, reducing the volume by 40% compared to the distributed architecture, and adapting to the payload limitations of UAVs.

[0085] Dynamic Power Management: The intelligent processing module dynamically adjusts the allocation of AI computing resources and hardware acceleration strategies according to the task load, with the overall power consumption of the machine ≤ 150W. Example 4:

[0086] Apply the optoelectronic system integrated management system described in Example 1 above to an intelligent patrol and intrusion detection system.

[0087] Application Scenario: A UAV patrol system is deployed in a certain plateau to monitor activities under complex terrains (such as mountains and jungles) for 24 hours, with a coverage range of up to 50 kilometers.

[0088] System Realized Functions: 1. Wide-area Target Reconnaissance Infrared Imaging Module: It adopts a cooled infrared focal plane detector (NETD ≤ 40mK), realizes the detection of human heat sources within a range of 5 km through the 8 - 14μm band (temperature resolution ±0.5°C), and cooperates with chalcogenide glass lenses (F number 1.2) to improve the imaging quality in low - light environments.

[0089] Laser - assisted Ranging: The 1550nm laser module emits 50kHz pulses, conducts real - time ranging on moving targets (persons / vehicles) (accuracy ±0.3m@3km), and synchronously generates a geofence warning area.

[0090] 2. Intelligent Behavior Analysis AI Abnormality Recognition: The intelligent processing module deploys a spatio - temporal fusion network model, analyzes the target movement trajectory (speed / direction / aggregation state), and distinguishes normal patrols from suspicious activities (accuracy ≥ 97%).

[0091] Multi - source Data Fusion: The navigation board integrates Beidou differential positioning (accuracy ±0.1m), combines inertial navigation data to calibrate the UAV attitude, and realizes the precise superposition of the target geographical coordinates (latitude and longitude) and the surveillance video.

[0092] 3. Quick Response Mechanism Spot Tracking and Guidance: After locking the target, the spot tracking module drives the servo mechanism to adjust the laser indication direction (angular velocity ≥ 30° / s) to guide the interception.

[0093] Real - time Monitoring: H.265 encoding is used to compress multiple video streams in real - time (compression ratio 120:1), and they are synchronized to the command post through a wireless network (delay < 50ms).

[0094] Technical Advantages: The optical window component uses diamond - coated fused quartz (transmittance ≥ 92%@8 - 14μm), and is resistant to sandstorms (visibility < 50m) and low temperatures of - 40°C.

[0095] The operating system DeltaSVM realizes the privilege isolation between the kernel state and the user state, and the key process fault recovery time < 50ms.

[0096] Outstanding Expandability: It supports the cascading of dual intelligent processing modules, and the target tracking capacity expands from 50 to 200, meeting the extended requirements.

Claims

1. An integrated management system for an optoelectronic system, characterized in that, It includes an optoelectronic sub-system and a comprehensive processing sub-system; the optoelectronic sub-system adopts a receiving aperture synthesis configuration; the comprehensive processing sub-system includes a system management module, an intelligent processing module and a navigation module; The optoelectronic sub-system includes a laser emission module with an independent emission aperture and a co-aperture receiving assembly; the co-aperture receiving assembly includes an infrared imaging module, a laser receiving module and a spot tracking module; The data interaction between the system management module and the intelligent processing module is driven and realized through RapidIO interfaces, PCIE interfaces and RS422 interfaces; The system management module is used to implement one or more of system management, sensor management, data communication, sensor interface management, servo control and spot tracking algorithms; the intelligent processing module is used to implement one or more of image processing, AI target recognition, map matching, video compression, image storage and image transmission; the navigation module is cross-linked with the aircraft inertial navigation and satellite communication antenna and is used to generate aircraft position, aircraft attitude, sensor attitude and time information data for the system management module and the intelligent processing module to call.

2. The integrated management system for an optoelectronic system according to claim 1, characterized in that: The co-aperture receiving assembly includes a Cassegrain antenna with an optically synthesized aperture in various combination modes; the combination modes include any combination of two or more of short-wave & medium-wave & long-wave infrared, visible light and laser; light of each wavelength is separated by a dichroic and beam splitter and then enters different detector targets for signal modulation and signal filtering.

3. The integrated management system of the optoelectronic system according to claim 1, wherein: The comprehensive processing sub-system realizes video acquisition and preprocessing through the FPGA of the intelligent processing module, transmits video data to the intelligent processing module through the PCIe interface, distributes compressed video by using an Ethernet switch, the intelligent processing module supports the collaborative processing of high-computing power algorithms by at least two intelligent processing chips, and the video data transmission between the intelligent processing chips of the intelligent processing module and the FPGA of the intelligent processing module is driven and realized through the PCIE interface.

4. The integrated management system for an optoelectronic system according to claim 3, characterized in that: The video processing process in the comprehensive information processing sub-system includes: The infrared optical signal is transmitted to the FPGA of the intelligent processing module through an optoelectronic transceiver based on the SRIO protocol, and the FPGA of the intelligent processing module analyzes and preprocesses the data according to the SRIO protocol and then transmits it to the intelligent processing chip through the PCIE2.0 4X interface for processing; The white light AVT signal enters the FPGA of the intelligent processing module through the AVT protocol and a video receiving chip, and the FPGA of the intelligent processing module analyzes and preprocesses the data according to the AVT protocol and then transmits it to the intelligent processing chip through the PCIE2.0 4X interface for processing; Dedicated video transmission channel: the infrared video after being processed by the intelligent processing chip is transmitted to another intelligent processing chip through MIPI; The compressed video after being processed by the intelligent processing chip is distributed to 6-way 100-Mbps Ethernet outputs through an Ethernet switch.

5. The integrated management system of the optoelectronic system according to claim 1, wherein: The system management module includes a dual-core CPU unit, an FPGA unit capable of realizing multi-source data fusion, a 289A bus module and a storage unit; The first processor of the dual-core CPU unit is responsible for management tasks and algorithm tasks, and the second processor is responsible for communication tasks. The FPGA unit is connected to the CPU unit through the LBC interface, and the FPGA unit is externally connected to a storage unit. The FPGA unit expands multiple RS422 interfaces to communicate with the intelligent processing module and converges through the LBC bus to the dual-core CPU unit to implement system management, sensor management, data communication, sensor interface management, servo control, and spot tracking algorithm of the system management module. The dual-core CPU unit controls the 289A bus module through the PCIE bus to communicate with the mission computer.

6. The integrated management system for an optoelectronic system according to claim 1, characterized in that: The navigation module includes a navigation board, a navigation sub-card, a navigation sub-card 2, and a self-destruct memory card mounted on the navigation board.

7. The integrated management system for an optoelectronic system according to claim 1, characterized in that: The CPU of the system management module runs the domestic road partition operating system DeltaSVM. The operating system DeltaSVM includes a secure virtual machine kernel and a core-level board support package, runs in the kernel mode of the processor, and realizes isolation and scheduling of system resources through the hardware privilege management unit. The operating system DeltaSVM also includes a runtime library, which is configured to be callable in both the system state and the user state, and realizes dynamic memory allocation through the memory management unit. The operating system DeltaSVM also includes a partition-level board support package and an aircraft software running environment component, runs in the user state, and restricts direct access to kernel state resources through a privilege isolation mechanism. The operating system DeltaSVM also includes a processor architecture that supports multi-level privilege state switching, including kernel state, system state, and user state. The secure virtual machine kernel manages real-time task scheduling through the hardware interrupt controller. The core-level board support package interacts with physical peripheral drivers through register mapping. The runtime library provides a standardized interface across privilege states for user-state applications to call system-level functions.

8. The integrated management system for an optoelectronic system according to claim 1, wherein: The processor of the intelligent processing module runs the soft real-time operating system openEuler - 22.

03.

9. The integrated management system for an optoelectronic system according to claim 1, characterized in that: The integrated processing subsystem is installed in a fixed frame and does not rotate with the servo system, and all optoelectronic payloads are placed on the pitch axis system.

10. A method for comprehensively managing an optoelectronic system based on the optoelectronic system comprehensive management system according to any one of claims 1 to 9, characterized in that: The method specifically includes the following steps: S1: Construct a highly integrated hardware architecture, including constructing an optoelectronic subsystem using a receiving aperture synthesis configuration and setting up an integrated processing subsystem, and the setting up of the integrated processing subsystem includes integrating a system management module, an intelligent processing module, and a navigation module. S2: Implement real-time software driver management, including deploying the road partition operating system DeltaSVM in the system management module and deploying the soft real-time operating system openEuler - 22.03 in the intelligent processing module. S3: Establish a real-time low-latency data communication architecture, realize video acquisition and preprocessing through the FPGA, and transmit video data to the intelligent processing module using the PCIe interface; use an Ethernet switch to distribute compressed video and support the transmission of multiple 100-Mbps Ethernet signals. S4: Modular design, where each module exchanges data through standard interfaces, supporting the collaborative processing of high-computing-power algorithms by dual intelligent processing modules. S5: Low-power management, using a low-power processor and a dynamic power management strategy to reduce the overall power consumption of the system.

Images