Photoelectric wide-area monitoring system and method suitable for aerial platform

Abstract

The invention provides a photoelectric wide-area monitoring system and method suitable for an air platform. The system comprises a three-axis stable platform used for collecting mass image data, a ground control end used for controlling and displaying, and an air-ground radio station used for completing uplink and downlink information transmission between the three-axis stable platform and the ground control end. The method comprises the following steps: a three-axis stabilized platform points to a specified geographic position, and an ultrahigh-resolution camera performs real-time full-width imaging and roaming window imaging on a specified area; the image processing module receives and processes the real-time full-width image and receives and processes the roaming window image to obtain image information; image information is transmitted to the ground control end through the air-to-ground radio station, and an instruction command output by the ground control end is transmitted to the image processing module or the three-axis stable platform. According to the invention, the ultra-high-resolution camera is adopted to realize simultaneous large-range area imaging and local roaming imaging of a single camera, and the adaptive capacity of an aerial platform is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of optoelectronic monitoring, and in particular relates to an optoelectronic wide-area monitoring system and method suitable for an aerial platform. Background Art

[0002] With the rapid development of flight platform technology, higher requirements have been placed on the improvement of the surveillance capabilities of the optoelectronic equipment carried on the platform. Based on the characteristics of traditional optoelectronic pods and aerial camera systems, aerial platform surveillance equipment is required to have both large-scale imaging and high-resolution detail imaging. These two main surveillance capabilities require a new technical approach to break through the current technical status quo where traditional systems can only have one capability.

[0003] At present, regional monitoring systems are mainly implemented by panoramic shooting equipment. There are two main technical approaches for such equipment, namely, spatial splicing of multiple cameras and spatial scanning of a small number of cameras to achieve panoramic imaging, providing large-scale regional imaging and high-resolution detail imaging information. Multi-camera splicing equipment uses multiple cameras to form a system, which has the disadvantages of large size, heavy weight and complex information lines, and cannot meet the stringent requirements of aerial platforms for volume and weight; the system implemented by spatial scanning actually exchanges time for space, so there are problems in real-time monitoring.

[0004] At the same time, the aerial surveillance system is limited by the bandwidth of radio and satellite communications. It is difficult to transmit the massive data of large-scale regional surveillance imaging to the ground in real time, resulting in the inability to effectively use the real-time surveillance image information. Even if it is transmitted to the ground, the monitoring personnel are unable to identify all the image content within the regional imaging position. At present, a new technical approach is needed to solve the technical difficulties of large-scale imaging and high-resolution imaging, and an efficient human-machine combination method is needed to solve the efficiency problem of regional surveillance. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art that large-scale imaging and high-resolution imaging cannot be taken into account at the same time, and to provide an optoelectronic wide-area monitoring system and method suitable for aerial platforms.

[0006] To achieve the above purpose, the technical solution provided by the present invention is:

[0007] On the one hand, the present invention provides an optoelectronic wide-area monitoring system suitable for an aerial platform, comprising: a three-axis stabilized platform for collecting massive image data, a ground control terminal for control and display, and an air-to-ground radio station for completing uplink and downlink information transmission between the three-axis stabilized platform and the ground control terminal; wherein:

[0008] The three-axis stabilized platform is equipped with an ultra-high-resolution camera, a slip ring and an image processing module. The ultra-high-resolution camera generates a large amount of image data when monitoring the ground, which is divided into multiple channels and enters multiple parallel data channels accordingly. The data is transmitted to the image processing module through the slip ring, and the image processing module processes the large amount of images into image information in real time.

[0009] The image information is transmitted to the ground control terminal via downlink information of the air-to-ground radio station, and the instruction command output by the ground control terminal is transmitted to the image processing module via uplink information of the air-to-ground radio station.

[0010] Based on one aspect, in a possible implementation, the three-axis stabilized platform is installed on an aerial platform, the three-axis stabilized platform includes an upper cabin and a lower cabin, the image processing module is arranged in the upper cabin, a satellite combined inertial navigation module and a 10 Gigabit switch are also arranged in the upper cabin, and the ultra-high resolution camera is arranged in the lower cabin;

[0011] The three-axis stable platform provides a stable pointing and imaging environment for the ultra-high resolution camera, and the slip ring is installed on the three-axis stable platform;

[0012] The three-axis stabilized platform has the ability to adjust the pointing and stabilize the azimuth, pitch and roll sight axes, and the merge ring can achieve 360° continuous rotation.

[0013] Based on one aspect, in a possible implementation, the ultra-high resolution camera uses a single-chip high-resolution imaging chip, and the ultra-high resolution camera is equipped with a high-resolution large-field-of-view optical lens;

[0014] The ultra-high resolution camera provides regional monitoring images and multiple roaming windows, and the ultra-high resolution camera performs real-time full-frame imaging and roaming window imaging in a designated area of ​​ground monitoring;

[0015] The image processing module receives real-time full-frame imaging and roaming window imaging, and performs intelligent target recognition and extraction or moving target extraction on the real-time full-frame imaging and transmits it to the ground control end; the image processing module performs intelligent target recognition or moving target extraction on the roaming window imaging and transmits it to the ground control end.

[0016] Based on one aspect, in a possible implementation, the air-to-ground radio station includes an airborne radio station and a ground radio station, the airborne radio station communicates with the ground radio station, the image information is transmitted to the ground control terminal via the downlink information of the airborne radio station, and the command output by the ground control terminal is transmitted to the image processing module via the uplink information of the ground radio station.

[0017] Based on one aspect, in a possible implementation, the image processing module divides the massive image data into multiple paths, which are correspondingly entered into multiple parallel data channels and transmitted to multiple information processing boards;

[0018] The image processing module is installed on the aerial platform, and the number of computing boards is expanded according to the demand to increase the processing capacity; the image processing module includes a computer board, a distributed information processing board, a 10G switch, and a storage board; its functions at least include command reception, image preprocessing, target detection, image storage and image compression transmission, and it has the ability to process massive images, and can process images output by ultra-high-resolution cameras including regional images and roaming windows in real time, and encode and compress the images, information and system status that need to be transmitted to the aerial radio station, and receive commands from the ground control terminal through the aerial radio station;

[0019] The distributed information processing board receives the full-width massive image, divides it into smaller image blocks by overlapping and cutting according to the width of the full-width massive image, and directly sends the image blocks to the detection and reasoning software unit on the distributed information processing board, and uses the artificial intelligence algorithm to quickly complete the processing of the massive image blocks, so as to obtain the image content information such as the type, size, and position of the target in the image, and generate target recognition meta-information including the target sub-image; the distributed information processing board receives the roaming window image, and realizes the tracking and prediction of multiple targets in the window image through the information association of multiple frames of roaming window images, and generates target tracking information; the distributed information processing board transmits the target recognition meta-information and target tracking information to the computer board for information fusion and application distribution;

[0020] The ground control terminal includes an information computing unit, a display unit, a geographic information system, and an image-based computing server, and has the functions of generating and sending instructions, receiving information, and displaying image information, and is used to support the monitoring system to achieve geographic positioning and regional information display;

[0021] The ground control end receives the status information, full-frame sampling images, roaming window images, target identification information and target tracking information of the three-axis stabilized platform, performs real-time image feature matching on all the above-mentioned received information and the monitoring area image provided by its own geographic information system, realizes image geographic alignment, generates full-frame sampling images and roaming window images with geographic information, and sends them to the display unit to form a monitoring area image display; after the ground control end transmits the target identification information and target tracking information to the information calculation unit for processing, the time-space alignment of the target and the display unit image is realized, and the images are superimposed on the display image of the display unit; the ground control end operator realizes closed-loop control of the three-axis stabilized platform based on the status information of the three-axis platform and the display content of the display unit.

[0022] On the other hand, the present invention provides an optoelectronic wide-area monitoring method applicable to an aerial platform. The optoelectronic wide-area monitoring method is based on the optoelectronic wide-area monitoring system described above. The optoelectronic wide-area monitoring method includes:

[0023] The three-axis stabilized platform points to the specified geographic location, and the ultra-high-resolution camera performs real-time full-frame imaging and roaming window imaging of the specified area;

[0024] The image processing module receives and processes the real-time full-frame imaging, and receives and processes the roaming window imaging, to obtain image information;

[0025] The image information is transmitted to the ground control terminal through the air-to-ground radio, and the command output by the ground control terminal is transmitted to the image processing module or the three-axis stabilized platform.

[0026] Based on another aspect, in a possible implementation manner, the image processing module receives and processes real-time full-frame imaging, and receives and processes roaming window imaging, including:

[0027] The image processing module receives the real-time full-frame imaging, performs intelligent target recognition and extraction or moving target extraction on the full-frame image, and transmits the identification sub-image, identification type, position and other information in the full-frame imaging to the ground control end for target display superposition;

[0028] The image processing module receives the roaming window imaging in real time, performs intelligent target recognition and extraction or moving target extraction on the roaming window imaging, transmits the real-time image of the window, recognition type, position and other meta-information in the roaming window imaging to the ground control end for secondary information processing, and completes the target frame superposition and display.

[0029] Based on another aspect, in a possible implementation manner, the three-axis stabilized platform points to a specified geographical location, including:

[0030] The three-axis stabilized platform is pointed to different geographical locations at different times through time-division control, and each designated geographical location corresponds to a monitoring sub-area;

[0031] The optoelectronic wide-area monitoring system repeatedly traverses in any predetermined order to form a larger monitoring area. The high-resolution camera performs time-sharing full-frame imaging and roaming window imaging in the designated larger area. The image processing module receives the two types of imaging data and processes them separately.

[0032] The image processing module processes the full-frame imaging and roaming window imaging to form image videos and information, which are then transmitted to the ground control end for secondary information processing to complete target frame overlay and display.

[0033] Based on another aspect, in a possible implementation manner, the command output by the ground control end is transmitted to the image processing module or the three-axis stabilized platform, including:

[0034] The ground control terminal is equipped with regional digital maps or aerial remote sensing orthophoto mosaics and a geographic information system;

[0035] The ground control end collects the information transmitted by the aerial image processing module, including video, target information, real-time platform position information provided by satellite combined inertial navigation, and time stamp information after timing.

[0036] The ground control end performs image matching, positioning solution, and information fusion on the information to complete the positioning and display superposition of the downlinked target;

[0037] The ground control terminal displays and controls multiple roaming windows, allowing multiple users to simultaneously monitor multiple arbitrary locations within the monitoring area in real time.

[0038] The advantages of the present invention are:

[0039] 1. The present invention adopts an ultra-high-resolution camera to realize single-camera simultaneous imaging of a large area and local roaming imaging, which solves the current problems of large volume, heavy weight and poor real-time performance caused by multi-sensor splicing and spatial time-sharing scanning, and improves the adaptability of the aerial platform.

[0040] 2. The present invention adopts artificial intelligence recognition and moving target detection algorithms in wide-area monitoring, completes real-time processing of large-scale images of the entire area in the air, and displays and processes regional monitoring results in the form of information at the ground station in combination with the geographic information system, which solves the contradiction between limited communication bandwidth of the aerial platform and transmission of massive data for regional monitoring, and meets the regional monitoring needs under the conditions of limited data links of the aerial platform.

[0041] 3. The present invention adopts two working modes in wide-area monitoring to monitor a single designated gaze or multiple designated locations in a time-sharing manner, providing a balanced method for real-time and constructing a larger monitoring area, while supporting high-frame rate monitoring of multiple roaming windows at any position in the monitoring area. The monitoring method of the present invention is flexible and efficient in adapting to a variety of aerial monitoring scenarios, effectively improving the monitoring efficiency, and can be applied to a series of national and social activities such as military conflicts, key security, search and rescue, smuggling, and border control.

[0042] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0044] Figure 1: A principle block diagram of an optoelectronic wide-area monitoring system suitable for an aerial platform provided by the present invention;

[0045] Figure 2 : Schematic diagram of the relationship between the full-frame image and the roaming window of the high-resolution camera provided by the present invention;

[0046] Figure 3 : A schematic diagram of monitoring a single designated area provided by the present invention;

[0047] Figure 4 : Schematic diagram of monitoring multiple designated areas with seamless boundaries provided by the present invention;

[0048] Figure 5 : A schematic diagram of monitoring multiple designated areas independent of boundaries provided by the present invention. DETAILED DESCRIPTION

[0049] Embodiments of the present invention are described in detail below. The embodiments are exemplary and intended to be used to explain the present invention, but should not be construed as limiting the present invention.

[0050] See also Figure 1 The embodiment of the present invention provides an optoelectronic wide-area monitoring system suitable for an aerial platform, comprising: a three-axis stabilized platform for collecting massive image data, a ground control terminal for control and display, and an air-to-ground radio station for completing uplink and downlink information transmission between the three-axis stabilized platform and the ground control terminal; wherein: an ultra-high-resolution camera, a collector ring and an image processing module are installed on the three-axis stabilized platform; the massive image data generated by the ultra-high-resolution camera monitoring the ground is divided into multiple channels, correspondingly enters multiple parallel data channels, and is transmitted to the image processing module through the collector ring; the image processing module processes the massive images into image information in real time; the image information is transmitted to the ground control terminal through the downlink information of the air-to-ground radio station, and the command command output by the ground control terminal is transmitted to the image processing module through the uplink information of the air-to-ground radio station.

[0051] The optoelectronic wide-area surveillance system of an embodiment of the present invention is carried on an aerial platform to form a large-scale regional surveillance of the ground, and includes a three-axis stabilized platform, an ultra-high-resolution camera, an information processing module, a satellite combined inertial navigation, an air-to-ground radio station and a ground control terminal, wherein the ultra-high-resolution camera, the information processing module and the satellite combined inertial navigation are all integrated on the three-axis stabilized platform, and the aerial radio station is generally installed on the aerial platform.

[0052] The three-axis stabilized platform of the embodiment of the present invention is installed on an aerial platform, and the three-axis stabilized platform includes an upper cabin and a lower cabin, the image processing module is arranged in the upper cabin, and a satellite combined inertial navigation module and a 10 Gigabit switch are also arranged in the upper cabin, and an ultra-high resolution camera is arranged in the lower cabin; the three-axis stabilized platform provides a stable pointing and imaging environment for the ultra-high resolution camera, and a merge ring is installed on the three-axis stabilized platform; the three-axis stabilized platform has the ability to adjust and stabilize the pointing of the azimuth line of sight, the pitch line of sight, and the roll line of sight, and the merge ring realizes 360° continuous turnover.

[0053] In the embodiment of the present invention, the three-axis stabilized platform is fixed to the aerial platform by hoisting, and the installation position is connected by a metal spring rubber shock absorber to reduce the impact of the aerial platform vibration on the sensor on the three-axis stabilized platform; the three-axis stabilized platform is preferably divided into two cabins, the lower cabin is airtight and used to install ultra-high resolution cameras, and the upper cabin is watertight and used to install electronic components such as image processing modules and satellite combined inertial navigation. The servo drive part of the three-axis stabilized platform is generally driven by the azimuth, pitch, and roll frames and their respective motors, angle sensors, and inertial sensors; preferably, the above-mentioned motors use DC torque motors or DC brushless permanent magnet motors, the above-mentioned angle sensors use encoders or rotary transformers, and the above-mentioned inertial sensors use various types of gyroscopes. The three-axis stabilized platform of the embodiment of the present invention has the ability to adjust and stabilize the visual axis pointing in the three directions of azimuth, pitch, and roll, and provides a stable pointing and imaging environment for the ultra-high resolution camera installed thereon, wherein the azimuth frame is equipped with a collector ring to achieve 360° continuous turnover.

[0054] The ultra-high resolution camera of the embodiment of the present invention adopts a single-chip high-resolution imaging chip, and the ultra-high resolution camera is equipped with a high-resolution large-field-of-view optical lens; the ultra-high resolution camera provides regional monitoring images and multiple roaming windows, and the ultra-high resolution camera performs real-time full-frame imaging and roaming window imaging in the designated area of ​​ground monitoring; the image processing module receives the real-time full-frame imaging and roaming window imaging, and the image processing module performs intelligent target recognition and extraction or moving target extraction on the real-time full-frame imaging and transmits it to the ground control end; the image processing module performs intelligent target recognition or moving target extraction on the roaming window imaging and transmits it to the ground control end.

[0055] The embodiment of the present invention preferably adopts a single-chip detector with a scale of more than 100 million pixels. The resolution of the single-chip detector is 16720×9256. The ultra-high-resolution camera provides source data for the monitoring system. The ultra-high-resolution camera uses a single-chip high-resolution imaging chip and is equipped with a high-resolution large-field optical lens. It also provides regional monitoring images and multiple roaming windows. Among them, the single-chip detector supports the simultaneous output of high-frame rate roaming windows and low-frame rate full-frame images. The roaming window is set to 640×512 according to the field of view of the optical lens. The size and number of roaming windows are planned according to the number of information processing boards carried by the image processing module and the bandwidth of the air-to-ground radio station. Please refer to Figure 2 The roaming window of the embodiment of the present invention can be set to any position in the full image.

[0056] The air-to-ground radio station of the embodiment of the present invention includes an airborne radio station and a ground radio station. The airborne radio station communicates with the ground radio station. Image information is transmitted to the ground control terminal via the downlink information of the airborne radio station. The command output by the ground control terminal is transmitted to the image processing module via the uplink information of the ground radio station.

[0057] The image processing module of the embodiment of the present invention divides the massive image data into multiple paths, which enter the multiple parallel data channels and are transmitted to the information processing module through the convergence ring; the information processing module is installed on the aerial platform and has the hardware expansion capability, and the number of computing boards can be expanded according to the needs to increase the processing capacity. The information processing module is mainly composed of a computer board, a distributed information processing board, a 10 Gigabit switch, and a storage board. Its functions include at least command acceptance, image preprocessing, target detection, image storage and image compression transmission. It has the ability to process massive images, and can process images output by ultra-high-resolution cameras including regional images and roaming windows in real time. It can encode and compress information such as images, information and system status that need to be transmitted to the aerial radio station, and receive commands from the ground control terminal through the aerial radio station. The distributed information processing board receives full-frame massive images, divides them into smaller image blocks through overlapping cutting according to the width of the full-frame massive images, and directly sends the image blocks to the detection and reasoning software unit on the distributed information processing board, and uses artificial intelligence algorithms to quickly complete the processing of massive image blocks, so as to obtain image content information such as the type, size, and position of the target in the image, and generate target recognition meta-information including target sub-images; the distributed information processing board receives roaming window images, and realizes the tracking and prediction of multiple targets in the window image through information association of multiple frames of roaming window images, and generates target tracking information; the distributed information processing board transmits the target recognition meta-information and target tracking information to the computer board for information fusion and application distribution; the ground control end includes an information computing unit, a display unit, a geographic information system, and an image-based computing server, which has the ability to generate and send instructions. , information receiving, and image information display functions are used to support the monitoring system to achieve such as geographic positioning and regional information display; the ground control end receives the status information, full-frame sampling images, roaming window images, target identification information and target tracking information of the three-axis stabilized platform, and performs real-time image feature matching on all the above-mentioned received information and the monitoring area image provided by its own geographic information system to achieve image geographic alignment, and generate full-frame sampling images and roaming window images with geographic information to send to the display unit to form a monitoring area image display; the ground control end transmits the target identification information and target tracking information to the information calculation unit for processing, and realizes the spatiotemporal alignment of the target and the display unit image, which is superimposed on the display image of the display unit; the ground control end operator realizes closed-loop control of the three-axis stabilized platform based on the status information of the three-axis platform and the display content of the display unit.

[0058] The air-to-ground radio station of the embodiment of the present invention includes an air-to-ground radio station and a ground radio station, which are used to complete downlink information transmission from air to ground and uplink information transmission from ground to air. Preferably, the transmission interface is in the format of Ethernet, serial port RS422, SDI, etc.

[0059] The embodiment of the present invention preferably uses a 10G switch as a node for mass data distribution, and the mass data generated by the ultra-high resolution camera is transmitted to the image processing module through an optoelectronic confluence ring for information processing. In the image processing module of the embodiment of the present invention, the computer board is responsible for controlling the working mode of the entire system, coordinating multiple tasks such as the camera, the stable platform, the servo control, information processing and information transmission, and the distributed information processing board receives the roaming window image, realizes the tracking and prediction of multiple targets in the window image through the association of multi-frame image information, and generates target tracking information; the distributed information processing board transmits the target identification meta-information and target tracking information to the computer board for information fusion and application distribution; the number of distributed information processing boards is expanded to information processing board 1 to information processing board n according to the use requirements, and is responsible for the automatic parallel processing of massive images, and the intelligent recognition algorithm includes YOLO, SSD, etc., the moving target detection algorithm includes optical flow detection, registration image difference, etc., the target tracking algorithm includes MOSSE, KCF, etc., and the multi-target tracking algorithm includes Kalman filtering, etc. The processed information and image videos are distributed to the storage board and the airborne radio station as needed via the switch. The images and information required by the ground station are transmitted from the airborne radio station to the ground radio station and then enter the ground control terminal for post-processing and display.

[0060] Please continue reading Figure 1 The satellite combined inertial navigation system in the optoelectronic regional monitoring system of the embodiment of the present invention is controlled by the computer board in the image processing module, which can receive the global satellite navigation system capabilities, provide the monitoring system with air platform positioning and system reference time acquisition, and work with the computer board to realize the system positioning and timing, and mark all the image data and information generated with time, providing information calculation time base for distributed information processing and air-ground collaborative post-processing.

[0061] The ground control terminal of the embodiment of the present invention includes an information computing unit, a display unit, a geographic information system, and an image-based computing server, and has the functions of generating and sending instructions, receiving information, and displaying image information, and is used to support the monitoring system to achieve such as geographic positioning and regional information display.

[0062] Furthermore, the ground control terminal of the embodiment of the present invention, as the final node for information processing and display, preferably includes an information computing unit, a display unit, a geographic information system, and a computing server, which are used to support the monitoring system to implement such as geographic positioning and regional information display.

[0063] The optoelectronic area monitoring system of the embodiment of the present invention can realize large-scale monitoring of the ground area along with the flight mission of the aerial platform, and can realize monitoring of fixed ground areas while hovering in the air, flying back and forth in a straight line, and circling in the air. The monitoring performance achieved under different movement modes of the aerial platform is slightly different, but the basic monitoring system realizes large-scale regional imaging and roaming window imaging through the cooperation of the ultra-high resolution camera and the three-axis stabilized platform. The image processing module processes the massive images received according to different modes, and the processed information is transmitted to the ground control end to complete the secondary processing of the information, thereby completing large-scale area monitoring.

[0064] See also Figure 3 The optoelectronic regional monitoring system of the embodiment of the present invention receives the command from the ground control end and controls the three-axis stabilized platform to always point to the specified geographical location when the platform is hovering or moving. The ultra-high resolution camera on the platform performs real-time full-frame imaging and roaming window imaging in the specified area. The image processing module processes the two types of imaging data separately after receiving them. When the platform is hovering and circling in the air, the three-axis stabilized platform senses the changes in the spatial position and posture of the aerial platform through the satellite combined inertial navigation and inertial devices, and the computer board in the image processing module controls compensation to achieve stable geographical pointing, providing a stable imaging environment for the imaging of the ultra-high resolution camera. When the platform is making a straight line back and forth or a large maneuver, the regional monitoring task can only be carried out within the limited angle range of the pitch and roll of the three-axis stabilized platform.

[0065] See also Figure 4 and Figure 5 The photoelectric area monitoring system of an embodiment of the present invention receives instructions from the ground control terminal and points to different geographical locations at different times through time-division control when the platform is hovering or moving. Each designated geographical location corresponds to a monitoring sub-area. The photoelectric area monitoring system repeatedly traverses in any predetermined order to form a larger monitoring area. The large monitoring area formed by splicing can be a seamless splicing of multiple sub-areas, or it can be any combination of multiple sub-areas with no relationship between boundaries.

[0066] In the above two monitoring modes, the image processing module processes the two types of imaging data separately after receiving them; the image processing module processes the full-frame imaging and roaming window imaging in the time-sharing processing, and transmits the formed image video and information to the ground control end for secondary information processing, and completes the target frame superposition and display. The ground control end provided in this embodiment is mainly used for the control of the optoelectronic regional monitoring system and the comprehensive display of image target information. It uses the system's own aerial full-frame image, the digital map of the monitoring area or the aerial remote sensing ortho-mosaic map as a map, combined with the geographical location information of the geographic information system (GIS), and comprehensively processes various sub-images, videos, inertial navigation information and target information transmitted by the airborne radio station, completes multiple image matching, positioning solution, information fusion and display, and finally realizes high frame rate real-time video and target information of the area of ​​interest within the regional monitoring range, and low-frequency target information status display in a large area, and supports multiple users to use roaming windows to perform key human-machine combined monitoring of key areas or target prompt areas.

[0067] In addition, an embodiment of the present invention also provides an optoelectronic wide-area monitoring method suitable for an aerial platform. The optoelectronic wide-area monitoring method is based on the above-mentioned optoelectronic wide-area monitoring system. The optoelectronic wide-area monitoring method includes: a three-axis stabilized platform points to a specified geographical location, and an ultra-high-resolution camera performs real-time full-frame imaging and roaming window imaging of the specified area; an image processing module receives and processes real-time full-frame imaging, and receives and processes roaming window imaging to obtain image information; the image information is transmitted to a ground control terminal via an air-to-ground radio station, and the command output by the ground control terminal is transmitted to the image processing module or the three-axis stabilized platform.

[0068] The optoelectronic area monitoring system of the embodiment of the present invention performs large-scale monitoring of the ground area along with the aerial platform flight mission action. The aerial platform flight mode includes at least multiple modes such as hovering in the air, round-trip straight flight, and circling in the air. Based on different flight routes, the system realizes large-scale regional imaging and roaming window imaging through the cooperation of an ultra-high-resolution camera and a three-axis stabilized platform. The image processing module processes the massive images received according to different modes. The processed information is transmitted to the ground control end to complete secondary processing of the information, thereby completing large-scale area monitoring.

[0069] The above-mentioned image processing module of the embodiment of the present invention receives and processes real-time full-frame imaging, and receives and processes roaming window imaging, including: the image processing module receives real-time full-frame imaging, performs intelligent target recognition and extraction or moving target extraction on the full-frame image, and transmits the recognition sub-image, recognition type, position and other meta-information in the full-frame imaging to the ground control end for target display superposition; the image processing module receives roaming window imaging in real time, performs intelligent target recognition and extraction or moving target extraction on the roaming window imaging, and transmits the real-time window image, recognition type, position and other meta-information in the roaming window imaging to the ground control end for secondary information processing, and completes the target frame superposition and display.

[0070] The three-axis stabilized platform of the embodiment of the present invention always points to the specified geographical location, and the ultra-high resolution camera on the platform performs real-time full-frame imaging and roaming window imaging of the specified area. The image processing module processes the two types of imaging data separately after receiving them; intelligent target recognition and extraction or moving target extraction are performed on the full-frame image, and various meta-information such as the recognition sub-image, recognition type, and position are transmitted to the ground control end for target display superposition; the roaming window resolution is much smaller than the full-frame image resolution, and the roaming window frame rate is higher than the full-frame image frame rate. The roaming window can be controlled to open a window at any position of the full-frame image to achieve imaging of any position in the area corresponding to the full-frame image, and intelligent target recognition or moving target extraction is performed on the roaming window image, and various meta-information such as the window real-time image, recognition type, and position are transmitted to the ground control end for secondary information processing, and the target frame superposition and display are completed.

[0071] The above-mentioned three-axis stabilized platform of the embodiment of the present invention points to a specified geographical location, including: the three-axis stabilized platform points to different geographical locations at different times through time-division control, and each specified geographical location corresponds to a monitoring sub-area; the optoelectronic wide-area monitoring system repeatedly traverses in an arbitrary predetermined order to form a larger monitoring area, and the high-resolution camera performs time-division full-frame imaging and roaming window imaging in the specified larger area, and the image processing module processes the two types of imaging data separately after receiving them; the image processing module processes the full-frame imaging and roaming window imaging and forms image video and information, which is transmitted to the ground control end for secondary information processing to complete the target frame superposition and display.

[0072] The three-axis stabilized platform of the embodiment of the present invention points to different geographical locations at different times through time-division control, and each designated geographical location corresponds to a monitoring sub-area. The optoelectronic area monitoring system repeatedly traverses in an arbitrary predetermined order to form a larger monitoring area, and the high-resolution camera thereon performs time-division full-frame imaging and roaming window imaging in the designated larger area. The image processing module receives the two types of imaging data and processes them separately; in the time-division processing, the image processing module processes the full-frame imaging and roaming window imaging, transmits the formed image video and information to the ground control end for secondary information processing, and completes the target frame superposition and display.

[0073] The command command output by the above-mentioned ground control end of the embodiment of the present invention is transmitted to the image processing module or the three-axis stabilized platform, including: the ground control end is equipped with a regional digital map or an aerial remote sensing orthophoto mosaic, and has a geographic information system; the ground control end collects the information transmitted by the aerial image processing module, the information includes video, target information and the real-time position information of the platform provided by the satellite combined inertial navigation and the time scale information after time synchronization; the ground control end performs image matching, positioning solution and information fusion on the information to complete the positioning and display superposition of the transmitted target; the ground control end displays and controls multiple roaming windows for multiple users to simultaneously monitor multiple arbitrary positions in the monitoring area in real time.

[0074] The ground control end of the embodiment of the present invention is equipped with a regional digital map or an aerial remote sensing ortho-mosaic map and has a geographic information system (GIS) capability; the ground control end collects video and target information transmitted from the air, and the real-time position information of the platform provided by the satellite combined inertial navigation and the time scale information after time synchronization; the ground control end performs image matching, positioning solution, and information fusion on the above information to complete the positioning and display superposition of the transmitted target; the ground control end displays and controls multiple roaming windows, so that multiple users can simultaneously perform real-time gaze monitoring of multiple arbitrary locations in the monitoring area.

[0075] The embodiment of the present invention uses a single-chip ultra-high resolution detector to realize the full-frame image acquisition and processing of regional monitoring under a single camera and the output of the roaming window, solving the current problems of large volume, heavy weight and poor real-time performance caused by multi-sensor splicing and spatial time-sharing scanning. The image processing module adopts a distributed and open architecture, integrates the required information processing board according to demand, and uses artificial intelligence recognition and moving target detection algorithms to complete the processing of massive data in the air, and only transmits useful information to the ground control terminal for monitoring tasks, solving the contradiction between the limited communication bandwidth of general aerial platforms and the transmission of massive data for regional monitoring.

[0076] In order to further adapt to the complex situations of regional monitoring tasks in reality, the embodiments of the present invention propose two monitoring efficiency modes, which can use single designated area staring and multi-area time-sharing traversal according to different monitoring needs, and provide a balance method between the contradiction between real-time and monitoring area scope, so that the photoelectric regional monitoring system is more adaptable, the monitoring method is flexible and diverse, and the monitoring efficiency is higher.

[0077] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. An optoelectronic wide-area surveillance system suitable for an aerial platform, characterized in that: include: A three-axis stabilized platform for collecting massive image data, a ground control terminal for control and display, and an air-to-ground radio station for completing uplink and downlink information transmission between the three-axis stabilized platform and the ground control terminal; wherein: The three-axis stabilized platform is equipped with an ultra-high-resolution camera, a slip ring and an image processing module. The ultra-high-resolution camera generates a large amount of image data when monitoring the ground, which is divided into multiple channels and enters multiple parallel data channels accordingly. The data is transmitted to the image processing module through the slip ring, and the image processing module processes the large amount of images into image information in real time. The image information is transmitted to the ground control terminal via downlink information of the air-to-ground radio station, and the instruction command output by the ground control terminal is transmitted to the image processing module via uplink information of the air-to-ground radio station.

2. The optoelectronic wide-area surveillance system suitable for an aerial platform according to claim 1, characterized in that: The three-axis stabilized platform is installed on the aerial platform, and the three-axis stabilized platform includes an upper cabin and a lower cabin, the image processing module is arranged in the upper cabin, a satellite combined inertial navigation module and a 10 Gigabit switch are also arranged in the upper cabin, and the ultra-high resolution camera is arranged in the lower cabin; The three-axis stable platform provides a stable pointing and imaging environment for the ultra-high resolution camera, and the slip ring is installed on the three-axis stable platform; The three-axis stabilized platform has the ability to adjust the pointing and stabilize the azimuth, pitch and roll sight axes, and the merge ring can achieve 360° continuous rotation.

3. The optoelectronic wide-area monitoring system suitable for an aerial platform according to claim 1, characterized in that: The ultra-high resolution camera adopts a single-chip high-resolution imaging chip, and the ultra-high resolution camera is equipped with a high-resolution large-field optical lens; The ultra-high resolution camera provides regional monitoring images and multiple roaming windows, and the ultra-high resolution camera performs real-time full-frame imaging and roaming window imaging in a designated area of ​​ground monitoring; The image processing module receives real-time full-frame imaging and roaming window imaging, and performs intelligent target recognition and extraction or moving target extraction on the real-time full-frame imaging and transmits it to the ground control end; the image processing module performs intelligent target recognition or moving target extraction on the roaming window imaging and transmits it to the ground control end.

4. The optoelectronic wide-area surveillance system suitable for an aerial platform according to claim 1, characterized in that: The air-to-ground radio station includes an airborne radio station and a ground radio station. The airborne radio station communicates with the ground radio station. The image information is transmitted to the ground control terminal via the downlink information of the airborne radio station. The command output by the ground control terminal is transmitted to the image processing module via the uplink information of the ground radio station.

5. The optoelectronic wide-area surveillance system suitable for an aerial platform according to claim 1, characterized in that: The image processing module divides the massive image data into multiple paths, which are then transmitted to multiple information processing boards through corresponding multiple parallel data channels; The image processing module is installed on the aerial platform, and the number of computing boards is expanded according to demand to increase processing capacity; the image processing module includes a computer board, a distributed information processing board, a 10G switch, and a storage board; Its functions include at least command reception, image preprocessing, target detection, image storage and image compression transmission. It has the ability to process massive images, process images output by ultra-high-resolution cameras including regional images and roaming windows in real time, encode and compress images, information and system status that need to be transmitted to the airborne radio station, and receive commands from the ground control terminal through the airborne radio station. The distributed information processing board receives the full-width massive image, divides it into smaller image blocks by overlapping and cutting according to the width of the full-width massive image, and directly sends the image blocks to the detection and reasoning software unit on the distributed information processing board, and uses the artificial intelligence algorithm to quickly complete the processing of the massive image blocks, so as to obtain the image content information such as the type, size, and position of the target in the image, and generate target recognition meta-information including the target sub-image; the distributed information processing board receives the roaming window image, and realizes the tracking and prediction of multiple targets in the window image through the information association of multiple frames of roaming window images, and generates target tracking information; the distributed information processing board transmits the target recognition meta-information and target tracking information to the computer board for information fusion and application distribution; The ground control terminal includes an information computing unit, a display unit, a geographic information system, and an image-based computing server, and has the functions of generating and sending instructions, receiving information, and displaying image information, and is used to support the monitoring system to achieve geographic positioning and regional information display; The ground control end receives the status information, full-frame sampling images, roaming window images, target identification information and target tracking information of the three-axis stabilized platform, performs real-time image feature matching on all the above-mentioned received information and the monitoring area image provided by its own geographic information system, realizes image geographic alignment, generates full-frame sampling images and roaming window images with geographic information, and sends them to the display unit to form a monitoring area image display; after the ground control end transmits the target identification information and target tracking information to the information calculation unit for processing, the time-space alignment of the target and the display unit image is realized, and the images are superimposed on the display image of the display unit; the ground control end operator realizes closed-loop control of the three-axis stabilized platform based on the status information of the three-axis platform and the display content of the display unit.

6. An optoelectronic wide-area monitoring method suitable for an aerial platform, characterized in that: Based on the optoelectronic wide-area monitoring system according to any one of claims 1 to 5, the optoelectronic wide-area monitoring method comprises: The three-axis stabilized platform points to the specified geographic location, and the ultra-high-resolution camera performs real-time full-frame imaging and roaming window imaging of the specified area; The image processing module receives and processes the real-time full-frame imaging, and receives and processes the roaming window imaging, to obtain image information; The image information is transmitted to the ground control terminal through the air-to-ground radio, and the command output by the ground control terminal is transmitted to the image processing module or the three-axis stabilized platform.

7. The optoelectronic wide-area monitoring method applicable to an aerial platform according to claim 6, characterized in that: The image processing module receives and processes real-time full-frame imaging, and receives and processes roaming window imaging, including: The image processing module receives the real-time full-frame imaging, performs intelligent target recognition and extraction or moving target extraction on the full-frame image, and transmits the identification sub-image, identification type, position and other information in the full-frame imaging to the ground control end for target display superposition; The image processing module receives the roaming window imaging in real time, performs intelligent target recognition and extraction or moving target extraction on the roaming window imaging, transmits the real-time image of the window, recognition type, position and other meta-information in the roaming window imaging to the ground control end for secondary information processing, and completes the target frame superposition and display.

8. The optoelectronic wide-area monitoring method applicable to an aerial platform according to claim 6, characterized in that: The three-axis stabilized platform points to a specified geographic location, including: The three-axis stabilized platform is pointed to different geographical locations at different times through time-division control, and each designated geographical location corresponds to a monitoring sub-area; The optoelectronic wide-area monitoring system repeatedly traverses in any predetermined order to form a larger monitoring area. The high-resolution camera performs time-sharing full-frame imaging and roaming window imaging in the designated larger area. The image processing module receives the two types of imaging data and processes them separately. The image processing module processes the full-frame imaging and roaming window imaging to form image videos and information, which are then transmitted to the ground control end for secondary information processing to complete target frame overlay and display.

9. The optoelectronic wide-area monitoring method applicable to an aerial platform according to claim 6, characterized in that: The command output by the ground control terminal is transmitted to the image processing module or the three-axis stabilized platform, including: The ground control terminal is equipped with regional digital maps or aerial remote sensing orthophoto mosaics and a geographic information system; The ground control end collects the information transmitted by the aerial image processing module, including video, target information, real-time platform position information provided by satellite combined inertial navigation, and time stamp information after timing. The ground control end performs image matching, positioning solution, and information fusion on the information to complete the positioning and display superposition of the downlinked target; The ground control terminal displays and controls multiple roaming windows, allowing multiple users to simultaneously monitor multiple arbitrary locations within the monitoring area in real time.

Images