Photoelectric radar integrated detection method and system based on target autonomous identification

Abstract

The invention relates to a photoelectric radar integrated detection method and system based on target autonomous identification, and relates to the technical field of photoelectric system and radar system integrated detection. The method comprises the following steps: step 1, area selection; 2, preset photographing point position conversion is carried out; step 3, carrying out azimuth assembly sector scanning; step 4, splicing module processing; 5, processing by an information fusion identification module; step 6, processing by a display and decision module; and step 7, photoelectric video tracking mode switching. According to the photoelectric radar integrated detection method and system based on autonomous target identification, the authenticity and tracking priority of the target are judged through combination of the target detected by the radar and the target identification result of the photoelectric system on the wide-width wide-angle image shot by the photoelectric system, the tracking control precision of the photoelectric system is improved, the radar false alarm rate is reduced, and the detection accuracy of the photoelectric system is improved. The guiding efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated detection of optoelectronic systems and radar systems, and in particular to an optoelectronic radar integrated detection method and system based on autonomous target recognition. Background Art

[0002] With the increasing demand for security protection in China, various security measures are widely used in personal safety, security facilities, prisons, coastlines and border control, etc. Common security measures include human area patrols, electronic fence warnings and photoelectric radar area detection, etc. These security measures can meet the needs of security area early warning, target intrusion tracking and threat target disposal.

[0003] In the field of optoelectronic and radar integrated detection systems, the combined use of optoelectronic systems and radar systems effectively solves the limitations of all-day and all-weather applications, gives full play to the advantages of individual devices, makes up for the defects of individual devices, and has better performance than human area patrols and electronic fence warnings. It is widely used in important areas, perimeters of important facilities, border defense, coastal defense, etc. The radar is responsible for discovering targets in the detection area and obtaining information such as the target's position and speed, and further guides the optoelectronic system to complete the acquisition, capture and tracking of the target image, thereby comprehensively realizing the precise positioning and identification of the target.

[0004] In response to the above needs, there are currently many relevant solutions at home and abroad.

[0005] Chinese patent document CN202410606505.6 discloses a target detection, tracking and identification method and system based on multi-data fusion. The invention uses infrared and radar to perform real-time infrared monitoring and radar detection, obtain infrared monitoring data and radar detection data, and perform data preprocessing to generate infrared processing data and radar processing data; perform target detection on the infrared processing data and the radar processing data to determine whether there is a suspicious target to be detected; when there is a suspicious target to be detected, determine the direction of the suspicious target according to the infrared processing data and the radar processing data, perform tracking shooting and preprocessing, and obtain shooting processing data; according to the shooting processing data, confirm and identify the suspicious target to be detected to determine whether there is a confirmed target to be detected; when there is a confirmed target to be detected, the infrared processing data, radar processing data and shooting processing data are combined to determine whether there is an abnormal action, and when there is an abnormal action, an abnormal action alarm is issued. However, the invention is only applicable to multi-target capture at close range and narrow field of view, and the effect will be significantly reduced for long distance and wide-angle field of view.

[0006] Chinese patent document CN202110227211.9 discloses a camera and laser radar time synchronization control system, which is a camera and laser radar time synchronization control system, including: a camera, a laser radar and an FPGA, the FPGA is provided with an image acquisition module, a clock module and a signal delay module, wherein: the clock module is used to receive the laser radar clock output by the laser radar, and the laser radar clock is used as the working clock of the camera; the signal delay module includes a delay unit and an asynchronous FIFO unit, the delay unit is used to receive the field synchronization signal output by the camera and forward the field synchronization signal to the asynchronous FIFO unit after delaying a preset interval, the asynchronous FIFO unit generates a frame synchronization signal according to the field synchronization signal, and the frame synchronization signal is used to wake up the laser radar to enter the working state; the image acquisition module is used to receive the image data collected by the camera. The invention realizes high-precision clock and frame rate synchronization between the camera and the laser radar at a low cost. However, when the clock synchronization between the camera and the radar is performed through the signal delay module, there will still be a communication signal delay, which is not applicable to situations where real-time synchronization is required.

[0007] In general, the optoelectronic and radar integrated detection systems in the prior art have the following problems in the process of discovering and identifying targets in actual applications:

[0008] (1) Regardless of whether mechanical radar or phased array radar is used, the radar system is used for large-angle detection (typical value ≥ 90°). Due to its own working mechanism, the scanning and processing speed leads to a low data refresh rate (1-2 Hz). The target position information obtained by the radar directly guides the optoelectronic system to track and control with low accuracy, and the picture is shaky and not smooth.

[0009] (2) When used in complex scenarios, radar scanning will be affected by clutter, such as waves, trees and vegetation in windy conditions, which may cause radar detection errors. Radar parameter adjustment for specific scenarios is cumbersome. In order to ensure an extremely low false alarm rate, clutter interference cannot be completely eliminated, resulting in false alarms in display alarms and guidance photoelectric displays.

[0010] (3) The system generally uses a long focal length narrow field of view optoelectronic imaging component and is equipped with a two-dimensional optoelectronic turntable to meet the needs of long-distance image recognition and tracking. Even when the radar detects multiple real targets, it is necessary to guide the optoelectronic system one by one to make judgments and tracks. At the same time, there is also the problem of repeated switching of targets for identification, resulting in low guidance efficiency. Summary of the invention

[0011] The present invention aims to solve the technical problems in the prior art and provide a photoelectric radar integrated detection method and system based on autonomous target identification.

[0012] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0013] An integrated detection method of photoelectric radar based on target autonomous identification includes the following steps:

[0014] Step 1: Region selection;

[0015] Calibrate the coordinate systems of the optoelectronic system and radar system, determine the sector scanning boundary of the optoelectronic system through the area type set on the electronic map, and convert the optical focal length and azimuth speed according to the image splicing overlap rate and image recognition pixel number requirements;

[0016] Step 2: Preset the photo point conversion;

[0017] Calculate the optical focal length, the field of view of the optoelectronic system, determine the number of times the optoelectronic system takes photos, and determine the photo taking position of the optoelectronic system;

[0018] Step 3: Azimuth component fan scanning;

[0019] Determine the scanning rotation speed of the optoelectronic system and trigger the photo-taking;

[0020] Step 4: splicing module processing;

[0021] Based on the images obtained by taking photos, the image overlap rate parameter is used to remove overlapping images, and the parts beyond the radar detection area are calculated and removed. Then, the processed images are registered and spliced ​​to form a wide-angle image with a long-range view to meet the requirements of target detection.

[0022] Step 5: Information fusion recognition module processing;

[0023] Target detection is performed on the stitched wide-angle images of the distant view. The target type and position information in the field of view are fused and identified with the target position information provided by the radar. Targets in the same direction and angle are regarded as common real targets, and other targets are regarded as false targets and eliminated. The fused information is sent to the display and decision module.

[0024] Step 6: Display and decision module processing;

[0025] The radar electronic map will be synchronously superimposed and updated with the real target type, coordinate position, and speed information, and all invalid interference targets will be filtered out; if there are multiple real targets, the target threat level will be comprehensively judged based on the distance from the target to the edge of the radar defense zone / protection area, the target movement speed, and the target type information, and the most threatening target will be selected, and autonomous decision-making will be made to provide target priority guidance information for the optoelectronic video tracking mode;

[0026] Step 7: Switch the photoelectric video tracking mode;

[0027] The optoelectronic system switches to the target area and turns on the image tracker based on the guidance information of the target to be tracked, locking and continuously tracking the target.

[0028] In the above technical solution, the area types in step 1 include: the entire area and sensitive areas.

[0029] In the above technical solution, step 2 is specifically as follows:

[0030] First calculate the optical focal length:

[0031]

[0032] Among them, f is the optical focal length, PPT is the number of pixels required for image recognition, S is the target distance, Pixel size is the pixel size of the camera sensor, and h is the target size;

[0033] Then calculate the field of view of the optoelectronic system:

[0034]

[0035] Where θ0 is the field of view of the optoelectronic system, P 宽 is the width of the camera sensor, and f is the optical focal length;

[0036] Determine the number of times the optoelectronic system takes photos:

[0037]

[0038] Where M is the number of times the optoelectronic system takes pictures, and R is the image stitching overlap rate. In the case of sensitive area division, the radar field of view angle θ r The angle formed by the two points on the left and right boundaries of the sensitive area is replaced, and θ0 is the field of view of the optoelectronic system;

[0039] Determine the photo-taking position of the optoelectronic system. In the case of sensitive area division, the radar field of view angle θ r The angle formed by the two points on the left and right boundaries of the sensitive area will be replaced;

[0040] The position of the photoelectric system when scanning from left to right is:

[0041]

[0042] When the photoelectric system scans from right to left, the position is:

[0043]

[0044] Among them, N is 0, 1, ..., M in sequence, R is the image stitching overlap rate, and θ0 is the field of view of the optoelectronic system.

[0045] In the above technical solution, step 3 is specifically as follows:

[0046] Determine the scanning rotation speed of the optoelectronic system:

[0047]

[0048] Where v0 is the scanning rotation speed of the photoelectric system, t r is the radar refresh cycle; in the case of sensitive area division, the radar field of view angle θ r The angle formed by the two points on the left and right boundaries of the sensitive area is replaced, and θ0 is the field of view angle of the optoelectronic system.

[0049] In the above technical solution, step 4 is specifically as follows:

[0050] Based on the M images obtained from M shots, the image overlap ratio parameter R is used to remove overlapping images, and the formula M×θ0-θ r The parts beyond the radar detection area are calculated and eliminated, and then the processed pictures are aligned and spliced ​​to form a wide-angle image with a long-range view to meet the requirements of target detection.

[0051] A photoelectric radar integrated detection system based on target autonomous identification is applicable to the above-mentioned photoelectric radar integrated detection method, wherein the radar system and the photoelectric system are coaxially arranged.

[0052] The present invention has the following beneficial effects:

[0053] The optoelectronic radar integrated detection method and system based on autonomous target identification of the present invention, the combined use of the optoelectronic system and the radar system, effectively solves the limitations of all-day and all-weather application. The radar detects the target and obtains the target position and speed information, and further guides the optoelectronic system to complete the acquisition, capture and tracking of the target image, thereby comprehensively realizing the accurate positioning and identification of the target, and is widely used in scenarios such as border and sea defense and facility perimeter security.

[0054] The optoelectronic radar integrated detection method and system based on autonomous target identification of the present invention combines the target detected by the radar and the target identification result of the optoelectronic system on the wide-width and wide-angle image pictures taken by the radar to judge the authenticity and tracking priority of the target, improve the tracking control accuracy of the optoelectronic system, reduce the radar false alarm rate, and improve the guidance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0056] Figure 1 The figure is a schematic flow chart of the steps of the photoelectric radar integrated detection method based on autonomous target identification of the present invention.

[0057] Figure 2 Schematic diagram of the spatial coordinates of the optoelectronic system and radar system.

[0058] Figure 3 Schematic diagram of the field of view and trigger matching relationship between the optoelectronic system and the radar system in the full working range.

[0059] Figure 4 Schematic diagram of the field of view and trigger matching relationship between the optoelectronic system and the radar system to delineate the working range of sensitive areas.

[0060] Figure 5 Schematic diagram of target information fusion and pseudo target elimination. DETAILED DESCRIPTION

[0061] The inventive concept of the present invention is:

[0062] The optoelectronic radar integrated detection method and system based on autonomous target identification of the present invention combines the target detected by the radar system and the target identification results of the optoelectronic system on the wide-width and wide-angle image pictures taken by the radar system to judge the authenticity and tracking priority of the target, improve the tracking control accuracy of the optoelectronic system, reduce the radar false alarm rate, and improve the guidance efficiency.

[0063] The present invention is described in detail below with reference to the accompanying drawings.

[0064] The optoelectronic radar integrated detection method based on target autonomous identification of the present invention is mainly used in the inspection and identification mode of the optoelectronic and radar integrated monitoring system, and completes the target discovery and identification by combining radar scanning with optoelectronic fan scanning.

[0065] The optoelectronic radar integrated detection method based on target autonomous identification of the present invention is suitable for the situation where the radar system and the optoelectronic system are coaxially installed.

[0066] The process of the photoelectric radar integrated detection method based on target autonomous identification of the present invention is as follows: Figure 1 As shown. The optoelectronic imaging system has two working modes: triggered photo taking and video preview. In the triggered photo taking mode, the optoelectronic imaging component works in a long focal length fixed focus state that meets the system working distance, and can meet the basic requirements of image recognition at the longest detection distance of the radar's entire range or the designated sensitive area. Both the camera and the radar have built-in independent GPS clock synchronization devices to handle the synchronization of the scanning frequency, synchronizing the time information once per second to ensure that the camera's photo taking time accurately matches the radar scanning cycle. Within a refresh cycle of the radar, the optoelectronic imaging component is triggered at a specific time and position to take a photo and acquire an image. In the video preview mode, the optoelectronic imaging component and the image tracking component are used together, mainly for tracking and collecting evidence of specific targets.

[0067] The optoelectronic system azimuth component fan-scans, and the fan-scan frequency is consistent with the radar scanning refresh frequency, that is, within one refresh cycle, the optoelectronic system azimuth component completes a unidirectional fan-scan motion consistent with the radar's horizontal working angle range or the set sensitive area boundary angle range; within two refresh cycles, the optoelectronic system azimuth component realizes one round-trip motion.

[0068] At the same time, during the azimuth fan scanning process, the imaging component of the optoelectronic system converts the angle of rotation of the optical axis according to a certain field of view overlap rate (such as a 1 / 4 overlap rate between the front and rear frames), triggers photography at relevant positions, and sends the images to the image stitching module for registration, stitching, and cropping to form a wide-angle image.

[0069] The spliced ​​single-frame image is sent to the information fusion target recognition module, which uses the target detection algorithm to identify the target type and position, and compares and fuses it with the radar scanning target. The system will conduct a comprehensive assessment of the danger level based on the target type, position, speed, etc., import the target into the subsequent module, switch to the pure optoelectronic video tracking mode, and conduct tracking and evidence collection.

[0070] The photoelectric radar integrated detection method based on target autonomous identification of the present invention comprises the following steps:

[0071] Step 1: Region selection;

[0072] This step involves the calibration of the coordinate systems of the optoelectronic system and the radar system. The sector scanning boundary of the optoelectronic system is determined by the area type (global / sensitive area) set on the electronic map. The optical focal length, azimuth speed, etc. are converted according to the image stitching overlap rate and the number of image recognition pixels required.

[0073] According to the coaxial installation of the radar system and the optoelectronic system, it can be simplified as follows: Figure 2 coordinate system, where θ0 is the field of view of the optoelectronic system, θ r is the field of view of the radar system. The red area is the scanning area of ​​the optoelectronic system, and the blue area is the detection area of ​​the radar system.

[0074] exist Figure 3 In the full-domain working range, the initial coverage area of ​​the optoelectronic system is FOV1, and the coverage area after the first rotation is FOV2. The overlapping area of ​​the field of view of the two rotations is the field of view overlap rate (Overlap). After M rotations, the entire radar detection area is covered.

[0075] exist Figure 4 Within the working range of the sensitive area, the area that the photoelectric system needs to cover is the area enclosed by the red straight line AB. If there are multiple sensitive areas, they will rotate in sequence according to the number of sensitive areas and the refresh cycle.

[0076] Step 2: Preset the photo point conversion;

[0077] First calculate the optical focal length:

[0078]

[0079] Among them, f is the optical focal length, PPT is the number of pixels required for image recognition, S is the target distance, Pixel size is the pixel size of the camera sensor, and h is the target size.

[0080] Then calculate the field of view of the optoelectronic system:

[0081]

[0082] Where θ0 is the field of view of the optoelectronic system, P 宽 is the width of the camera sensor and f is the optical focal length.

[0083] Determine the number of times the optoelectronic system takes photos:

[0084]

[0085] Where M is the number of times the optoelectronic system takes pictures, and R is the image stitching overlap rate. In the case of sensitive area division, the radar field of view angle θ r The angle formed by the two points on the left and right boundaries of the sensitive area (such as Figure 4 ∠AB) in the figure is replaced by θ0, and θ0 is the field of view of the optoelectronic system.

[0086] Determine the photo-taking position of the optoelectronic system. In the case of sensitive area division, the radar field of view angle θ r The angle formed by the two points on the left and right boundaries of the sensitive area (such as Figure 4 is replaced by ∠AB) in the figure.

[0087] The position of the photoelectric system when scanning from left to right is:

[0088]

[0089] When the photoelectric system scans from right to left, the position is:

[0090]

[0091] R is the image stitching overlap ratio, and θ0 is the field of view of the optoelectronic system.

[0092] Step 3: Azimuth component fan scanning;

[0093] Determine the scanning rotation speed of the optoelectronic system:

[0094]

[0095] Where v0 is the scanning rotation speed of the photoelectric system, t ris the radar refresh cycle. In the case of sensitive area division, the radar field of view angle θ r The angle formed by the two points on the left and right boundaries of the sensitive area (such as Figure 4 ∠AB) in the figure is replaced by θ0, and θ0 is the field of view of the optoelectronic system.

[0096] To ensure the synchronization of the scanning frequency of the optoelectronic system and the radar system, the camera and radar are equipped with independent GPS clock synchronization devices and processors, which use the radar time as the benchmark to calibrate the time information of the optoelectronic system. When the system is running, the time information is synchronized once a second to ensure that the camera shooting time accurately matches the radar scanning cycle.

[0097] Step 4: splicing module processing;

[0098] Based on the M images obtained from M shots, the image overlap ratio parameter R is used to remove overlapping images, and the formula M×θ0-θ r The parts beyond the radar detection area are calculated and eliminated, and then the processed pictures are registered and spliced ​​to form a wide-angle image with a long-range view, which can meet the requirements of target detection.

[0099] Step 5: Information fusion recognition module processing;

[0100] like Figure 5 As shown in the figure, the wide-angle image with a wide view after stitching is used for target detection. The target type and position information in the field of view are fused and identified with the target position information provided by the radar. Targets in the same direction and angle are regarded as common real targets, and other targets are regarded as false targets and eliminated. The fused information is sent to the next level module.

[0101] Step 6: Display and decision module processing;

[0102] The radar electronic map will be refreshed synchronously (the refresh rate is the same as the radar refresh cycle t r The real target type, coordinate position, speed and other information are kept consistent, and all invalid interference targets are filtered out. If there are multiple real targets, the target threat level can be comprehensively judged based on the distance of the target from the edge of the radar defense zone / protection area, the target movement speed, the target type and other information, and the most threatening target can be selected, and autonomous decision-making can be made to provide target priority guidance information for the optoelectronic video tracking mode.

[0103] Step 7: Switch the photoelectric video tracking mode;

[0104] The optoelectronic system switches to the target area and turns on the image tracker according to the target guidance information to be tracked, locks and continuously tracks the target. The content in step 7 belongs to a typical common method, which is not the focus of the description of the present invention and will not be repeated.

[0105] The optoelectronic radar integrated detection method and system based on autonomous target identification of the present invention, the combined use of the optoelectronic system and the radar system, effectively solves the limitations of all-day and all-weather application. The radar detects the target and obtains the target position and speed information, and further guides the optoelectronic system to complete the acquisition, capture and tracking of the target image, thereby comprehensively realizing the accurate positioning and identification of the target, and is widely used in scenarios such as border and sea defense and facility perimeter security.

[0106] The optoelectronic radar integrated detection method and system based on autonomous target identification of the present invention combines the target detected by the radar and the target identification result of the optoelectronic system on the wide-width and wide-angle image pictures taken by the radar to judge the authenticity and tracking priority of the target, improve the tracking control accuracy of the optoelectronic system, reduce the radar false alarm rate, and improve the guidance efficiency.

[0107] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A photoelectric radar integrated detection method based on target autonomous identification, characterized in that: The following steps are involved: Step 1: Region selection; Calibrate the coordinate systems of the optoelectronic system and radar system, determine the sector scanning boundary of the optoelectronic system through the area type set on the electronic map, and convert the optical focal length and azimuth speed according to the image splicing overlap rate and image recognition pixel number requirements; Step 2: Preset the photo point conversion; Calculate the optical focal length, the field of view of the optoelectronic system, determine the number of times the optoelectronic system takes photos, and determine the photo taking position of the optoelectronic system; Step 3: Azimuth component fan scanning; Determine the scanning rotation speed of the optoelectronic system and trigger the photo-taking; Step 4: splicing module processing; Based on the images obtained by taking photos, the image overlap rate parameter is used to remove overlapping images, and the parts beyond the radar detection area are calculated and removed. Then, the processed images are registered and spliced ​​to form a wide-angle image with a long-range view to meet the requirements of target detection. Step 5: Information fusion recognition module processing; Target detection is performed on the stitched wide-angle images of the distant view. The target type and position information in the field of view are integrated with the target position information provided by the radar for identification. Targets in the same direction and angle are regarded as common real targets, and other targets are regarded as false targets and eliminated. Send the fused information to the display and decision-making module; Step 6: Display and decision module processing; The radar electronic map will be synchronously superimposed and updated with the real target type, coordinate position, and speed information, and all invalid interference targets will be filtered out; if there are multiple real targets, the target threat level will be comprehensively judged based on the distance from the target to the edge of the radar defense zone or the protection area, the target movement speed, and the target type information, and the most threatening target will be selected, and autonomous decision-making will be made to provide target priority guidance information for the optoelectronic video tracking mode; Step 7: Switch the photoelectric video tracking mode; The optoelectronic system switches to the target area and turns on the image tracker based on the guidance information of the target to be tracked, locking and continuously tracking the target.

2. The photoelectric radar integrated detection method based on target autonomous identification according to claim 1 is characterized in that: The area types in step 1 include: global area and sensitive area.

3. The photoelectric radar integrated detection method based on target autonomous identification according to claim 1 is characterized in that: Step 2 is as follows: First calculate the optical focal length: Where f is the optical focal length, PPT is the number of image recognition pixels required, S is the target distance, Pixel size is the pixel size of the camera sensor, and h is the target size; Then calculate the field of view of the optoelectronic system: Where θ0 is the field of view of the optoelectronic system, P 宽 is the width of the camera sensor, and f is the optical focal length; Determine the number of times the optoelectronic system takes photos: Where M is the number of times the optoelectronic system takes pictures, and R is the image stitching overlap rate. In the case of sensitive area division, the radar field of view angle θ r The angle formed by the two points on the left and right boundaries of the sensitive area is replaced, and θ0 is the field of view angle of the optoelectronic system; Determine the photo-taking position of the optoelectronic system. In the case of sensitive area division, the radar field of view angle θ r The angle formed by the two points on the left and right boundaries of the sensitive area will be replaced; The position of the photoelectric system when scanning from left to right is: The position of the photoelectric system when scanning from right to left is: Among them, N is 0, 1, ..., M in sequence, R is the image stitching overlap rate, and θ0 is the field of view of the optoelectronic system.

4. The photoelectric radar integrated detection method based on target autonomous identification according to claim 3 is characterized in that: Step 3 is as follows: Determine the scanning rotation speed of the optoelectronic system: Where v0 is the scanning rotation speed of the photoelectric system, t r is the radar refresh cycle; in the case of sensitive area division, the radar field of view angle θ r The angle formed by the two points on the left and right boundaries of the sensitive area is replaced, and θ0 is the field of view angle of the optoelectronic system.

5. The photoelectric radar integrated detection method based on target autonomous identification according to claim 4 is characterized in that: Step 4 is as follows: Based on the M images obtained from M shots, the image overlap ratio parameter R is used to remove overlapping images, and the formula M×θ0-θ r The parts beyond the radar detection area are calculated and eliminated, and then the processed pictures are aligned and spliced ​​to form a wide-angle image with a long-range view to meet the requirements of target detection.

6. An optoelectronic radar integrated detection system based on target autonomous identification applicable to the optoelectronic radar integrated detection method according to any one of claims 1 to 5, characterized in that: The radar system is coaxially arranged with the optoelectronic system.

Images