A target detection display and control method and device supporting photoelectric cross positioning in radar failure scenario

Abstract

The present invention provides a target detection display and control method and device that supports photoelectric cross positioning in radar failure scenarios; the method includes protocol conversion, target data processing, equipment health detection, multi-source radar fusion, multi-source photoelectric cross positioning target classification and recognition, and display and control management. The present invention is based on a streaming data processing framework to achieve multi-source signal fusion display when radar equipment and photoelectric equipment are working simultaneously. Relying on the advantages of photoelectric detection and combining the technology of cross positioning of multiple photoelectric devices, it can achieve target classification, recognition and tracking while achieving more accurate positioning of aerial targets, and can also use the target position data generated by photoelectric cross positioning to carry out business in the case of radar failure.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a target detection display and control method and device supporting photoelectric cross positioning in a radar failure scenario. Background Art

[0002] UAV targets have the characteristics of "low flying altitude, slow flying speed, small target RCS, strong maneuverability, and strong anti-interference ability". Traditional means of detecting them have great limitations, and detection is difficult, making it difficult to stably detect, track, and identify them.

[0003] At present, the detection methods for low-altitude UAVs at home and abroad are mainly based on radar detection, supplemented by optoelectronics, spectrum detection, etc. Although radar detection of low-altitude UAV targets can work all day and all weather, and has better detection distance and target tracking performance, it cannot achieve effective target identification, and cannot distinguish between bird targets and UAV targets. At the same time, radar cannot be used for detection in certain special scenes or terrain conditions, or radar equipment is prone to failure.

[0004] At this time, it is necessary to utilize multi-source photoelectric detection methods, relying on the advantages of photoelectric detection and combining the technology of cross-positioning of multiple photoelectric detection devices to achieve more accurate positioning of aerial targets while realizing advanced functions such as video tracking and classification recognition of targets.

[0005] Photoelectric detection equipment can not only provide video signals, optical information, and visual features of detected objects, but also rely on its integrated turntable, direction controller, field of view management device, etc. to provide some relative position information of aerial targets, such as pitch, heading, etc., while existing UAV target detection devices are either only equipped with a single photoelectric detection device, which fails to form a networking advantage and only serves as additional information input under radar detection; or although equipped with multiple photoelectric detection devices to form a networking structure, the relative position information provided by these photoelectric detection devices is not fully utilized, and more blind spot video monitoring is achieved through networking.

[0006] In other words, in the scenario where the radar fails, the existing UAV target detection device can hardly provide relatively accurate positioning information of the UAV target by relying solely on photoelectric detection equipment. It can only provide some visual characteristics and video information of the UAV. These visual characteristics and video information are not very useful for the subsequent UAV countermeasures and management business. Therefore, the existing technology cannot meet the business characteristics and requirements of low-altitude detection and countermeasures of UAVs. Summary of the invention

[0007] The present invention aims to provide a target detection display and control method and device supporting photoelectric cross positioning in a radar failure scenario, so as to solve the problem that the existing technology cannot meet the business characteristics and requirements of low-altitude detection and countermeasure of unmanned aerial vehicles.

[0008] The present invention provides a target detection display and control method supporting photoelectric cross positioning in a radar failure scenario, comprising the following steps:

[0009] S1, the protocol conversion module receives the optoelectronic message and optoelectronic video stream from the optoelectronic device and the lightning message from the radar device, sends the received optoelectronic video stream directly to the target classification and recognition module, and converts the received optoelectronic message and radar message into a unified data format and sends it to the target data processing module; wherein:

[0010] Radar messages include: heartbeat data of radar equipment and radar target data;

[0011] Photoelectric messages include: heartbeat data and photoelectric detection data of photoelectric equipment;

[0012] S2, the target data processing module processes the optoelectronic message and radar message after protocol conversion:

[0013] Radar target data is aggregated into local radar tracks and then output to the multi-source radar fusion module;

[0014] The photoelectric detection data is directly forwarded to the multi-source photoelectric cross positioning module;

[0015] The heartbeat data of radar equipment and optoelectronic equipment are directly forwarded to the equipment health detection module;

[0016] S3, the equipment health detection module performs real-time detection of the health status of the radar equipment and the optoelectronic equipment according to the heartbeat data of the radar equipment and the heartbeat data of the optoelectronic equipment, and outputs a data source failure instruction, a data source removal instruction or a data source recovery instruction to the multi-source radar fusion module and the multi-source optoelectronic cross positioning module according to the detected health status, so that the multi-source radar fusion module and the multi-source optoelectronic cross positioning module perform failure, removal or recovery operations on the data from the corresponding radar equipment and optoelectronic equipment during processing;

[0017] S4, the multi-source radar fusion module uses the SF distributed fusion algorithm to correlate and fuse multiple local radar tracks from the target data processing module to form the system radar track of the current fusion cycle;

[0018] S5, the multi-source photoelectric cross positioning module uses a cross positioning algorithm based on spatial stereo computational geometry to provide target positioning data for the photoelectric detection data from the target data processing module;

[0019] S6, the target classification and recognition module uses a deep learning convolutional neural network with YoloV3 as the skeleton to classify and recognize targets on the optoelectronic video stream to obtain target classification and recognition results;

[0020] S7, the display and control management module displays the system radar track, target positioning data and target classification and recognition results, and plays the optoelectronic video stream.

[0021] Optionally, the unified data format in step S1 adopts JSON format.

[0022] Furthermore, in step S2, the method for aggregating radar target data into local radar tracks and then outputting them to the multi-source radar fusion module includes:

[0023] The Mahalanobis distance between each radar target data is calculated using the tracking algorithm;

[0024] Comparing the calculated Mahalanobis distance with a preset Mahalanobis distance threshold;

[0025] The radar target data corresponding to the distance less than the preset Mahalanobis distance threshold are aggregated into local radar tracks.

[0026] Furthermore, step S3 includes the following sub-steps:

[0027] S31, the device health detection module starts multiple status detection threads to perform real-time detection of the health status of the radar device and the optoelectronic device according to the heartbeat data of the radar device and the heartbeat data of the optoelectronic device;

[0028] S32, each status detection thread extracts the device ID number that is still running healthily from the heartbeat data, and caches the device ID number and the number of consecutive heartbeat data received and the number of consecutive heartbeat data not received for real-time detection of the health status of the device:

[0029] (1) If a device fails to receive heartbeat data for a number of times greater than the failure threshold, the health status of the device is set to failure, and the wake-up instruction sending thread sends a data source failure instruction to the multi-source radar fusion module and the multi-source optoelectronic cross positioning module, so that the data from the device is invalidated when processed in the multi-source radar fusion module and the multi-source optoelectronic cross positioning module, that is, the data from the device does not participate in the processing of the multi-source radar fusion module and the multi-source optoelectronic cross positioning module;

[0030] (2) If a device fails to receive heartbeat data for a number of times greater than the rejection threshold, the health status of the device is set to rejection, and the relevant data of the device is deleted from the cache. At the same time, the instruction sending thread is awakened to send a data source rejection instruction to the multi-source radar fusion module and the multi-source optoelectronic cross positioning module, so that the multi-source radar fusion module and the multi-source optoelectronic cross positioning module cannot read the data from the device during processing;

[0031] (3) If a device in a failed state or a discarded state continuously receives heartbeat data for a number of times greater than the recovery threshold, the data of this device is cached and its health state is set to healthy. At the same time, the instruction sending thread is awakened to send a data source recovery instruction to the multi-source radar fusion module and the multi-source optoelectronic cross-positioning module, so that the data from this device is restored during processing in the multi-source radar fusion module and the multi-source optoelectronic cross-positioning module, that is, the data from this device re-participates in the processing of the multi-source radar fusion module and the multi-source optoelectronic cross-positioning module.

[0032] Furthermore, in step S4, assuming that there are N local radar tracks from the target data processing module, the calculation formula of the SF distributed fusion algorithm is as follows:

[0033]

[0034]

[0035] in, represents the optimal estimate of the i-th local radar track at the k-th time, P i (k|k) represents the mean square error of the i-th local radar track at the k-th time, represents the optimal estimate of the system radar track at the kth moment, and P(k|k) represents the mean square error of the system radar track at the kth moment.

[0036] Furthermore, step S5 includes the following sub-steps:

[0037] S51, obtain the longitude and latitude of the optoelectronic devices through the positioning modules carried by the two optoelectronic devices themselves, assume that all calculations are performed in the xyz coordinate system, set the coordinates of the location of optoelectronic device 1 to (x1, y1, z1), and the coordinates of the location of optoelectronic device 2 to (x2, y2, z2), and assume that the target direction angles detected by optoelectronic device 1 and optoelectronic device 2 at the kth moment are α1 and α2 respectively, and the target pitch angles are β1 and β2 respectively, then the coordinates of the target location at the kth moment are calculated according to the following formula: (x(k), y(k), z(k)):

[0038]

[0039] S52, in the multi-source photoelectric cross-positioning module, select the photoelectric detection data provided by two photoelectric devices that can form an intersection, and then start the calculation thread to perform a calculation at a fixed interval according to the method of step S51. If the photoelectric device is active during this process and the orientation positions of the two photoelectrics continue to have an intersection, then the subsequent position point coordinates of the target will be continuously generated to obtain the target positioning data.

[0040] Furthermore, step S6 includes the following sub-steps:

[0041] S61, sampling and segmenting the optoelectronic video stream using image extraction technology to obtain a plurality of video frame images;

[0042] S62, performing image quality improvement processing on the video frame images by using an inter-frame difference processing technology;

[0043] S63, performing image processing on the video frame image processed by the inter-frame difference processing technology using a deep learning convolutional neural network with YoloV3 as the skeleton and extracting image features;

[0044] S64, inputting the extracted image features into a target classification and recognition model to perform target classification and recognition to obtain a target classification and recognition result.

[0045] Furthermore, in step S7, the display control management module displays the system radar track, target positioning data and target classification and recognition results in the form of a human-computer interaction interface, and plays the optoelectronic video stream.

[0046] The present invention also provides a device for implementing the above-mentioned target detection display and control method supporting photoelectric cross positioning in a radar failure scenario, comprising a display and control management module and a protocol conversion module controlled by the display and control management module, a target data processing module, an equipment health detection module, a multi-source radar fusion module, a multi-source photoelectric cross positioning module and a target classification and recognition module;

[0047] The input end of the protocol conversion module is used to connect the radar device and the optoelectronic device; the protocol conversion module is connected to the input end of the target classification and recognition module on the one hand, and connected to the target data processing module on the other hand;

[0048] The target data processing module is respectively connected to the equipment health detection module, the multi-source radar fusion module and the multi-source photoelectric cross-positioning module, and the equipment health detection module, the multi-source radar fusion module, the multi-source photoelectric cross-positioning module and the target classification and identification module are all connected to the display and control management module.

[0049] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0050] 1. The present invention is based on a streaming data processing framework to achieve multi-source signal fusion display when radar equipment and optoelectronic equipment are working simultaneously. Relying on the advantages of optoelectronic detection and combining the technology of cross-positioning of multiple optoelectronic devices, it can achieve more accurate positioning of aerial targets while realizing target classification, identification and tracking. In addition, even when the radar fails, it can also use the target position data generated by optoelectronic cross-positioning to carry out business.

[0051] 2. The present invention can realize the automatic analysis, cross-positioning, track initiation and display management of the relative position of the target detected by the optoelectronic equipment;

[0052] 3. The present invention supports the fusion of radar tracks and optoelectronic cross-positioning tracks, and also supports the separate display and mutual switching of the two;

[0053] 4. The present invention supports low-altitude target monitoring functions such as system radar track, target positioning data and target classification and recognition result display, as well as optoelectronic video stream playback;

[0054] 5. The present invention utilizes unified protocol conversion, which can support the introduction of different monitoring source data according to actual needs with relatively few changes, and achieve consistency of data transmission within different monitoring source data messages. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 This is a schematic diagram of the principle architecture of a target detection display and control method and device that supports photoelectric cross-positioning in a radar failure scenario according to an embodiment of the present invention.

[0057] Figure 2 This is a flow chart of data processing by a device health detection module according to an embodiment of the present invention.

[0058] Figure 3 This is a schematic diagram of the principle of data processing by the multi-source optoelectronic cross-positioning module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Example

[0062] This embodiment proposes a target detection and display control method that supports optoelectronic cross-positioning in a radar failure scenario, which mainly adopts a streaming processing technology framework to complete the reception, processing, fusion, detection, cross-positioning, display, control, and management of radar messages, optoelectronic messages, and optoelectronic video streams.

[0063] like Figure 1 As shown, the display control method includes the following steps:

[0064] S1, the protocol conversion module receives the photoelectric message and photoelectric video stream from the photoelectric device and the lightning message from the radar device, sends the received photoelectric video stream directly to the target classification and recognition module, and converts the received photoelectric message and radar message into a unified data format and sends it to the target data processing module; in this embodiment, the unified data format adopts the JSON format. Among them:

[0065] Radar messages include: heartbeat data of radar equipment and radar target data; radar target data mainly includes longitude, latitude, pitch, azimuth, and speed;

[0066] Photoelectric messages include: heartbeat data and photoelectric detection data of photoelectric equipment; photoelectric detection data mainly includes pitch and direction;

[0067] S2, the target data processing module processes the optoelectronic message and radar message after protocol conversion:

[0068] (1) Radar target data is aggregated into local radar tracks and then output to the multi-source radar fusion module; specifically:

[0069] The Mahalanobis distance between each radar target data is calculated based on the longitude, latitude, pitch, azimuth and speed contained in the radar target data by using a tracking algorithm; wherein the tracking algorithm is a prior art and will not be described in detail here;

[0070] Comparing the calculated Mahalanobis distance with a preset Mahalanobis distance threshold; the Mahalanobis distance threshold is preset according to requirements;

[0071] Aggregate radar target data corresponding to distances less than the preset Mahalanobis distance threshold into local radar tracks

[0072] (2) The photoelectric detection data is directly forwarded to the multi-source photoelectric cross positioning module;

[0073] (3) The heartbeat data of the radar device and the heartbeat data of the optoelectronic device are directly forwarded to the device health detection module;

[0074] S3, the equipment health detection module performs real-time detection of the health status of the radar equipment and the optoelectronic equipment according to the heartbeat data of the radar equipment and the heartbeat data of the optoelectronic equipment, and outputs a data source invalidation instruction, a data source removal instruction or a data source recovery instruction to the multi-source radar fusion module and the multi-source optoelectronic cross positioning module according to the detected health status, so that the multi-source radar fusion module and the multi-source optoelectronic cross positioning module perform invalidation, removal or recovery operations on the data from the corresponding radar equipment and optoelectronic equipment during processing; Figure 2 As shown, specifically:

[0075] S31, the device health detection module starts multiple status detection threads to perform real-time detection of the health status of the radar device and the optoelectronic device according to the heartbeat data of the radar device and the heartbeat data of the optoelectronic device;

[0076] S32, each status detection thread extracts the device ID number that is still running healthily from the heartbeat data, and caches the device ID number and the number of consecutive heartbeat data received and the number of consecutive heartbeat data not received for real-time detection of the health status of the device:

[0077] (1) If a device fails to receive heartbeat data for a number of times greater than the failure threshold, the health status of the device is set to failure, and the wake-up instruction sending thread sends a data source failure instruction to the multi-source radar fusion module and the multi-source optoelectronic cross positioning module, so that the data from the device is invalidated when processed in the multi-source radar fusion module and the multi-source optoelectronic cross positioning module, that is, the data from the device does not participate in the processing of the multi-source radar fusion module and the multi-source optoelectronic cross positioning module;

[0078] (2) If a device fails to receive heartbeat data for a number of times greater than the rejection threshold, the health status of the device is set to rejection, and the relevant data of the device is deleted from the cache. At the same time, the instruction sending thread is awakened to send a data source rejection instruction to the multi-source radar fusion module and the multi-source optoelectronic cross positioning module, so that the multi-source radar fusion module and the multi-source optoelectronic cross positioning module cannot read the data from the device during processing;

[0079] (3) If a device in a failed state or a discarded state continuously receives heartbeat data for a number of times greater than the recovery threshold, the data of this device is cached and its health state is set to healthy. At the same time, the instruction sending thread is awakened to send a data source recovery instruction to the multi-source radar fusion module and the multi-source optoelectronic cross-positioning module, so that the data from this device is restored during processing in the multi-source radar fusion module and the multi-source optoelectronic cross-positioning module, that is, the data from this device re-participates in the processing of the multi-source radar fusion module and the multi-source optoelectronic cross-positioning module.

[0080] S4, the multi-source radar fusion module uses the SF distributed fusion algorithm to correlate and fuse multiple local radar tracks from the target data processing module to form the system radar track of the current fusion cycle; where, assuming that there are N local radar tracks from the target data processing module, the calculation formula of the SF distributed fusion algorithm is as follows:

[0081]

[0082]

[0083] in, represents the optimal estimate of the i-th local radar track at the k-th time, P i (k|k) represents the mean square error of the i-th local radar track at the k-th time, represents the optimal estimate of the system radar track at the kth moment, and P(k|k) represents the mean square error of the system radar track at the kth moment.

[0084] S5, the multi-source photoelectric cross positioning module uses a cross positioning algorithm based on spatial stereo computational geometry to provide target positioning data for the photoelectric detection data from the target data processing module; Figure 3 As shown, specifically:

[0085] S51, obtain the longitude and latitude of the optoelectronic devices through the positioning modules carried by the two optoelectronic devices themselves, assume that all calculations are performed in the xyz coordinate system, set the coordinates of the location of optoelectronic device 1 to (x1, y1, z1), and the coordinates of the location of optoelectronic device 2 to (x2, y2, z2), and assume that the target direction angles detected by optoelectronic device 1 and optoelectronic device 2 at the kth moment are α1 and α2 respectively, and the target pitch angles are β1 and β2 respectively, then the coordinates of the target location at the kth moment are calculated according to the following formula: (x(k), y(k), z(k)):

[0086]

[0087] S52, in the multi-source photoelectric cross-positioning module, select the photoelectric detection data provided by two photoelectric devices that can form an intersection, and then start the calculation thread to perform a calculation at a fixed interval according to the method of step S51. If the photoelectric device is active during this process and the orientation positions of the two photoelectrics continue to have an intersection, then the subsequent position point coordinates of the target will be continuously generated to obtain the target positioning data.

[0088] S6, the target classification and recognition module uses a deep learning convolutional neural network with YoloV3 as the skeleton to classify and recognize the target on the optoelectronic video stream to obtain the target classification and recognition result; specifically:

[0089] S61, sampling and segmenting the optoelectronic video stream using image extraction technology to obtain a plurality of video frame images;

[0090] S62, performing image quality improvement processing on the video frame images by using an inter-frame difference processing technology;

[0091] S63, using a deep learning convolutional neural network with YoloV3 as the skeleton to process the video frame image processed by the inter-frame difference processing technology and extract image features; wherein the deep learning convolutional neural network with YoloV3 as the skeleton is an existing technology and will not be described in detail here;

[0092] S64, input the extracted image features into a target classification recognition model for target classification recognition to obtain a target classification recognition result. The target classification recognition model is a classifier pre-trained with training samples, and the classifier can be a commonly used classifier for target classification recognition as needed.

[0093] S7, the display and control management module displays the system radar track, target positioning data and target classification and recognition results, and plays the optoelectronic video stream. In this embodiment, the display and control management module can be based on the geographic data and spatial information calculation and display capabilities provided by the GIS engine, and apply the technologies in the fields of computer graphics, UI interaction, computer vision, network communication, etc. to read the system radar track, target positioning data, target classification and recognition results, optoelectronic video stream, etc. After combination, packaging, rendering and other processing, the system radar track, target positioning data and target classification and recognition results are displayed in the form of a human-computer interaction interface, and the optoelectronic video stream is played. At the same time, the user's interface operation and command operation can also be monitored on the human-computer interaction interface, and feedback is given to each functional module in the form of events, network streams, etc.

[0094] like Figure 1 As shown, in order to implement the above-mentioned target detection display and control method supporting photoelectric cross positioning in the radar failure scenario, the display and control device provided in this embodiment includes a display and control management module and a protocol conversion module controlled by the display and control management module, a target data processing module, an equipment health detection module, a multi-source radar fusion module, a multi-source photoelectric cross positioning module and a target classification and recognition module;

[0095] The input end of the protocol conversion module is used to connect the radar device and the optoelectronic device; the protocol conversion module is connected to the input end of the target classification and recognition module on the one hand, and connected to the target data processing module on the other hand;

[0096] The target data processing module is respectively connected to the equipment health detection module, the multi-source radar fusion module and the multi-source photoelectric cross-positioning module, and the equipment health detection module, the multi-source radar fusion module, the multi-source photoelectric cross-positioning module and the target classification and identification module are all connected to the display and control management module.

[0097] The protocol conversion module, target data processing module, equipment health detection module, multi-source radar fusion module, multi-source optoelectronic cross-positioning module, target classification and recognition module and display and control management module in the display and control device work in accordance with the above-mentioned display and control method and will not be repeated here.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A target detection display and control method supporting photoelectric cross positioning in a radar failure scenario, characterized in that: The steps include: S1, the protocol conversion module receives the optoelectronic message and optoelectronic video stream from the optoelectronic device and the lightning message from the radar device, sends the received optoelectronic video stream directly to the target classification and recognition module, and converts the received optoelectronic message and radar message into a unified data format and sends it to the target data processing module; wherein: Radar messages include: heartbeat data of radar equipment and radar target data; Photoelectric messages include: heartbeat data and photoelectric detection data of photoelectric equipment; S2, the target data processing module processes the optoelectronic message and radar message after protocol conversion: Radar target data is aggregated into local radar tracks and then output to the multi-source radar fusion module; The photoelectric detection data is directly forwarded to the multi-source photoelectric cross positioning module; The heartbeat data of radar equipment and optoelectronic equipment are directly forwarded to the equipment health detection module; S3, the equipment health detection module performs real-time detection of the health status of the radar equipment and the optoelectronic equipment according to the heartbeat data of the radar equipment and the heartbeat data of the optoelectronic equipment, and outputs a data source failure instruction, a data source removal instruction or a data source recovery instruction to the multi-source radar fusion module and the multi-source optoelectronic cross positioning module according to the detected health status, so that the multi-source radar fusion module and the multi-source optoelectronic cross positioning module perform failure, removal or recovery operations on the data from the corresponding radar equipment and optoelectronic equipment during processing; S4, the multi-source radar fusion module uses the SF distributed fusion algorithm to correlate and fuse multiple local radar tracks from the target data processing module to form the system radar track of the current fusion cycle; S5, the multi-source photoelectric cross positioning module uses a cross positioning algorithm based on spatial stereo computational geometry to provide target positioning data for the photoelectric detection data from the target data processing module; S6, the target classification and recognition module uses a deep learning convolutional neural network with YoloV3 as the skeleton to classify and recognize targets on the optoelectronic video stream to obtain target classification and recognition results; S7, the display and control management module displays the system radar track, target positioning data and target classification and recognition results, and plays the optoelectronic video stream; Step S3 includes the following sub-steps: S31, the device health detection module starts multiple status detection threads to perform real-time detection of the health status of the radar device and the optoelectronic device according to the heartbeat data of the radar device and the heartbeat data of the optoelectronic device; S32, each status detection thread extracts the device ID number that is still running healthily from the heartbeat data, and caches the device ID number and the number of consecutive heartbeat data received and the number of consecutive heartbeat data not received for real-time detection of the health status of the device: (1) If a device fails to receive heartbeat data for a number of times greater than the failure threshold, the health status of the device is set to failure, and the wake-up instruction sending thread sends a data source failure instruction to the multi-source radar fusion module and the multi-source optoelectronic cross positioning module, so that the data from the device is invalidated when processed in the multi-source radar fusion module and the multi-source optoelectronic cross positioning module, that is, the data from the device does not participate in the processing of the multi-source radar fusion module and the multi-source optoelectronic cross positioning module; (2) If a device fails to receive heartbeat data for a number of times greater than the rejection threshold, the health status of the device is set to rejection, and the relevant data of the device is deleted from the cache. At the same time, the instruction sending thread is awakened to send a data source rejection instruction to the multi-source radar fusion module and the multi-source optoelectronic cross positioning module, so that the multi-source radar fusion module and the multi-source optoelectronic cross positioning module cannot read the data from the device during processing; (3) If a device in a failed state or a discarded state continuously receives heartbeat data for a number of times greater than the recovery threshold, the data of this device is cached and its health state is set to healthy. At the same time, the instruction sending thread is awakened to send a data source recovery instruction to the multi-source radar fusion module and the multi-source optoelectronic cross-positioning module, so that the data from this device is restored during processing in the multi-source radar fusion module and the multi-source optoelectronic cross-positioning module, that is, the data from this device re-participates in the processing of the multi-source radar fusion module and the multi-source optoelectronic cross-positioning module.

2. The target detection display and control method supporting photoelectric cross positioning in a radar failure scenario according to claim 1 is characterized in that: The unified data format in step S1 adopts JSON format.

3. The target detection display and control method supporting photoelectric cross positioning in a radar failure scenario according to claim 1 is characterized in that: The method for aggregating radar target data into local radar tracks and then outputting them to a multi-source radar fusion module in step S2 includes: The Mahalanobis distance between each radar target data is calculated using the tracking algorithm; Comparing the calculated Mahalanobis distance with a preset Mahalanobis distance threshold; The radar target data corresponding to the distance less than the preset Mahalanobis distance threshold are aggregated into local radar tracks.

4. The target detection display and control method supporting photoelectric cross positioning in a radar failure scenario according to claim 1 is characterized in that: In step S4, assuming that there are N local radar tracks from the target data processing module, the calculation formula of the SF distributed fusion algorithm is as follows: in, represents the optimal estimate of the i-th local radar track at the k-th time, P i (k|k) represents the mean square error of the i-th local radar track at the k-th time, represents the optimal estimate of the system radar track at the kth moment, and P(k|k) represents the mean square error of the system radar track at the kth moment.

5. The target detection display and control method supporting photoelectric cross positioning in a radar failure scenario according to claim 1 is characterized in that: Step S5 includes the following sub-steps: S51, obtain the longitude and latitude of the optoelectronic devices through the positioning modules carried by the two optoelectronic devices themselves, assume that all calculations are performed in the xyz coordinate system, set the coordinates of the location of optoelectronic device 1 to (x1, t1, z1), and the coordinates of the location of optoelectronic device 2 to (x2, y2, z2), and assume that the target direction angles detected by optoelectronic device 1 and optoelectronic device 2 at the kth moment are α1 and α2 respectively, and the target pitch angles are β1 and β2 respectively, then the coordinates of the target location at the kth moment are calculated according to the following formula (x(k), y(k), z(k)): S52, in the multi-source photoelectric cross-positioning module, select the photoelectric detection data provided by two photoelectric devices that can form an intersection, and then start the calculation thread to perform a calculation at a fixed interval according to the method of step S51. If the photoelectric device is active during this process and the orientation positions of the two photoelectrics continue to have an intersection, then the subsequent position point coordinates of the target will be continuously generated to obtain the target positioning data.

6. The target detection display and control method supporting photoelectric cross positioning in a radar failure scenario according to claim 1 is characterized in that: Step S6 includes the following sub-steps: S61, sampling and segmenting the optoelectronic video stream using image extraction technology to obtain a plurality of video frame images; S62, using an inter-frame difference processing technology to improve the image quality of the video frame image; S63, performing image processing on the video frame image processed by the inter-frame difference processing technology using a deep learning convolutional neural network with YoloV3 as the skeleton and extracting image features; S64, inputting the extracted image features into a target classification and recognition model to perform target classification and recognition to obtain a target classification and recognition result.

7. The target detection display and control method supporting photoelectric cross positioning in a radar failure scenario according to claim 1 is characterized in that: In step S7, the display control management module displays the system radar track, target positioning data and target classification and recognition results in the form of a human-computer interaction interface, and plays the optoelectronic video stream.

8. A device for implementing the target detection display and control method supporting photoelectric cross positioning in a radar failure scenario as described in any one of claims 1 to 7, characterized in that: It includes a display and control management module and a protocol conversion module, a target data processing module, an equipment health detection module, a multi-source radar fusion module, a multi-source optoelectronic cross positioning module and a target classification and recognition module controlled by the display and control management module; The input end of the protocol conversion module is used to connect the radar device and the optoelectronic device; the protocol conversion module is connected to the input end of the target classification and recognition module on the one hand, and connected to the target data processing module on the other hand; The target data processing module is respectively connected to the equipment health detection module, the multi-source radar fusion module and the multi-source photoelectric cross-positioning module, and the equipment health detection module, the multi-source radar fusion module, the multi-source photoelectric cross-positioning module and the target classification and identification module are all connected to the display and control management module.

Images