Method for detecting coast and sea targets by guiding photoelectric equipment through radar

Abstract

The invention discloses a method, device and equipment for detecting coast and sea targets by guiding photoelectric equipment through a radar, and the method comprises the steps: carrying out the wide-area search through the radar, obtaining the azimuth and pitching information of a target in real time, and guiding the photoelectric equipment to be precisely aligned with the target; the video stream collected by the photoelectric equipment is analyzed in real time by the edge computing equipment, so that efficient target detection and recognition are realized; a dynamic duration threshold adjustment strategy is adopted, different weather conditions are adapted, and the accuracy and timeliness of detection are ensured; according to the method, the advantages of the radar and the photoelectric equipment are combined, the target detection efficiency and the response speed are improved, and the method is suitable for application scenes such as offshore and coastal security and protection monitoring.

Description

Technical Field

[0001] The present application relates to technical fields such as computer vision, edge computing, and target detection, and in particular to a method, apparatus, and equipment for radar-guided optoelectronic equipment to detect coastal and offshore targets. Background Art

[0002] Radar and optoelectronic devices are currently core technologies for maritime and coastal target detection, widely used in security monitoring, maritime regulation, and other fields. Radar, with its all-weather, wide-range detection capabilities, can quickly detect targets and provide location information, while optoelectronic devices offer high-precision imaging and recognition capabilities, enabling further target identification and classification. However, relying solely on one technology has limitations. For example, radar struggles to accurately identify target types, while optoelectronic devices have a limited search range. Therefore, combining the two is a key approach to improving target detection capabilities.

[0003] With the development of edge computing, using edge computing devices to perform local real-time video processing can effectively reduce data transmission pressure and improve system response speed. However, the marine environment is complex and changeable. When running on resource-constrained edge devices, traditional target detection algorithms may face problems such as excessive computational burden and detection delays. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device and equipment for radar-guided optoelectronic equipment to detect coastal and maritime targets, aiming to optimize the linkage mechanism between radar and optoelectronic equipment, and combine it with efficient edge computing methods to achieve accurate and efficient maritime and coastal target detection.

[0005] To achieve the above-mentioned purpose, the present application provides a method for radar-guided optoelectronic equipment to detect coastal and marine targets, including: in response to the normal communication signal between the target detection equipment, controlling the maritime radar to start the transmission mode to conduct a wide-area search for coastal and marine targets, wherein the target detection equipment includes a maritime radar, an optoelectronic device and an edge computing device; in response to the target search success signal, forwarding the azimuth and pitch information of the maritime radar to the optoelectronic servo of the optoelectronic device in real time to guide the optoelectronic device to aim at the target; in response to the target alignment signal of the optoelectronic device, controlling the optoelectronic device to adjust to the specified search range, and detecting and analyzing the video images collected by the optoelectronic device in real time through the edge computing device to extract target information: in response to the signal that the edge computing device detects the target from the target information, controlling the edge computing device to send the detected target information to the target server via the WebSocket transmission protocol. When the target server determines that the target belongs to a preset threat category, it generates alarm data and activates the alarm mechanism.

[0006] Optionally, the method further includes: if the communication between the target detection devices is abnormal, reconnecting the maritime radar, optoelectronic equipment and edge computing equipment, and recording a communication abnormality log.

[0007] Optionally, before the control edge computing device sends the detected target information to the target server through the WebSocket transmission protocol, the method also includes: if it is determined that the detection time of the edge computing device for the target information exceeds a set threshold and no target is found, the maritime radar is controlled to switch to manual search mode.

[0008] Optionally, the optoelectronic servo forwards the azimuth and elevation information of the maritime radar to the optoelectronic device in real time to guide the optoelectronic device to align with the target, including: converting the radar azimuth and elevation information into the longitude and latitude of the target; calculating the azimuth and elevation of the target in the optoelectronic device based on the longitude and latitude of the target, and determining the optoelectronic calibration value based on the azimuth and elevation of the target in the optoelectronic device; and sending the optoelectronic calibration value to the optoelectronic device to control the optoelectronic device to align with the target.

[0009] Optionally, the real-time detection and analysis of the video images collected by the optoelectronic device by the edge computing device to extract target information includes: receiving the real-time video stream collected by the optoelectronic device based on the RTSP video stream address; using the ffmpeg algorithm to hard-decode the real-time video stream to obtain each video stream; controlling the edge computing device to use multiple image detection models to locally process each video stream to obtain target information with preset threat category information.

[0010] Optionally, the target information includes: target image features, location coordinates and category labels

[0011] Optionally, before the target server determines that the target belongs to a preset threat category, the method also includes: controlling the target server to listen to a designated port and establishing a WebSocket connection with the optoelectronic device based on the listening result; controlling the target server to receive JSON format data sent by the edge computing device, parsing the IP address of the optoelectronic device in the JSON format data and the target category in the target information, and verifying the format correctness of the JSON format data; and checking a first match between the video_code field in any JSON format data and the specified IP address, and checking a second match between the identification result of the target category and the device information. If one of the first match and the second match does not match, the JSON format data is recorded in the device log and discarded.

[0012] Optionally, when the target server determines that the target belongs to a preset threat category, it generates alarm data and activates an alarm mechanism, including: obtaining a preset threat category, where the threat category includes people, beach cars, motorboats, sampans, fishing boats, cruise ships and other ships; if both the first matching situation and the second matching situation match, then determining the threat category to which the target belongs; generating alarm data according to the threat category to which the target belongs and activating the alarm mechanism.

[0013] In addition, to achieve the above-mentioned purpose, the present application also provides a device for detecting coastal and offshore targets by radar-guided optoelectronic equipment, which is characterized by comprising:

[0014] A wide-area search module, configured to control a maritime radar to activate a transmit mode in response to a normal communication signal between target detection devices, wherein the target detection devices include maritime radars, optoelectronic devices, and edge computing devices, to conduct a wide-area search for coastal and offshore targets;

[0015] The equipment guidance module is used to forward the azimuth and elevation information of the maritime radar to the optoelectronic servo of the optoelectronic equipment in real time in response to the target search success signal, so as to guide the optoelectronic equipment to align with the target;

[0016] The information extraction module is used to control the optoelectronic device to move to the specified search range in response to the target alignment signal from the optoelectronic device, and to extract target information by real-time detection and analysis of the video images collected by the optoelectronic device through the edge computing device:

[0017] The alarm module is used to respond to the signal of the target detected by the edge computing device from the target information, control the edge computing device to send the detected target information to the target server through the WebSocket transmission protocol, and when the target server determines that the target belongs to the preset threat category, it generates alarm data and activates the alarm mechanism.

[0018] To achieve the above-mentioned purpose, the present application also provides an electronic device, which includes: at least one processor, a memory and an input and output unit; wherein the memory is used to store computer programs, and the processor is used to call the computer program stored in the memory to execute the method provided in any of the aforementioned embodiments for radar-guided optoelectronic equipment to detect coastal and sea targets.

[0019] The present invention provides a method, apparatus, and device for detecting coastal and offshore targets using radar-guided optoelectronic devices. The method, apparatus, and device control the maritime radar to activate a transmission mode in response to a normal communication signal between the target detection devices to conduct a wide-area search for coastal and offshore targets. The target detection devices include a maritime radar, an optoelectronic device, and an edge computing device. In response to a target search success signal, the method forwards the maritime radar's azimuth and elevation information to the optoelectronic device's optoelectronic servo in real time to guide the optoelectronic device toward the target. In response to the optoelectronic device's target alignment signal, the method controls the optoelectronic device to adjust to a specified search range, and uses the edge computing device to detect and analyze the video footage captured by the optoelectronic device in real time to extract target information. In response to a target signal detected by the edge computing device from the target information, the method controls the edge computing device to transmit the detected target information to a target server via the WebSocket transport protocol. When the target server determines that the target belongs to a preset threat category, the method generates alarm data and activates an alarm mechanism. The present invention uses radar to accurately guide the optoelectronic device toward the target in real time, and combines the edge computing device to perform video stream analysis to quickly identify and classify preset threat targets in complex environments, significantly improving the efficiency and accuracy of target detection. The present invention realizes rapid detection of threat targets through the linkage of radar and optoelectronic equipment, improves system response speed and detection efficiency, and meets the actual needs of maritime and coastal security monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart illustrating an embodiment of a method for detecting coastal and offshore targets using radar-guided optoelectronic equipment is provided in this application;

[0021] Figure 2 A schematic diagram of the functional modules of an embodiment of the apparatus for detecting coastal and maritime targets by radar-guided optoelectronic equipment is provided in the present application.

[0022] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0023] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0024] Reference Figure 1 , Figure 1 This is a flowchart of a method for detecting coastal and offshore targets using radar-guided optoelectronic equipment, provided in the first embodiment of the present application. The method can be executed by a processor of a control device, wherein the control device is connected to a maritime radar, an optoelectronic device, and an edge computing device. The method for detecting coastal and offshore targets using radar-guided optoelectronic equipment may include:

[0025] S10. In response to a normal communication signal between target detection devices, controlling a maritime radar to start a transmission mode to conduct a wide-area search for coastal and offshore targets, wherein the target detection devices include a maritime radar, an optoelectronic device, and an edge computing device;

[0026] In one embodiment of the present application, the method for radar-guided optoelectronic equipment to detect coastal and offshore targets may further include the following execution process:

[0027] If the communication between the target detection devices is abnormal, reconnect the maritime radar, optoelectronic equipment and edge computing equipment, and record the communication abnormality log.

[0028] During the actual execution process, the processor detects whether the communication between the maritime radar, optoelectronic equipment and edge computing equipment is normal. If the communication is abnormal, it tries to reconnect the maritime radar, optoelectronic equipment and edge computing equipment and record the log. If the communication is normal, it turns on the radar transmission mode and conducts a wide-area search for coastal and sea targets. After searching the target, it obtains the radar's azimuth and pitch information and sends a target search success signal to the control device.

[0029] S20, in response to the target search success signal, forwarding the azimuth and elevation information of the maritime radar to the optoelectronic servo of the optoelectronic device in real time to guide the optoelectronic device to align with the target;

[0030] In an embodiment of the present application, step S20 may include the following execution process:

[0031] S201, converting radar azimuth and elevation information into target latitude and longitude;

[0032] S202, calculating the azimuth and elevation of the target in the optoelectronic device based on the longitude and latitude of the target, and determining an optoelectronic calibration value based on the azimuth and elevation of the target in the optoelectronic device;

[0033] S203: Send the photoelectric calibration value to the photoelectric device, and control the photoelectric device to align the target.

[0034] During actual execution, the processor converts the radar's azimuth and elevation information into the target's longitude and latitude. It then determines the optoelectronic calibration based on the target's azimuth and elevation as viewed by the optoelectronic device. Finally, the processor transmits the optoelectronic calibration value to the optoelectronic device, controlling it to align with the target. The processor also transmits the target's longitude and latitude to the optoelectronic servo stage in real time, guiding the optoelectronic device to quickly align with the target. The processor then sends the optoelectronic device's target alignment signal to the control device.

[0035] S30. In response to the target alignment signal of the optoelectronic device, control the optoelectronic device to move to the specified search range, and use the edge computing device to detect and analyze the video images collected by the optoelectronic device in real time to extract target information:

[0036] In an embodiment of the present application, step S30 may include the following execution process:

[0037] S301, receiving a real-time video stream collected by an optoelectronic device based on an RTSP video stream address;

[0038] S302, using ffmpeg algorithm to perform hard decoding on the real-time video stream to obtain various video streams;

[0039] S303: Control the edge computing device to locally process each video stream using multiple image detection models to obtain target information with preset threat category information.

[0040] During the actual execution process, the processor controls the edge computing device to access the optoelectronic device via the RTSP video stream address to receive the captured real-time video stream, and then performs ffmpeg hard decoding on the real-time video stream. It is worth noting that the edge computing device can access 15-25 real-time RTSP video streams, and each video stream can simultaneously process 3-4 algorithm models for target detection and recognition of the video data collected by the optoelectronic device, identifying images belonging to the preset threat category. Finally, the processor computing device uses multiple image detection models to locally process each video stream, obtain target information with preset threat category information, and send a signal to the control device indicating that the edge computing device has detected the target from the target information, where the target information includes: the target's image features, location coordinates, and category label.

[0041] S40. In response to the edge computing device detecting a target signal from the target information, the edge computing device is controlled to send the detected target information to the target server through the WebSocket transmission protocol. When the target server determines that the target belongs to a preset threat category, alarm data is generated and an alarm mechanism is activated.

[0042] In one embodiment of the present application, before the edge computing device is controlled to send the detected target information to the target server through the WebSocket transmission protocol, the method for radar-guided optoelectronic equipment to detect coastal and maritime targets also includes the following execution process: if it is determined that the detection time of the edge computing device for the target information exceeds the set threshold and no target is found, the maritime radar is controlled to switch to manual search mode.

[0043] During the actual execution process, the time threshold for edge computing devices to detect targets can be dynamically adjusted according to weather conditions, and the time threshold is set to no more than 15s.

[0044] In one embodiment of the present application, before the target server determines that the target belongs to a preset threat category, the method for radar-guided optoelectronic equipment to detect coastal and offshore targets further includes the following execution process:

[0045] Control the target server to listen to the specified port and establish a WebSocket connection with the optoelectronic device based on the listening results;

[0046] Control the target server to receive the JSON format data sent by the edge computing device, parse the IP address of the optoelectronic device in the JSON format data and the target category in the target information, and verify the format correctness of the JSON format data; and check the first match between the video_code field in any JSON format data and the specified IP address, and check the second match between the identification result of the target category and the device information. If one of the first match and the second match does not match, record and discard the JSON format data in the device log.

[0047] During the actual execution process, after receiving the target information, the processor controls the target server to listen to the specified port and establish a WebSocket connection to receive JSON format data from the edge computing device; the server parses the received information, extracts the optoelectronic device IP address and target category data, and verifies the correctness of the data format; then, the server makes a judgment based on the value of the video_code field. When the video_code is the specified IP, it performs target category identification and device information matching. For data that does not meet the predetermined conditions, the server records the log and discards invalid information.

[0048] In one embodiment of the present application, when the target server determines that the target belongs to a preset threat category, alarm data is generated and an alarm mechanism is activated, which may include the following execution process: obtaining a preset threat category, where the threat category includes people, beach cars, motorboats, sampans, fishing boats, cruise ships and other ships; if both the first matching situation and the second matching situation match, the threat category to which the target belongs is determined; and alarm data is generated according to the threat category to which the target belongs and the alarm mechanism is activated.

[0049] Reference Figure 2The second embodiment of the present application provides a device for detecting coastal and marine targets by radar-guided optoelectronic equipment. The device 100 for detecting coastal and marine targets by radar-guided optoelectronic equipment may include a wide-area search module 1001, an equipment guidance module 1002, an information extraction module 1003, and an alarm module 1004. The wide-area search module 1001 is used to control the maritime radar to start the transmission mode in response to the normal communication signal between the target detection equipment to perform a wide-area search for coastal and marine targets. The target detection equipment includes a maritime radar, an optoelectronic equipment, and an edge computing device; the equipment guidance module 1002 is used to send an information extraction module 1003 to the optoelectronic equipment in response to the target search success signal. The optoelectronic servo forwards the azimuth and pitch information of the maritime radar in real time to guide the optoelectronic device to align with the target; the information extraction module 1003 is used to respond to the target alignment signal of the optoelectronic device, control the optoelectronic device to adjust to the specified search range, and detect and analyze the video images collected by the optoelectronic device in real time through the edge computing device to extract the target information: the alarm module 1004 is used to respond to the signal of the edge computing device detecting the target from the target information, control the edge computing device to send the detected target information to the target server through the WebSocket transmission protocol, and when the target server determines that the target belongs to the preset threat category, it generates alarm data and starts the alarm mechanism

[0050] It is not difficult to find that this embodiment is an apparatus embodiment corresponding to the above-mentioned method embodiment, and this embodiment can be implemented in conjunction with the above-mentioned method embodiment. The relevant technical details and technical effects mentioned in the above-mentioned embodiments are still valid in this embodiment, and to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above-mentioned embodiments.

[0051] It is worth mentioning that all modules involved in this embodiment are logical modules. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, to highlight the innovation of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application. However, this does not mean that other units do not exist in this embodiment.

[0052] Another embodiment of the present application proposes an electronic device, comprising: at least one processor, a memory, and an input-output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute a multi-target tracking and area dynamic counting method based on edge computing.

[0053] The memory and processor are connected using a bus, which can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and therefore will not be described further in this article. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. Data processed by the processor is transmitted on a wireless medium via an antenna. Furthermore, the antenna also receives data and transmits it to the processor.

[0054] The processor is responsible for managing the bus and general processing, and may also provide various functions, including timing, peripheral interfacing, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor while performing operations.

[0055] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps in the various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0056] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting coastal and offshore targets using radar-guided optoelectronic equipment, characterized in that: Applied to control equipment, including: In response to a normal communication signal between target detection devices, controlling a maritime radar to start a transmission mode to conduct a wide-area search for coastal and offshore targets, wherein the target detection devices include a maritime radar, an optoelectronic device, and an edge computing device; In response to the target search success signal, the azimuth and elevation information of the maritime radar is forwarded to the optoelectronic servo of the optoelectronic device in real time to guide the optoelectronic device to aim at the target; In response to the target alignment signal from the optoelectronic device, the optoelectronic device is controlled to move to the specified search range, and the video images collected by the optoelectronic device are detected and analyzed in real time by the edge computing device to extract target information: In response to the edge computing device detecting a target signal from the target information, the edge computing device is controlled to send the detected target information to the target server through the WebSocket transmission protocol. When the target server determines that the target belongs to a preset threat category, alarm data is generated and an alarm mechanism is activated.

2. The method for detecting coastal and offshore targets using radar-guided optoelectronic equipment according to claim 1, wherein: The method further comprises: If the communication between the target detection devices is abnormal, reconnect the maritime radar, optoelectronic equipment and edge computing equipment, and record the communication abnormality log.

3. The method for detecting coastal and offshore targets using radar-guided optoelectronic equipment according to claim 1, wherein: Before the control edge computing device sends the detected target information to the target server through the WebSocket transmission protocol, the method further includes: If it is determined that the detection time of the edge computing device for the target information exceeds the set threshold and no target is found, the maritime radar is controlled to switch to manual search mode.

4. The method for detecting coastal and offshore targets using radar-guided optoelectronic equipment as claimed in claim 1, wherein: The real-time forwarding of the azimuth and elevation information of the maritime radar to the optoelectronic servo of the optoelectronic device to guide the optoelectronic device to align with the target includes: Convert radar azimuth and elevation information into target latitude and longitude; Calculating the azimuth and elevation of the target in the optoelectronic device based on the target's latitude and longitude, and determining an optoelectronic calibration value based on the azimuth and elevation of the target in the optoelectronic device; The photoelectric calibration value is sent to the photoelectric device to control the photoelectric device to align the target.

5. The method for detecting coastal and offshore targets using radar-guided optoelectronic equipment according to claim 2, wherein: The real-time detection and analysis of the video images collected by the optoelectronic device by the edge computing device to extract target information includes: Receive real-time video stream collected by optoelectronic equipment based on RTSP video stream address; Use ffmpeg algorithm to perform hard decoding on real-time video stream to obtain various video streams; The control edge computing device uses multiple image detection models to locally process each video stream and obtain target information with preset threat category information.

6. The method for detecting coastal and offshore targets using radar-guided optoelectronic equipment according to claim 1, wherein: The target information includes: The image features, location coordinates and category labels of the target.

7. The method for detecting coastal and offshore targets using radar-guided optoelectronic equipment according to claim 1, wherein: Before the target server determines that the target belongs to a preset threat category, the method further includes: Control the target server to listen to the specified port and establish a WebSocket connection with the optoelectronic device based on the listening results; Control the target server to receive the JSON format data sent by the edge computing device, parse the IP address of the optoelectronic device in the JSON format data and the target category in the target information, and verify the format correctness of the JSON format data; and check the first match between the video_code field in any JSON format data and the specified IP address, and check the second match between the identification result of the target category and the device information. If one of the first match and the second match does not match, record and discard the JSON format data in the device log.

8. The method for detecting coastal and offshore targets using radar-guided optoelectronic equipment according to claim 7, wherein: When the target server determines that the target belongs to the preset threat category, it generates alarm data and activates the alarm mechanism, including: Get preset threat categories, including people, ATVs, motorboats, sampans, fishing boats, pleasure boats, and other vessels If both the first matching situation and the second matching situation match, the threat category to which the target belongs is determined; Generate alarm data and activate the alarm mechanism based on the threat category to which the target belongs.

9. A device for detecting coastal and offshore targets using radar-guided optoelectronic equipment, characterized in that: include: A wide-area search module, configured to control a maritime radar to activate a transmit mode in response to a normal communication signal between target detection devices, wherein the target detection devices include maritime radars, optoelectronic devices, and edge computing devices, to conduct a wide-area search for coastal and offshore targets; The equipment guidance module is used to forward the azimuth and elevation information of the maritime radar to the optoelectronic servo of the optoelectronic equipment in real time in response to the target search success signal, so as to guide the optoelectronic equipment to align with the target; The information extraction module is used to control the optoelectronic device to move to the specified search range in response to the target alignment signal from the optoelectronic device, and to extract target information by real-time detection and analysis of the video images collected by the optoelectronic device through the edge computing device: The alarm module is used to respond to the signal of the target detected by the edge computing device from the target information, control the edge computing device to send the detected target information to the target server through the WebSocket transmission protocol, and when the target server determines that the target belongs to the preset threat category, it generates alarm data and activates the alarm mechanism.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor, memory, and input-output unit; The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the method for detecting coastal and sea targets using a radar-guided optoelectronic device according to any one of claims 1 to 8.

Images