Coast defense monitoring target real-time extraction system and method based on edge calculation

Abstract

The invention discloses a coast defense monitoring target real-time extraction system and method based on edge calculation, and belongs to the field of coast defense monitoring. Comprising coast defense monitoring equipment used for generating video images in real time, an edge computing unit connected with the coast defense monitoring equipment and used for performing real-time analysis and storage on an end side, and a rear-end server communicating with the edge computing unit and used for receiving end side data and transmitting superior instructions. The edge calculation unit at least comprises a target detection tracking module which is used for carrying out target detection and multi-target tracking based on the video image; the target detection tracking module carries out target detection through the following steps: outputting a prediction frame marked with a ship by using a pre-trained target detection model; and performing post-processing on the prediction frame, and performing screening to obtain a target prediction frame. According to the scheme, high-precision extraction and transient storage of the target can be realized on the end side, so that the delay and risk of system operation are reduced, the unattended operation capability of the system is improved, and the occurrence of a control vacuum period is avoided.

Description

Technical Field

[0001] The present invention relates to the field of coastal defense monitoring technology, and in particular to a system and method for real-time extraction of coastal defense monitoring targets based on edge computing. Background Art

[0002] Traditional coastal defense monitoring systems usually rely on site industrial computers or central servers for image data processing. This centralized processing method is limited by network bandwidth and transmission link length, which is prone to data transmission delays. Second, once the network connection between the front-end coastal defense monitoring and the back-end is disconnected due to an accident, the front-end coastal defense monitoring loses its control capabilities during the network disconnection period, resulting in a control vacuum period. In addition, the traditional target detection model only uses non-maximum suppression (NMS) to remove redundant frames during post-processing, but there will be repeated detection of multiple target frames of different sizes and overlapping for the same ship target, or repeated marking of the bow part. In actual application, these above-mentioned phenomena will lead to incorrect ship target positioning and tracking, thereby affecting the accuracy of ship target detection.

[0003] Therefore, there is an urgent need for a coastal defense monitoring system that can achieve high-precision extraction and short-term storage of targets on the terminal side to reduce the delay and risk of system operation and improve the system's unmanned capability. Summary of the invention

[0004] In order to solve the problem that traditional coastal defense monitoring systems are unable to transmit monitoring videos to the back end in time for high-precision target detection and tracking during network disconnection periods, resulting in a control vacuum period, an embodiment of the present invention provides a real-time extraction system and method for coastal defense monitoring targets based on edge computing.

[0005] On the one hand, a real-time extraction system for coastal defense monitoring targets based on edge computing is provided, the system comprising: A coastal defense monitoring device for generating video images in real time, an edge computing unit connected to the coastal defense monitoring device for performing real-time analysis and storage on the end side, and a backend server communicating with the edge computing unit for receiving end-side data and transmitting superior instructions; The edge computing unit includes at least a target detection and tracking module, and the target detection and tracking module is used to perform target detection and multi-target tracking based on the video image; The target detection and tracking module performs target detection in the following manner: Use the pre-trained object detection model to output the predicted box labeled with the ship; The prediction box is post-processed to obtain a target prediction box through screening.

[0006] On the other hand, a method for real-time extraction of coastal defense monitoring targets based on any system embodiment of the specification is provided, comprising: An edge computing unit connected to the coastal defense monitoring device is provided on the terminal side thereof, and the video image captured in real time by the coastal defense monitoring device is acquired by using the edge computing unit; wherein the edge computing unit is provided with a target detection and tracking module, and the target detection and tracking module contains a pre-trained target detection model; Using the target detection model to perform target detection on the video image, after outputting a prediction frame marked with a ship, post-processing the prediction frame to screen and obtain a target prediction frame; Using the target detection and tracking module to perform multi-target tracking; The edge computing unit sends the stored end-side data to the back-end server.

[0007] The technical solution provided by the present invention can at least bring the following beneficial effects: By combining edge computing units with coastal defense monitoring equipment, traditional coastal defense equipment is given independent control capabilities, target extraction and short-term storage are achieved on the end side, system latency and risk are reduced, and the unmanned capability of the coastal defense monitoring system is enhanced; in addition, the accuracy and robustness of ship target detection are improved by post-processing the prediction frame output by the target detection model. Therefore, this solution can achieve high-precision extraction and short-term storage of targets on the end side to reduce the latency and risk of system operation, improve the unmanned capability of the system, and avoid the occurrence of a control vacuum period. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0009] Figure 1 It is a schematic diagram of a real-time extraction system for coastal defense monitoring targets based on edge computing provided by an embodiment of the present invention; Figure 2 It is a flowchart of a method for real-time extraction of coastal defense monitoring targets based on edge computing provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0010] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0011] The specific implementation of the above concept is described below.

[0012] Please refer to Figure 1 , an embodiment of the present invention provides a real-time extraction system for coastal defense monitoring targets based on edge computing, the system comprising: a coastal defense monitoring device for generating video images in real time, an edge computing unit connected to the coastal defense monitoring device for performing real-time analysis and storage on the terminal side, and a backend server communicating with the edge computing unit for receiving terminal side data and transmitting superior instructions; The edge computing unit includes at least a target detection and tracking module, which is used to perform target detection and multi-target tracking based on video images; The target detection and tracking module performs target detection in the following ways: Use the pre-trained object detection model to output the predicted box labeled with the ship; The prediction box is post-processed and the target prediction box is obtained by screening.

[0013] In the embodiment of the present invention, by combining the edge computing unit with the coastal defense monitoring equipment, the traditional coastal defense equipment is endowed with independent management and control capabilities, the target extraction and short-term storage on the terminal side are realized, the delay and risk of the system are reduced, and the unmanned capability of the coastal defense monitoring system is strengthened; in addition, by post-processing the prediction frame output by the target detection model, the accuracy and robustness of ship target detection are improved. Therefore, this solution can realize high-precision extraction and short-term storage of targets on the terminal side, reduce the delay and risk of system operation, improve the unmanned capability of the system, and avoid the occurrence of a control vacuum period.

[0014] In some embodiments, the edge computing unit further includes: a video acquisition module and an image processing module; The video acquisition module uses a hardware interface to access the coastal defense monitoring equipment to obtain real-time video images; The image processing module is used to pre-process the video image and send it to the edge computing unit.

[0015] In this embodiment, in order to ensure the delay, the video acquisition module uses the SDI interface to access the original video image of the coastal defense monitoring equipment.

[0016] The image processing module is mainly used to convert the original image data of the video frame for use by the subsequent target detection and tracking module.

[0017] The specific preprocessing steps include: Use the edge computing video processing engine to convert the original YUV format image into RGB format; Scale the image size according to the input size required by the target detection model. For example, when the original image size is 1920*1080, use bilinear interpolation to scale the original image to the input size of 640*640 of the target detection model. Normalize the channel values ​​of the image, that is, divide the values ​​by 256.

[0018] When scaling the image, the image is scaled proportionally while keeping the aspect ratio of the original image unchanged, so that the resolution of the longest side is the same as the input size of the target detection model, and the missing part of the short side is supplemented with 0; When coastal defense monitoring collects video images under rainy and foggy weather conditions, the dark channel prior algorithm is first used to defog the image, and then the image is scaled and normalized.

[0019] In some embodiments, when the target detection and tracking module uses a pre-trained target detection model to output a predicted box marked with a ship, it includes: The video image is input into the pre-trained YOLOv7 target detection model, and the feature maps of the video image at three different scales are obtained through the extraction of the backbone network; The feature maps at three different scales are fused through the FPN network to obtain prediction results of three different sizes; The prediction results are sent to the prediction head structure, and after the heavy parameter convolution and non-maximum suppression algorithm, the prediction box marked with the ship is output.

[0020] In this embodiment, the edge computing unit on the terminal side uses the YOLOv7 target detection model to perform target detection and improve detection accuracy. However, if only the non-maximum suppression algorithm (NMS) is used to remove the detected redundant frames, multiple target frames of different sizes and overlapping with each other may be repeatedly detected for the same ship target, or the bow part may be repeatedly marked. In actual application, these above-mentioned phenomena will lead to incorrect ship target positioning and tracking, thereby affecting the accuracy of ship target detection.

[0021] Therefore, in some implementations, when the target detection and tracking module performs post-processing on the prediction frame and screens the target prediction frame, it includes: According to the size and overlap of the prediction box, the prediction box is screened to obtain the intermediate prediction box; According to the relative position relationship of the bow of the ship, the intermediate prediction frame is screened to obtain the target prediction frame.

[0022] In this embodiment, based on the size and overlap of the prediction frames, the existing prediction frames are screened respectively to remove redundant prediction frames and prediction frames with many overlapping similarities. Then, based on the relative position relationship between the ship and its bow, the possible bow prediction frames are further screened to ensure that the prediction frames are not located at the four corners of the ship, thereby effectively solving the common target overlap and repeated detection problems in ship target detection and improving the accuracy of the detection results and the robustness in complex overlapping scenarios.

[0023] In some implementations, the prediction boxes are screened according to the size and overlap of the prediction boxes to obtain the intermediate prediction boxes, including: Pair the prediction boxes in pairs to obtain matching pairs, and calculate the maximum intersection area of ​​the matching pairs; Determine the matching pair corresponding to the maximum intersection area greater than the first preset threshold as an overlapping matching pair; For each overlapping matching pair, a similarity score value of the overlapping matching pair is calculated according to the size and overlap of the two prediction boxes of the overlapping matching pair; When the similarity score value is less than the preset score threshold, the prediction boxes in the overlapping matching pair are all used as intermediate prediction boxes, and the overlapping matching pair is used as a screening matching pair; When the similarity score value is greater than or equal to a preset score threshold, the prediction box with the smaller maximum intersection area in the overlapping matching pair is used as the intermediate prediction box.

[0024] It should be noted that for the prediction box a and prediction box b Matching pairs, prediction boxes a The intersection area IOA a =( S a ∩ S b ) / S a ; Prediction box b The intersection area IOA b =( S a ∩ S b ) / S b ;in, S a , S b The prediction boxes are a The area and prediction box b The area ofS a ∩ S b For the prediction box a With prediction box b The maximum intersection area of ​​the matching pair = max( IOA a , IOA b ); for example, if IOA a > IOA b , then the maximum intersection area of ​​the matching pairs is IOA a .

[0025] Specifically, the prediction boxes output by the target prediction model are traversed in a pairwise manner to calculate the maximum intersection area between the matching pairs ( IOA ), if the maximum of a pair of matching IOA If the intersection area is greater than the first preset threshold (i.e., the preset maximum intersection area threshold), the pair of matches is regarded as a suspicious overlapping match pair, and then a list of suspicious overlapping match pairs is constructed. Then, the ship prediction frames with high overlapping similarity are further screened out by the preset scoring threshold to obtain the remaining screened match pairs and intermediate prediction frames. For example, in the actual execution process, setting the first preset threshold to 0.85 can effectively screen out the prediction frames with high overlapping areas.

[0026] It should be noted that, for the matching pair corresponding to the maximum intersection area not greater than the first preset threshold, it is directly retained, that is, determined as the intermediate prediction box.

[0027] Specifically, if the similarity score of an overlapping match pair is less than the preset score threshold, the two first prediction frames in the overlapping match pair are retained; otherwise, the prediction frame with the larger IOA in the overlapping match pair is deleted, and the overlapping match pair is removed from the list of suspicious overlapping match pairs. The retained overlapping match pair is the screened match pair, and the retained prediction frame is the intermediate prediction frame. For example, in the actual implementation process, the preset score threshold is set to 0.7, which can more accurately filter out the repeated prediction frames of the same ship target.

[0028] In some embodiments, a similarity score is calculated as follows: Determine the aspect ratio according to the sizes of the two prediction boxes of the overlapping matching pair; The ratio of the minimum aspect ratio to the maximum aspect ratio of the two prediction boxes of the overlapping matching pair is calculated, and the product of the ratio and the maximum intersection area of ​​the two prediction boxes of the overlapping matching pair is used as the similarity score value of the overlapping matching pair.

[0029] Specifically, for each overlapping matching pair in the list of suspicious overlapping matching pairs, the similarity score value of the overlapping matching pair is calculated using the following formula: in, yes ab To include the prediction box a With prediction box b Similarity scores of overlapping matching pairs; IOA a , IOA b They are respectively the predicted boxes of overlapping matching pairs a , prediction box b The maximum intersection area of γ a , γ b The prediction boxes are a With prediction box b aspect ratio; for γ a and γ b The minimum aspect ratio in ; for γ a and γ b The maximum aspect ratio in .

[0030] Since the target detection model for ships usually includes several detection heads for detecting features of different scales, it is used to detect targets of multiple scales. Therefore, for the same ship, due to the different observation scales of different detection heads, different detection heads for the ship output detection results of different sizes, and there is only one ship target, so it is necessary to filter out the redundant detection frames. In the present invention, by considering the aspect ratio to determine the shape similarity of the two prediction frames, not only the interference of the scale change of the prediction frame on the similarity evaluation is avoided, but also the overlapping frames with similar aspect ratios can be more accurately eliminated, and the overlapping frames with dissimilar shapes can be retained; at the same time, by combining the maximum intersection area, the adaptive recognition of ship targets is improved.

[0031] In some implementations, the intermediate prediction frames are screened according to the relative position relationship of the bow of the ship to obtain the target prediction frame, including: Define the intermediate prediction box with a larger size in the screening matching pair as the first prediction box, and the intermediate prediction box with a smaller size as the second prediction box; For each filter match pair, execute: Taking each vertex of the first prediction box as an anchor point, constructing a virtual box with the same size as the second prediction box inside the first prediction box; wherein the number of virtual boxes is the same as the number of anchor points; Calculate the intersection-over-union ratio between the second prediction box and the virtual box; When the intersection-over-union ratio is greater than or equal to a second preset threshold, the second prediction box is deleted to obtain the target prediction box.

[0032] In this embodiment, each screening matching pair includes a first prediction box A with a larger size and a second prediction box B with a smaller size (i.e., the size of A is larger than that of B). For each screening matching pair, the four vertices of A are ( x A1 , y A1 )、( x A2 , y A2 )、( x A3 , y A3 )、( x A4 , y A4 ), the width of A is w B Gao Wei h B . Take the four vertices of A as anchor points and construct a virtual box with the same width and height as B inside A. x A1 , y A1 ) point as an example, the coordinates of the four vertices of the constructed virtual frame are ( x A1 , y A1 )、( x A1 , y A1 + h B )、( x A1 + w B , y A1 + h B )、( x A1 + w B , y A1 );by( x A2 , y A2 ) point as an example, the coordinates of the four vertices of the constructed virtual frame are ( x A2 ,y A2 - h B )、( x A2 , y A2 )、( x A2 + w B , y A2 )、( x A2 + w B , y A2 - h B );by( x A3 , y A3 ) point as an example, the coordinates of the four vertices of the constructed virtual frame are ( x A3 - w B , y A3 - h B )、( x A3 - w B , y A3 )、( x A3 , y A3 )、( x A3 , y A3 - h B );by( x A4 , y A4 ) point as an example, the coordinates of the four vertices of the constructed virtual frame are ( x A4 - w B , y A4 )、( x A4 - w B , y A4 + h B )、( xA4 , y A4 + h B )、( x A4 , y A4 ). After the four virtual frames are constructed, these virtual frames are used to calculate the intersection and union ratio with B. If the calculated intersection and union ratio values ​​are all less than the second preset threshold (i.e., the preset bow intersection and union ratio threshold), B is retained, otherwise B is deleted. In this way, the intermediate prediction frame that is finally retained is the final target prediction frame.

[0033] In the present invention, if the bow of a ship is repeatedly detected, the prediction frame of the bow must be located at the corner of the prediction frame of the ship. Based on the relative position relationship between the bow and the ship target, for the intermediate prediction frame retained after the screening, the prediction frame with a larger IOA in the screening matching pair is continuously filtered to remove the smaller prediction frame that may exist as the bow, and finally the target prediction frame for the ship target is obtained, thereby solving the problem of target frame overlap and repeated detection of the bow position in the existing detection method, and effectively improving the accuracy and robustness of ship target detection.

[0034] The method of the present invention improves the post-processing part of the traditional target detection algorithm. Starting from the prediction frame overlap problem encountered in the actual detection task, the prediction frame is further filtered by using the aspect ratio relationship and IOA. Through multiple screenings and the introduction of relevant knowledge of the inherent characteristics of ship targets, the problems of target frame overlap and repeated detection of the bow position in the detection results are greatly alleviated, so that the ship target detection results can be more accurate, providing more reliable technical support for the smart coastal defense system.

[0035] In some implementations, the target detection and tracking module performs multi-target tracking in the following manner: For each target detection result, execute: Based on the ship type marked in the current target prediction box, the current target is divided. When there is no Kalman tracker under the current ship type, it is determined that the current target appears for the first time, a corresponding potential Kalman tracker is initialized for it, and the first appearance time, ship type, and target prediction box of the ship target are retained; among which, the ship target and the Kalman tracker are in a one-to-one correspondence; When a Kalman tracker exists for the current ship type, the current target prediction frame is used to perform area intersection and union operations with the target frame predicted by each Kalman tracker for the current ship type, and the cost matrix is ​​constructed using the area intersection and union operations. Use the Hungarian algorithm to find the optimal match of the cost matrix. If the match is successful, the target box predicted by the Kalman tracker itself is updated based on the current target prediction box. If the match is not successful, the current target is determined to appear for the first time, a corresponding potential Kalman tracker is initialized for it, and the first appearance time, ship type, and target prediction frame of the ship target are retained; When a potential Kalman tracker successfully matches three times in a row, it is considered that the ship target is real, and a number is assigned to the Kalman tracker based on the order in which the same type of targets appear on the same day; For each numbered Kalman tracker, the target box predicted and updated in real time by the Kalman filter is used as the position of the target for tracking; If a potential Kalman tracker fails to match successfully for five consecutive times, the Kalman tracker will be deleted and the ship target will be considered to have disappeared.

[0036] In this embodiment, by using the target detection result output by the target detection model to update and correct the prediction result of the Kalman tracker, it can be understood that the detection frequency of the target detection model is lower than the prediction frequency of the Kalman tracker. By judging whether the potential Kalman tracker has successfully matched three times in a row, it is judged whether the ship target in the Kalman tracker really exists, and the Kalman tracker containing the real ship target is assigned a number. For each numbered Kalman tracker, the target frame predicted and updated in real time by the Kalman filter is used to track the position of the target. When the tracked target frame presses the line or invades the alarm area, an alarm signal is generated and a screenshot is retained.

[0037] In some embodiments, the edge computing unit further includes: a target storage reporting module; The target storage reporting module is used to store target detection results and target tracking results based on a pre-set storage strategy, and to report results based on the communication quality between the edge computing unit and the backend server.

[0038] In this embodiment, the target storage reporting module mainly includes two processes, the target storage process and the target reporting process. Among them, the target storage process includes: Once the Kalman tracker corresponding to a target generates a number, the original screen image corresponding to the reporting time is saved locally, and the specific information of the target, that is, the appearance date, category, number within the category, first reporting time, device status, target status, target frame position and the storage path of the original image are stored in the client-side database; For each numbered target, execute: Store the target's specific information every 5 seconds and save the corresponding original screen image; If the target triggers a line-crossing or intrusion alarm, the target location, alarm status, and alarm setting information are stored once, and the image of the alarm-triggering behavior is saved; If the target stops triggering the alarm, the target position is stored once and the alarm status is updated; When it disappears, the target information including the disappearance time and the last appearance position of the target is stored.

[0039] In addition, in order to ensure the normal operation of the edge computing unit, the target information and images whose recording time exceeds the specified time limit are regularly cleaned up. For each target information in the end-side database, a reporting flag is used to indicate the transmission status of the information.

[0040] Specifically, the target reporting process includes: When the communication with the back-end network is unobstructed, once there is an unreported target information in the client database, the information and the corresponding image are sent to the back-end receiving server using the Http protocol, and the information is marked as reported; When communication with the back-end network is unavailable, the target information is first stored locally. After the network is unblocked, the target information and corresponding images are reported in the order of reporting time.

[0041] It should be noted that, in order to reduce the pressure on the transmission link, the original image corresponding to the target information may be encoded into Base64 format and then sent to the back-end receiving server.

[0042] In the embodiment of the present invention, the back-end server includes a data receiving module and an instruction transmission module; The data receiving module receives the target data and images sent back by the front end through the Http protocol, stores the target data in the back-end database, decodes the image data and stores it in the back-end image storage device, and cleans up all data that has existed for more than the retention period.

[0043] The command transmission module uses the HTTP protocol to send the target category and alarm area that needs to be controlled, the type of alarm line, endpoint coordinates, and duration to the edge computing unit, thereby realizing remote control of the edge device.

[0044] Please refer to Figure 2 The embodiment of the present invention provides a method for real-time extraction of coastal defense monitoring targets based on any system embodiment of the specification, the method comprising: 200, an edge computing unit connected to the coastal defense monitoring device is provided on the terminal side thereof, and a video image captured in real time by the coastal defense monitoring device is acquired by using the edge computing unit; wherein a target detection and tracking module is provided in the edge computing unit, and the target detection and tracking module contains a pre-trained target detection model; 202, performing target detection on the video image using the target detection model, outputting a prediction frame marked with the ship, and then post-processing the prediction frame to filter out the target prediction frame; 204, performing multi-target tracking using a target detection and tracking module; 206. The edge computing unit sends the stored end-side data to the backend server.

[0045] The content of the above method is based on the same concept as the embodiment of the system of the present invention. For the specific content, please refer to the description in the embodiment of the system of the present invention, and it will not be repeated here.

[0046] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0047] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time extraction system for coastal defense monitoring targets based on edge computing, characterized in that: include: A coastal defense monitoring device for generating video images in real time, an edge computing unit connected to the coastal defense monitoring device for performing real-time analysis and storage on the end side, and a backend server communicating with the edge computing unit for receiving end-side data and transmitting superior instructions; The edge computing unit includes at least a target detection and tracking module, and the target detection and tracking module is used to perform target detection and multi-target tracking based on the video image; The target detection and tracking module performs target detection in the following manner: Use the pre-trained object detection model to output the predicted box labeled with the ship; The prediction box is post-processed to obtain a target prediction box through screening.

2. The system according to claim 1, characterized in that The edge computing unit also includes: a video acquisition module and an image processing module; The video acquisition module is connected to the coastal defense monitoring equipment through a hardware interface to obtain real-time video images; The image processing module is used to pre-process the video image and then send it to the edge computing unit.

3. The system according to claim 1, characterized in that When the target detection and tracking module executes the method of outputting a prediction frame marked with a ship using a pre-trained target detection model, the target detection and tracking module includes: Input the video image into a pre-trained YOLOv7 target detection model, and obtain feature maps of the video image at three different scales through backbone network extraction; The feature maps at three different scales are fused through the FPN network to obtain prediction results of three different sizes; The prediction result is sent to the prediction head structure, and after the heavy parameter convolution and non-maximum suppression algorithm, the prediction box marked with the ship is output.

4. The system according to claim 1, characterized in that When the target detection and tracking module performs post-processing on the prediction frame to screen and obtain the target prediction frame, it includes: According to the size and overlap of the prediction boxes, the prediction boxes are screened to obtain intermediate prediction boxes; According to the relative position relationship of the bow of the ship, the intermediate prediction frame is screened to obtain the target prediction frame.

5. The system according to claim 4, characterized in that The step of screening the prediction boxes according to the size and overlap of the prediction boxes to obtain the intermediate prediction boxes includes: Pair the prediction frames in pairs to obtain matching pairs, and calculate the maximum intersection area of ​​the matching pairs; Determine the matching pairs corresponding to the maximum intersection areas greater than a first preset threshold as overlapping matching pairs; For each of the overlapping matching pairs, calculating a similarity score value of the overlapping matching pair according to the sizes and overlapping conditions of the two prediction boxes of the overlapping matching pair; When the similarity score value is less than a preset score threshold, the prediction boxes in the overlapping matching pair are all used as intermediate prediction boxes, and the overlapping matching pair is used as a screening matching pair; When the similarity score value is greater than or equal to the preset score threshold, the prediction frame with the smaller maximum intersection area in the overlapping matching pair is used as the intermediate prediction frame.

6. The system according to claim 5, characterized in that The similarity score is calculated as follows: Determine the aspect ratio according to the sizes of the two prediction boxes of the overlapping matching pair; The ratio of the minimum aspect ratio to the maximum aspect ratio of the two prediction boxes of the overlapping matching pair is calculated, and the product of the ratio and the maximum intersection area of ​​the two prediction boxes of the overlapping matching pair is used as the similarity score value of the overlapping matching pair.

7. The system according to claim 5, characterized in that The method of screening the intermediate prediction frame according to the relative position relationship of the bow of the ship to obtain the target prediction frame includes: Define the intermediate prediction box with a larger size in the screening matching pair as a first prediction box, and the intermediate prediction box with a smaller size as a second prediction box; For each of the screening matching pairs, execute: Taking each vertex of the first prediction box as an anchor point, constructing a virtual box with the same size as the second prediction box inside the first prediction box; wherein the number of the virtual boxes is the same as the number of the anchor points; Calculating an intersection-over-union ratio between the second prediction box and the virtual box; When the intersection-over-union ratio is greater than or equal to a second preset threshold, the second prediction box is deleted to obtain a target prediction box.

8. The system of claim 1, wherein: The target detection and tracking module performs multi-target tracking in the following manner: For each target detection result, execute: Based on the ship type marked in the current target prediction box, the current target is divided. When there is no Kalman tracker under the current ship type, it is determined that the current target appears for the first time, a corresponding potential Kalman tracker is initialized for it, and the first appearance time, ship type, and target prediction box of the ship target are retained; wherein the ship target and the Kalman tracker are in a one-to-one correspondence; When a Kalman tracker exists for the current ship type, the current target prediction frame is used to perform area intersection and union operations with the target frame predicted by each Kalman tracker for the current ship type, and the cost matrix is ​​constructed using the area intersection and union operations. The Hungarian algorithm is used to find the optimal match of the cost matrix. If the match is successful, the target frame predicted by the Kalman tracker itself is updated based on the current target prediction frame. If the match is not successful, the current target is determined to appear for the first time, a corresponding potential Kalman tracker is initialized for it, and the first appearance time, ship type, and target prediction frame of the ship target are retained; When a potential Kalman tracker successfully matches three times in a row, it is considered that the ship target is real, and a number is assigned to the Kalman tracker based on the order in which the same type of targets appear on the same day; For each numbered Kalman tracker, the target box predicted and updated in real time by the Kalman filter is used as the position of the target for tracking; If a potential Kalman tracker fails to match successfully for five consecutive times, the Kalman tracker will be deleted and the ship target will be considered to have disappeared.

9. The system according to claim 1, characterized in that The edge computing unit also includes: a target storage reporting module; The target storage reporting module is used to store the target detection results and the target tracking results based on a preset storage strategy, and to report the results based on the communication quality between the edge computing unit and the backend server.

10. A method for real-time extraction of coastal defense monitoring targets based on the system according to any one of claims 1 to 9, characterized in that: include: An edge computing unit connected to the coastal defense monitoring device is provided on the terminal side thereof, and the video image captured in real time by the coastal defense monitoring device is acquired by using the edge computing unit; wherein the edge computing unit is provided with a target detection and tracking module, and the target detection and tracking module contains a pre-trained target detection model; Using the target detection model to perform target detection on the video image, after outputting a prediction frame marked with a ship, post-processing the prediction frame to screen and obtain a target prediction frame; Using the target detection and tracking module to perform multi-target tracking; The edge computing unit sends the stored end-side data to the back-end server.

Images