A marine target detection and positioning system and method combined with Beidou communication

Abstract

The present invention relates to a marine target detection and positioning system and method combined with Beidou communication. The terminal uses a detection module to obtain images of surrounding scenes at sea, identifies all marine targets therefrom, and then selects all abnormal targets therefrom, tracks one of the abnormal targets, and calculates the relative positions of other abnormal targets and the tracked abnormal targets through a multi-target position calculation method based on laser ranging calibration, and then calculates the absolute accurate position information of all abnormal targets based on the current position information, current heading information, horizontal and pitch angle information of the pan-tilt module, and target relative position obtained by the Beidou-3 positioning communication module, and finally packages and sends target-related data including target images and target positions to a server; the server decodes and integrates the data packets to form abnormal target position distribution visualization information containing various types of information. The system and method are conducive to accurately identifying and locating abnormal targets at sea, and then visually displaying them.

Description

Technical Field

[0001] The present invention relates to the technical field of marine target detection and positioning, and in particular to a marine target detection and positioning system and method combined with Beidou communication. Background Art

[0002] In the prior art, in order to obtain the relative position and distance of the target, optoelectronic detection equipment such as laser radar is generally used to detect the target. However, this target detection method cannot obtain images of the target and the surrounding scene, and it is difficult to distinguish whether the target is the type required for detection. Although some detection methods using visual imaging systems use deep learning neural networks to identify targets, they can only judge the type of target and cannot determine whether the target is an abnormal target. They also need to use other ranging equipment to obtain the relative position, and the information fusion of the target and the ranging equipment in the image is also relatively complex. Another solution is to only send the current position when reporting to the service center. This method cannot accurately obtain the target position when the target is far away from the current position. In addition, the existing detection equipment locks on a single target information, and it is difficult to analyze the on-site situation at sea after reporting to the background service center. Summary of the invention

[0003] The purpose of the present invention is to provide a marine target detection and positioning system and method combined with Beidou communication, which is conducive to accurately identifying and locating abnormal targets at sea, and then performing visual display.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is: a marine target detection and positioning system combined with Beidou communication, including a terminal and a server, wherein the terminal includes:

[0005] The detection module is installed on the pan-tilt module and includes an image acquisition module and a laser ranging module. The image acquisition module is used to acquire images of surrounding scenes at sea. The laser ranging module is used to measure the distance between the target located at the center of the image acquisition module screen and the detection module. The detection module sends the acquired image and ranging data to the calculation and storage module.

[0006] The pan / tilt module is used to change the detection angle of the detection module and send the detection angle data to the calculation and storage module;

[0007] A touch display module is used to display the image acquired by the image acquisition module in real time, and to control the horizontal and pitch rotation angles of the pan / tilt module by touch;

[0008] The computing and storage module is used for the logic control of the entire terminal and the processing and analysis of the received images, ranging, detection angle of view and satellite positioning data, including target recognition and abnormal target judgment, controlling the movement of the pan / tilt module to change the detection angle of view to track abnormal targets, calculating the position of abnormal targets, and compressing and packaging the final target data; and

[0009] BeiDou-3 positioning and communication module, used to obtain satellite positioning data and send it to the computing and storage module and the server, send target data to the server, and receive feedback information from the server;

[0010] The server includes:

[0011] The data transceiver module is used to receive the target data sent by the terminal and send the server feedback information to the terminal;

[0012] An information processing module, used to decode and merge data packets, recover target images and other target data; and

[0013] The visualization platform is used to display the location distribution of abnormal targets, as well as target images, time, target type, text notes, and trajectory information.

[0014] The present invention also provides a method for detecting and positioning marine targets in combination with Beidou communication based on the above system, wherein the terminal uses a detection module to detect the surrounding situation at sea in real time and obtain an on-site scene image, then identifies all marine targets from the obtained image, and then screens out all abnormal targets from all marine targets; then tracks the abnormal target closest to the center of the picture, and calculates the relative positions of other abnormal targets and the tracked abnormal target by a multi-target position calculation method based on laser ranging calibration; then calculates the absolute and accurate position information of all abnormal targets based on the current position information, current heading information, horizontal and pitch angle information of the pan-tilt module, and target relative position obtained by the Beidou-3 positioning and communication module; finally, compresses and encodes the current target image, and packages the target data including the abnormal target position, time, target type, and text remarks, and sends the target data in packets to the server through the Beidou-3 positioning and communication module;

[0015] After receiving the data packet from the terminal, the server decodes and integrates the data packet to restore the target image and the abnormal target location information at different times; the abnormal target is marked on the map according to the spatial position relationship, and multi-dimensional data including the target latitude and longitude, time, surrounding scene images, and trajectory are displayed to form a visualization information of the abnormal target location distribution containing various types of information.

[0016] Furthermore, all marine targets are identified from the acquired images, and then all abnormal targets are screened out from all marine targets by combining the pre-stored abnormal target label table and the pre-stored abnormal target template table, including sea surface boundary line calculation, first-stage target type screening and second-stage target individual screening. The specific implementation method is as follows:

[0017] 1) Marine targets only exist on the sea surface, so the sky and sea surface areas are first segmented to reduce the detection range and improve the detection time; when the optical axis of the image acquisition module is parallel to the sea level, it is considered that the horizontal midline of the image corresponds to the sea surface boundary line; therefore, the position of the sea surface boundary line on the image is calculated according to the pitch angle of the image acquisition module, that is, the angle β between the optical axis of the image acquisition module and the sea level;

[0018]

[0019] Among them, μ is the sensor pixel size, f is the focal length of the lens, Δy is the distance between the position of the sea surface boundary line on the image and the horizontal midline of the image, and the position of the sea surface boundary line on the image H is the height of the image; then according to y b Segmenting the image to perform target recognition on the segmented sea surface image;

[0020] 2) Use the neural network model to identify all sea targets from sea surface images;

[0021] The neural network model assigns different loss weights to feature maps of different scales to enhance the detection performance of small targets at sea;

[0022] L max =aL max class +bL max local +cL max con

[0023] L mid =aL mid class +bL mid local +cL mid con

[0024] L min =aL min class +bL min local +cL min con

[0025] Among them, L class , L local, L con are the classification loss, positioning loss and confidence loss of each scale respectively, L max , L mid , L min They are the total losses of each scale respectively; the total loss Loss of all scales is the weighted total loss of each scale:

[0026] Loss = cL max +dL mid +eL min

[0027] Among them, c, d, and e are the weights of the total loss at different scales;

[0028] The types of each marine target identified by the neural network model are checked one by one with all target types in the pre-stored abnormal target label table. If the same target type exists in the pre-stored abnormal target label table, the marine target is determined to be an abnormal target type. The first stage target type screening can only determine whether the marine target belongs to the abnormal target type that needs further judgment, but cannot determine whether the marine target is an abnormal target.

[0029] 3) For all marine targets that are determined to be abnormal target types, use the corresponding pre-stored abnormal target template table to compare and screen out abnormal targets; the comparison method is as follows:

[0030]

[0031] Where Ω is the pixel range of template T, T(x',y') is the value of the abnormal target template image in the abnormal target template table stored at the position (x',y'), the position (x,y) is the central pixel position of the target to be compared, and I(x+x',y+y') is the pixel value of the image where the target to be compared is located; when r is less than the set threshold r min , the sea target is determined to be an abnormal target.

[0032] Furthermore, the specific method for tracking abnormal targets is:

[0033] First, save the abnormal target image as a tracking template, then calculate the horizontal and pitch angles that the pan / tilt module needs to rotate, and place the abnormal target at the center of the current screen of the image acquisition module by rotating the pan / tilt module; if the coordinates of the abnormal target in the current screen are (x0, y0), calculate the required rotation angle of the pan / tilt module by the following formula:

[0034]

[0035]

[0036] Among them, θ xis the horizontal rotation angle, rightward is positive, θ y is the pitch rotation angle, with upward being positive, W and H are the width and height of the image respectively, μ is the sensor pixel size, and f is the focal length of the lens;

[0037] When the abnormal target is in motion, rotating the PTZ module once cannot completely move the abnormal target to the center of the current screen. The following formula is used to determine whether the abnormal target is already in the center of the current screen:

[0038] r T =∑ x',y' (M(x',y')-I R (x+x',y+y')) 2

[0039] Among them, M(x,y) is the pre-stored tracking template, I R (x, y) is the contrast image cropped at the center of the image according to the size of the tracking template; when r T Less than the set threshold T min When r T Greater than or equal to the set threshold T min Stop tracking when

[0040] Furthermore, the calculation method of the abnormal target position is:

[0041] 1) When the abnormal target being tracked is at the center of the image, the optical axis of the laser ranging module is aligned with the abnormal target, and the distance obtained by the laser ranging module is S. Then the relative distance between the center target of the image, that is, the abnormal target being tracked, and the detection module is l AC =Ssinβ, β is the pitch angle of the gimbal module;

[0042] 2) When there are multiple abnormal targets within the detection range, the relative positions of other abnormal targets and the central target of the image are calculated by a multi-target position calculation method based on laser ranging calibration, without the need to rotate the pan / tilt module to align other abnormal targets one by one with the detection optical axis;

[0043] Assume that the coordinates of the central target in the image are (x0, y0), and the coordinates of other abnormal targets in the image are (x1, y1), then the angle γ between the central target and other abnormal targets and the detection module is:

[0044]

[0045] The relative distance between other abnormal targets and the detection module in the y-axis direction is:

[0046] l AB =Stan(β-γ)cosβ

[0047] The relative distance between other abnormal targets and the detection module in the x-axis direction is:

[0048]

[0049] The relative distances between other abnormal targets and the detection module are:

[0050]

[0051] The absolute and accurate position information of all abnormal targets is calculated by combining the current position information, current heading information, and the horizontal and pitch angle information of the gimbal module obtained by the Beidou-3 positioning and communication module. The specific calculation formula is:

[0052]

[0053]

[0054] Among them, l is the relative distance between the abnormal target and the detection module. For the target in the center of the picture, it is the distance l AC , for other abnormal targets, it is the distance l′, Δα is the angle between the line connecting the abnormal target and the detection module and the meridian where the detection module is located, counterclockwise is positive, R is the radius of the earth, D and L are the south latitude and east longitude of the detection module, D0 and L0 are the south latitude and east longitude of the abnormal target; the calculation results of north latitude and west longitude can be opposite.

[0055] Compared with the prior art, the present invention has the following beneficial effects: the system and method combine image recognition, laser ranging and Beidou communication technology, and can conduct large-scale detection and monitoring of the marine environment, accurately identify abnormal targets from monitoring images, calculate the precise location of abnormal targets such as dangerous targets and distressed targets, and mark abnormal targets on the map; at the same time, the target distribution, target trajectory, and the scene image around the target, as well as the target position, time, type and other status information can be viewed through the server-side visualization platform. The present invention can provide the monitoring center with the overall distribution of abnormal targets at sea, as well as the multi-dimensional status data information of each abnormal target, and conduct comprehensive monitoring and management of abnormal events such as faulty ships, people in distress, illegal fishing, etc. at sea based on the visualization platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a block diagram of the system implementation principle of an embodiment of the present invention;

[0057] Figure 2 It is a schematic diagram of the multi-target position calculation principle in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0059] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0060] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0061] like Figure 1 As shown, this embodiment provides a marine target detection and positioning system combined with Beidou communication, including a terminal and a server. The terminal includes a detection module, a pan / tilt module, a touch display module, a computing and storage module, and a Beidou-3 positioning and communication module.

[0062] The detection module is installed on the pan-tilt module, and includes an image acquisition module and a laser ranging module. The image acquisition module is used to obtain images of surrounding scenes at sea, and the laser ranging module is used to measure the distance between the target located at the center of the image acquisition module screen and the detection module. The detection module sends the obtained image and ranging data to the calculation and storage module.

[0063] The pan / tilt module is used to change the detection viewing angle of the detection module and send the detection viewing angle data to the calculation and storage module.

[0064] The touch display module is used to display the image acquired by the image acquisition module in real time, and to control the horizontal and pitch rotation angles of the pan / tilt module through touch.

[0065] The computing and storage module is used for the logical control of the entire terminal and for processing and analyzing the received images, ranging, detection angles and satellite positioning data, including target recognition and abnormal target judgment in images, controlling the movement of the pan / tilt module to change the detection angle to track abnormal targets, calculating the positions of abnormal targets, and compressing and packaging the final target data.

[0066] The BeiDou-3 positioning and communication module is used to obtain satellite positioning data and send it to the computing and storage module and the server, send target data such as target image and position to the server, and receive feedback information from the server.

[0067] The server includes a data transceiver module, an information processing module and a visualization platform.

[0068] The data transceiver module is used to receive target data sent by the terminal and send server feedback information to the terminal.

[0069] The information processing module is used to decode and merge data packets, and restore target images and other target data.

[0070] The visualization platform is used to display the location distribution of abnormal targets, as well as target images, time, target type, text notes, trajectories and other information.

[0071] This embodiment also provides a method for detecting and positioning maritime targets in combination with Beidou communication based on the above system. The terminal uses a detection module to detect the surrounding situation at sea in real time and obtain an on-site scene image, then identifies all maritime targets from the acquired image, and then screens out all abnormal targets from all maritime targets; then tracks the abnormal target closest to the center of the picture, and calculates the relative positions of other abnormal targets and the tracked abnormal target through a multi-target position calculation method based on laser ranging calibration; then calculates the absolute and accurate position information of all abnormal targets based on the current position information, current heading information, horizontal and pitch angle information of the gimbal module, and target relative position obtained by the Beidou-3 positioning and communication module; finally, compresses and encodes the current target image, and packages target data such as the abnormal target position, time, target type, and text remarks, and sends the target data in packets to the server through the Beidou-3 positioning and communication module.

[0072] After receiving the data packet from the terminal, the server decodes and integrates the data packet to restore the target image, abnormal target location at different times and other information; the abnormal target is marked on the map according to the spatial position relationship, and multi-dimensional data such as the target latitude and longitude, time, surrounding scene image, trajectory, etc. are displayed to form a visualization information of the abnormal target location distribution containing various types of information.

[0073] In this embodiment, the terminal identifies all marine targets from the acquired images, and then screens out all abnormal targets from all marine targets in combination with the pre-stored abnormal target label table and the pre-stored abnormal target template table, including sea surface boundary line calculation, first-stage target type screening, and second-stage target individual screening. The specific implementation method is as follows:

[0074] 1) Maritime targets only exist on the sea surface, so the sky and sea surface areas are first segmented to reduce the detection range and improve the detection time. When the optical axis of the image acquisition module is parallel to the sea level, it is considered that the horizontal midline of the image corresponds to the sea surface boundary line. Therefore, the position of the sea surface boundary line on the image is calculated based on the pitch angle of the image acquisition module, that is, the angle β between the optical axis of the image acquisition module and the sea level.

[0075]

[0076] Among them, μ is the sensor pixel size, f is the focal length of the lens, Δy is the distance between the position of the sea surface boundary line on the image and the horizontal midline of the image, and the position of the sea surface boundary line on the image H is the height of the image; then according to y b The image is segmented to perform target recognition on the segmented sea surface image.

[0077] 2) Use the neural network model to identify all sea targets from sea surface images.

[0078] In the process of detecting dangerous and abnormal targets at sea, most of them are small targets at long distances. For the network structure that can extract multi-scale features, the same loss function will reduce the accuracy of small target prediction when training. In this method, the neural network model assigns different loss weights to feature maps of different scales to enhance the detection performance of small targets at sea.

[0079] L max =aL max class +bL max local +cL max con

[0080] L mid =aL mid class +bL mid local +cL mid con

[0081] L min =aL min class +bL min local +cL min con

[0082] Among them, L class , L local , L con are the classification loss, positioning loss and confidence loss of each scale respectively, L max , L mid , L min The total loss of each scale is the weighted total loss of each scale:

[0083] Loss = cL max +dL mid +eL min

[0084] Among them, c, d, and e are the weights of total losses at different scales. For targets at sea, c=0.2 and d=e=0.4 can be set.

[0085] There are fewer types of targets at sea than in public datasets. Therefore, during training, public data can be screened to remove targets that are unlikely to appear at sea, such as cars and trees, or target labels can be changed according to custom data labels. Then, the 1×1 convolution N channel of the neural network head can be changed to 3×(5+Num class ), Num class is the number of dataset labels.

[0086] The types of each maritime target identified by the neural network model are checked one by one with all target types in the pre-stored abnormal target label table. If the same target type exists in the pre-stored abnormal target label table, the maritime target is determined to be an abnormal target type. The first stage target type screening can only determine whether the maritime target belongs to the abnormal target type that needs further judgment, but cannot determine whether the maritime target is an abnormal target.

[0087] 3) For all marine targets that are determined to be abnormal target types, use the corresponding pre-stored abnormal target template table to compare and screen out abnormal targets; the comparison method is as follows:

[0088]

[0089] Where Ω is the pixel range of template T, T(x',y') is the value of the abnormal target template image in the abnormal target template table stored at the position (x',y'), the position (x,y) is the central pixel position of the target to be compared, and I(x+x',y+y') is the pixel value of the image where the target to be compared is located; when r is less than the set threshold r min , the sea target is determined to be an abnormal target.

[0090] In this embodiment, the specific method for tracking abnormal targets is:

[0091] First, save the abnormal target image as a tracking template, then calculate the horizontal and pitch angles that the pan / tilt module needs to rotate, and place the abnormal target at the center of the current screen of the image acquisition module by rotating the pan / tilt module; if the coordinates of the abnormal target in the current screen are (x0, y0), calculate the required rotation angle of the pan / tilt module by the following formula:

[0092]

[0093]

[0094] Among them, θ x is the horizontal rotation angle, rightward is positive, θ yis the pitch rotation angle, with upward being positive, W and H are the width and height of the image respectively, μ is the sensor pixel size, and f is the focal length of the lens.

[0095] When the abnormal target is in motion, rotating the PTZ module once cannot completely move the abnormal target to the center of the current screen. The following formula is used to determine whether the abnormal target is already in the center of the current screen:

[0096] r T =∑ x',y' (M(x',y')-I R (x+x',y+y')) 2

[0097] Among them, M(x,y) is the pre-stored tracking template, I R (x, y) is the contrast image cropped at the center of the image according to the size of the tracking template; when r T Less than the set threshold T min When r T Greater than or equal to the set threshold T min Stop tracking when

[0098] In this embodiment, the calculation method of the abnormal target position is:

[0099] 1) When the abnormal target being tracked is at the center of the image, the optical axis of the laser ranging module is aligned with the abnormal target, and the distance obtained by the laser ranging module is S. Then the relative distance between the center target of the image, that is, the abnormal target being tracked, and the detection module is l AC =Ssinβ, β is the pitch angle of the gimbal module.

[0100] 2) When there are multiple abnormal targets within the detection range, this embodiment calculates the relative positions of other abnormal targets and the central target of the picture through a multi-target position calculation method based on laser ranging calibration, without the need to rotate the pan-tilt module to align other abnormal targets one by one with the detection optical axis.

[0101] The principle of multi-target position calculation based on laser ranging calibration is as follows Figure 2 As shown in the figure, AC is the sea level line; C is the center target of the picture; B is the second target, that is, other abnormal targets except the center target of the picture; c and b are the pixel positions of the target corresponding to the image.

[0102] Assume that the coordinates of the central target in the image are (x0, y0), and the coordinates of other abnormal targets in the image are (x1, y1), then the angle γ between the central target and other abnormal targets and the detection module is:

[0103]

[0104] The direction parallel to AC is the y-axis direction, and the direction perpendicular to AC and parallel to the sea level is the x-axis direction. The relative distance between other abnormal targets and the detection module in the y-axis direction is:

[0105] l AB =Stan(β-γ)cosβ

[0106] The relative distance between other abnormal targets and the detection module in the x-axis direction is:

[0107]

[0108] Finally, the relative distance between other abnormal targets and the detection module is:

[0109]

[0110] The absolute and accurate position information of all abnormal targets is calculated by combining the current position information, current heading information, and the horizontal and pitch angle information of the gimbal module obtained by the Beidou-3 positioning and communication module. The specific calculation formula is:

[0111]

[0112]

[0113] Among them, l is the relative distance between the abnormal target and the detection module. For the target in the center of the picture, it is the distance l AC , for other abnormal targets, it is the distance l′, Δα is the angle between the line connecting the abnormal target and the detection module and the meridian where the detection module is located, counterclockwise is positive, R is the radius of the earth, D and L are the south latitude and east longitude of the detection module, D0 and L0 are the south latitude and east longitude of the abnormal target; the calculation results of north latitude and west longitude can be opposite.

[0114] In this embodiment, the target data is packaged and sent, including the target image, latitude and longitude coordinates, time, target type, remark message and other target multi-dimensional data, and sent through Beidou RDSS short message. First, the target image is compressed and encoded using the HEVC-MSP coding standard, and then the image data is divided into N packets according to the byte size occupied by the encoded image data. When each packet of data is sent, the total number of packets and the current number of packets are recorded. When the total number is equal to the current number of packets, it means that the image data has been sent successfully. Finally, the same type of data is packaged into a data packet and sent in sequence. The server starts decoding the target data when the last data type is received.

[0115] In this embodiment, the working process of the system is as follows:

[0116] 1) The terminal system is powered on and waits for program initialization, including the serial port initialization of the BeiDou-3 positioning communication module and the establishment of a connection between the terminal and the server.

[0117] 2) After the terminal program is initialized, you can choose between cruise mode and manual mode. In cruise mode, the gimbal direction and lens focal length rotate and detect according to a certain trajectory to automatically detect abnormal situations at sea; in manual mode, users can freely control the detection direction and field of view through the multi-touch display.

[0118] 3) When the terminal finds a target, it compares the target label with the abnormal target label table to determine whether it is an abnormal target type. You can also manually add the target to the abnormal target label table, and the next time you detect a target of this type, it will be directly judged as an abnormal target type.

[0119] 4) When the current target is an abnormal target type, the abnormal target individual screening is performed to determine whether it is an abnormal target. The target can also be manually added to the abnormal target template table.

[0120] 5) Users can add text messages through the soft keyboard of the touch display module. After confirming the abnormal target, the system waits for the user to add text notes, and sends the text notes and target image, location and other data to the server. If the user does not operate for a certain period of time, the system automatically reports the event. You can also set the full automatic mode to send abnormal events to the server directly without waiting for adding notes.

[0121] 6) After the terminal reports an abnormal event, the progress status will be displayed in the feedback bar of the terminal interface. The next event will be reported after the last event is reported. The user can also manually terminate the event reporting.

[0122] 7) After receiving the abnormal event reported by the terminal, the server decodes and restores the corresponding target position and the image around the target, marks the target on the map according to the multiple position data of the target, draws the target trajectory, and displays the target's position, time, type, surrounding scene image and the comment message added by the terminal user at the target mark point on the map.

[0123] The present invention effectively combines BeiDou satellite positioning, BeiDou-3 RDSS short message communication, optical monitoring, image processing and image target ranging technology to conduct large-scale detection and accurate identification of special targets such as abnormal targets and distressed targets at sea, and fuses the captured target image, target type and target position information calculated by BeiDou positioning to draw the distribution of various special targets at sea on the service end, including time, target trajectory, scene image of the target location and other types of information, so as to facilitate the service center to monitor and manage abnormal situations at sea. The system and method can be applied to:

[0124] 1) Fisheries monitoring: Monitor key fishing areas to ensure the protection of aquatic species and compliance with fishery regulations, and prevent illegal fishing and illegal operations.

[0125] 2) Water traffic monitoring: monitor international waters and straits, guide ships, monitor violations, and take photos of violation sites for evidence.

[0126] 3) Piracy and Illegal Activity Monitoring: Monitor maritime threats, and promptly detect and report attacks on ships and illegal activities.

[0127] 4) Maritime detection and search and rescue: Detect faulty ships and people in distress at sea, accurately calculate the location of the distressed targets, and send rescue messages and on-site images to the command center.

[0128] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0129] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0132] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A marine target detection and positioning system combined with Beidou communication, characterized in that: It includes a terminal and a server, and the terminal includes: The detection module is installed on the pan-tilt module and includes an image acquisition module and a laser ranging module. The image acquisition module is used to acquire images of surrounding scenes at sea. The laser ranging module is used to measure the distance between the target located at the center of the image acquisition module screen and the detection module. The detection module sends the acquired image and ranging data to the calculation and storage module. The pan / tilt module is used to change the detection angle of the detection module and send the detection angle data to the calculation and storage module; A touch display module is used to display the image acquired by the image acquisition module in real time, and to control the horizontal and pitch rotation angles of the pan / tilt module by touch; The computing and storage module is used for the logic control of the entire terminal and the processing and analysis of the received images, ranging, detection angle of view and satellite positioning data, including target recognition and abnormal target judgment, controlling the movement of the pan / tilt module to change the detection angle of view to track abnormal targets, calculating the position of abnormal targets, and compressing and packaging the final target data; and BeiDou-3 positioning and communication module, used to obtain satellite positioning data and send it to the computing and storage module and the server, send target data to the server, and receive feedback information from the server; The server includes: The data transceiver module is used to receive the target data sent by the terminal and send the server feedback information to the terminal; An information processing module, used to decode and merge data packets, recover target images and other target data; and The visualization platform is used to display the location distribution of abnormal targets, as well as target images, time, target type, text notes, and trajectory information.

2. A method for detecting and locating marine targets in combination with Beidou communication based on the system of claim 1, characterized in that: The terminal uses the detection module to detect the surrounding situation at sea in real time and obtain the scene image, then identifies all the targets at sea from the acquired images, and then selects all the abnormal targets from all the targets at sea; then tracks the abnormal target closest to the center of the picture, and calculates the relative position of other abnormal targets and the tracked abnormal target through the multi-target position calculation method based on laser ranging calibration; then calculates the absolute and accurate position information of all abnormal targets based on the current position information, current heading information, horizontal and pitch angle information of the gimbal module and the relative position of the target obtained by the Beidou-3 positioning and communication module; finally, compresses and encodes the current target image, and packages the target data including the abnormal target position, time, target type and text remarks, and sends the target data to the server through the Beidou-3 positioning and communication module; After receiving the data packet from the terminal, the server decodes and integrates the data packet to restore the target image and the abnormal target location information at different times; the abnormal target is marked on the map according to the spatial position relationship, and multi-dimensional data including the target latitude and longitude, time, surrounding scene images and trajectory are displayed to form a visualization information of the abnormal target location distribution containing various types of information.

3. The method for detecting and locating marine targets in combination with Beidou communication according to claim 2, characterized in that: Identify all marine targets from the acquired images, and then filter out all abnormal targets from all marine targets by combining the pre-stored abnormal target label table and the pre-stored abnormal target template table, including sea surface boundary line calculation, first-stage target type screening and second-stage target individual screening. The specific implementation method is as follows: 1) Marine targets only exist on the sea surface, so the sky and sea surface areas are first segmented to reduce the detection range and improve the detection time; when the optical axis of the image acquisition module is parallel to the sea level, it is considered that the horizontal midline of the image corresponds to the sea surface boundary line; therefore, the position of the sea surface boundary line on the image is calculated according to the pitch angle of the image acquisition module, that is, the angle β between the optical axis of the image acquisition module and the sea level; Among them, μ is the sensor pixel size, f is the focal length of the lens, Δy is the distance between the position of the sea surface boundary line on the image and the horizontal midline of the image, and the position of the sea surface boundary line on the image H is the height of the image; then according to y b Segmenting the image to perform target recognition on the segmented sea surface image; 2) Use the neural network model to identify all sea targets from sea surface images; The neural network model assigns different loss weights to feature maps of different scales to enhance the detection performance of small targets at sea; <h2 style=";text-align:left;direction:ltr">L<h2 style=";text-align:left;direction:ltr"> max <h2 style=";text-align:left;direction:ltr"> =aL<h2 style=";text-align:left;direction:ltr"> max <h2 style=";text-align:left;direction:ltr"> class <h2 style=";text-align:left;direction:ltr"> +bL<h2 style=";text-align:left;direction:ltr"> max <h2 style=";text-align:left;direction:ltr"> local <h2 style=";text-align:left;direction:ltr"> +cL<h2 style=";text-align:left;direction:ltr"> max <h2 style=";text-align:left;direction:ltr"> con <h2 style=";text-align:left;direction:ltr">L<h2 style=";text-align:left;direction:ltr"> mid <h2 style=";text-align:left;direction:ltr"> =aL<h2 style=";text-align:left;direction:ltr"> mid <h2 style=";text-align:left;direction:ltr"> class <h2 style=";text-align:left;direction:ltr"> +bL<h2 style=";text-align:left;direction:ltr"> mid <h2 style=";text-align:left;direction:ltr"> local <h2 style=";text-align:left;direction:ltr"> +cL<h2 style=";text-align:left;direction:ltr"> mid <h2 style=";text-align:left;direction:ltr"> con <h2 style=";text-align:left;direction:ltr">L<h2 style=";text-align:left;direction:ltr"> min <h2 style=";text-align:left;direction:ltr"> =aL<h2 style=";text-align:left;direction:ltr"> min <h2 style=";text-align:left;direction:ltr"> class <h2 style=";text-align:left;direction:ltr"> +bL<h2 style=";text-align:left;direction:ltr"> min <h2 style=";text-align:left;direction:ltr"> local <h2 style=";text-align:left;direction:ltr"> +cL<h2 style=";text-align:left;direction:ltr"> min <h2 style=";text-align:left;direction:ltr"> con Among them, L class , L local , L con are the classification loss, positioning loss and confidence loss of each scale respectively, L max , L mid , L min They are the total losses of each scale respectively; the total loss Loss of all scales is the weighted total loss of each scale: Loss=cL max +dL mid +eL min Among them, c, d, and e are the weights of the total loss at different scales; The types of each marine target identified by the neural network model are checked one by one with all target types in the pre-stored abnormal target label table. If the same target type exists in the pre-stored abnormal target label table, the marine target is determined to be an abnormal target type. The first stage target type screening can only determine whether the marine target belongs to the abnormal target type that needs further judgment, but cannot determine whether the marine target is an abnormal target. 3) For all marine targets that are determined to be abnormal target types, use the corresponding pre-stored abnormal target template table to compare and screen out abnormal targets; the comparison method is as follows: Where Ω is the pixel range of template T, T(x',y') is the value of the abnormal target template image in the abnormal target template table stored at the position (x',y'), the position (x,y) is the central pixel position of the target to be compared, and I(x+x',y+y') is the pixel value of the image where the target to be compared is located; when r is less than the set threshold r min , the sea target is determined to be an abnormal target.

4. The method for detecting and locating marine targets in combination with Beidou communication according to claim 2, characterized in that: The specific method for tracking abnormal targets is: First, save the abnormal target image as a tracking template, then calculate the horizontal and pitch angles that the pan / tilt module needs to rotate, and place the abnormal target at the center of the current screen of the image acquisition module by rotating the pan / tilt module; if the coordinates of the abnormal target in the current screen are (x0, y0), calculate the required rotation angle of the pan / tilt module by the following formula: Among them, θ x is the horizontal rotation angle, rightward is positive, θ y is the pitch rotation angle, with upward being positive, W and H are the width and height of the image respectively, μ is the sensor pixel size, and f is the focal length of the lens; When the abnormal target is in motion, rotating the PTZ module once cannot completely move the abnormal target to the center of the current screen. The following formula is used to determine whether the abnormal target is already in the center of the current screen: r T =∑ x',y' (M(x',y')-I R (x+x',y+y')) 2 Among them, M(x,y) is the pre-stored tracking template, I R (x, y) is the contrast image cropped at the center of the image according to the size of the tracking template; when r T Less than the set threshold T min When r T Greater than or equal to the set threshold T min Stop tracking when 5. The method for detecting and locating marine targets in combination with Beidou communication according to claim 2, characterized in that: The calculation method of abnormal target position is: 1) When the abnormal target being tracked is at the center of the image, the optical axis of the laser ranging module is aligned with the abnormal target, and the distance obtained by the laser ranging module is S. Then the relative distance between the center target of the image, that is, the abnormal target being tracked, and the detection module is l AC =Ssinβ, β is the pitch angle of the gimbal module; 2) When there are multiple abnormal targets within the detection range, the relative positions of other abnormal targets and the central target of the image are calculated by a multi-target position calculation method based on laser ranging calibration, without the need to rotate the pan / tilt module to align other abnormal targets one by one with the detection optical axis; Assume that the coordinates of the central target in the image are (x0, y0), and the coordinates of other abnormal targets in the image are (x1, y1), then the angle γ between the central target and other abnormal targets and the detection module is: The relative distance between other abnormal targets and the detection module in the y-axis direction is: l AB =S tan(β-γ)cosβ The relative distance between other abnormal targets and the detection module in the x-axis direction is: The relative distances between other abnormal targets and the detection module are: The absolute and accurate position information of all abnormal targets is calculated by combining the current position information, current heading information, and the horizontal and pitch angle information of the gimbal module obtained by the Beidou-3 positioning and communication module. The specific calculation formula is: Among them, l is the relative distance between the abnormal target and the detection module. For the target in the center of the picture, it is the distance l AC , for other abnormal targets, it is the distance l′, Δα is the angle between the line connecting the abnormal target and the detection module and the meridian where the detection module is located, counterclockwise is positive, R is the radius of the earth, D and L are the south latitude and east longitude of the detection module, D0 and L0 are the south latitude and east longitude of the abnormal target; the calculation results of north latitude and west longitude can be opposite.

Images