Marine visual tracking system and method

Abstract

The invention discloses a marine visual tracking system and method, and relates to the field of marine autonomous tracking; the system comprises: a three-axis pan-tilt head, a visual control system and a wave glider; the visual control system issues an operation instruction according to position information of a preset task area; the three-axis pan-tilt head determines a travel route according to the operation instruction; the wave glider travels to the preset task area according to the travel route; the visual control system acquires image data of the preset task area in real time, detects and identifies the image data to obtain identification data, then calculates state data of the three-axis pan-tilt head according to the identification data, and generates a tracking instruction; the three-axis pan-tilt head drives the wave glider according to the tracking instruction to track the ship; the invention can monitor the sea surface ship in real time and improve the detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of autonomous tracking at sea, and in particular to a marine visual tracking system and method. Background Art

[0002] At present, the activities of ships on the ocean are very frequent, and the number of personnel and equipment required to maintain the order of the ocean has also increased dramatically. It has become necessary to develop ship identification technology based on unmanned ships for maritime supervision, detection of suspicious ships and maintenance of maritime traffic safety. Since maritime inspections and surveillance require a lot of manpower and material resources and have high operational risks, it is impossible to monitor the sea surface in real time, which will lead to omissions in the detection of suspicious ships, and thus make the detection accuracy of ships on the sea surface low. Therefore, it is very important to be able to monitor ships on the sea surface in real time and improve the detection accuracy. Summary of the invention

[0003] The purpose of the present invention is to provide a marine visual tracking system and method, which can monitor sea surface ships in real time and improve detection accuracy.

[0004] To achieve the above-mentioned purpose, the present invention provides the following solution: a marine visual tracking system, the system comprising: a three-axis gimbal, a visual control system and a wave glider.

[0005] The three-axis gimbal and the visual control system are both arranged on the wave glider; the three-axis gimbal is connected to the wave glider; the visual control system is connected to the three-axis gimbal.

[0006] The visual control system is used to issue operation instructions according to the position information of the preset task area.

[0007] The three-axis gimbal is used to determine a travel route according to the operation instruction.

[0008] The wave glider is used to travel to a preset mission area according to the travel route.

[0009] The visual control system is also used to: acquire image data of a preset task area in real time.

[0010] The image data is detected and identified to obtain identification data; the identification data includes: the vessel and the location information of the vessel.

[0011] The state data of the three-axis gimbal is calculated according to the identification data, and a tracking instruction is generated; the state data includes: speed and acceleration.

[0012] The three-axis gimbal is also used to drive the wave glider according to the tracking instruction to track the ship.

[0013] Optionally, the visual control system includes: a controller, an image acquisition module, a recognition module and a tracking module.

[0014] The image acquisition module, the recognition module and the tracking module are all connected to the controller; the image acquisition module is connected to the recognition module; the recognition module is connected to the tracking module; and the tracking module is connected to the three-axis gimbal.

[0015] The controller is used to control the image acquisition module to acquire image data of a preset task area in real time.

[0016] The recognition module is used to: receive the image data, and perform region segmentation on the image data to obtain image segmentation data; perform frame selection and feature extraction on the image segmentation data to obtain image feature data; use the SSD algorithm to perform ship detection and recognition on the image feature data to obtain recognition data, and transmit the recognition data to the controller and the tracking module.

[0017] The tracking module is used to calculate the state data of the three-axis gimbal according to the identification data by adopting a feedforward pseudo-differential feedback control method.

[0018] The controller is further configured to generate a tracking instruction according to the identification data and the status data.

[0019] Optionally, the system further includes: a shore-based platform; the shore-based platform is connected to the controller.

[0020] The shore-based platform is used to send control information; the control information includes: location information of a preset mission area and collection instructions.

[0021] The controller is used to receive the control information, issue an operation instruction according to the position information of the preset task area, and control the image acquisition module to acquire image data of the preset task area in real time according to the acquisition instruction.

[0022] Optionally, the visual control system further includes: a storage module; the storage module is connected to the recognition module and the shore-based platform respectively.

[0023] The storage module is used to: store the image corresponding to the recognition data obtained by the recognition module to obtain a stored image; compress the stored image and transmit it to the shore-based platform.

[0024] Optionally, the three-axis gimbal includes: three DC brush motors; each of the DC brush motors is provided with a reducer; and each of the DC brush motors is connected to the visual control system.

[0025] Optionally, the wave glider includes: a floating vessel, a tractor and an armored cable; the tractor and the armored cable are both arranged on the floating vessel; and the tractor is connected to the three-axis gimbal.

[0026] Optionally, the image acquisition module adopts a CCD camera.

[0027] Optionally, the shore-based platform and the visual control system are connected by signal via a Beidou communication satellite or base station.

[0028] A marine visual tracking method adopts the above-mentioned system, and the method includes: issuing an operation instruction according to the position information of a preset task area; controlling a three-axis gimbal to determine a travel route according to the operation instruction; acquiring image data of the preset task area in real time; detecting and identifying the image data to obtain identification data; the identification data includes: a ship and the position information of the ship; calculating the status data of the three-axis gimbal according to the identification data, and generating a tracking instruction; the status data includes: speed and acceleration; controlling the three-axis gimbal to drive a wave glider according to the tracking instruction to track the ship.

[0029] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the present invention provides a marine visual tracking system and method, the system comprising: a three-axis gimbal, a visual control system and a wave glider; the image data of a preset task area is acquired in real time by the visual control system, and the image data is detected and identified to obtain identification data; then the status data of the three-axis gimbal is calculated according to the identification data, and a tracking instruction is generated; the wave glider is driven by the three-axis gimbal according to the tracking instruction to track the ship; the present invention combines the three-axis gimbal with the visual control system to perform real-time monitoring of surface ships, increase the monitoring time to avoid missing ships, and improve detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0031] Figure 1 A structural diagram of a marine visual tracking system provided in an embodiment of the present invention.

[0032] Figure 2 A schematic structural diagram of a three-axis gimbal provided in an embodiment of the present invention.

[0033] Figure 3A flow chart of a marine visual tracking method provided by an embodiment of the present invention.

[0034] Explanation of symbols: three-axis gimbal-1, visual control system-2, wave glider-3. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] The purpose of the present invention is to provide a marine visual tracking system and method, which can monitor sea surface ships in real time and improve detection accuracy.

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] Example 1

[0039] like Figure 1 As shown, an embodiment of the present invention provides a marine visual tracking system, which includes: a three-axis gimbal 1, a visual control system 2 and a wave glider 3.

[0040] The three-axis gimbal 1 and the visual control system 2 are both arranged on the wave glider 3; the three-axis gimbal 1 is connected to the wave glider 3; and the visual control system 2 is connected to the three-axis gimbal 1.

[0041] The wave glider 3 includes: a floating vessel, a tractor and an armored cable; the tractor and the armored cable are both arranged on the floating vessel; and the tractor is connected to the three-axis gimbal 1 .

[0042] The visual control system 2 is used to issue operation instructions according to the position information of the preset task area.

[0043] The three-axis gimbal 1 is used to determine a travel route according to an operation instruction.

[0044] The wave glider 3 is used to travel to a preset mission area according to a travel route.

[0045] The visual control system 2 is also used to obtain image data of a preset task area in real time; the visual control system 2 detects and identifies the image data to obtain identification data; the identification data includes: the ship and the location information of the ship.

[0046] The visual control system 2 calculates the state data of the three-axis gimbal 1 according to the recognition data and generates a tracking instruction; the state data includes: speed and acceleration.

[0047] The three-axis gimbal 1 is also used to drive the wave glider 3 according to the tracking instruction to track the ship.

[0048] Specifically, the visual control system 2 includes: a controller, an image acquisition module, a recognition module and a tracking module.

[0049] The image acquisition module, the recognition module and the tracking module are all connected to the controller; the image acquisition module is connected to the recognition module; the recognition module is connected to the tracking module; and the tracking module is connected to the three-axis gimbal 1. The image acquisition module can use a CCD camera.

[0050] The controller is used to control the image acquisition module to acquire image data of a preset task area in real time.

[0051] The recognition module is used to receive image data and perform region segmentation on the image data to obtain image segmentation data.

[0052] The recognition module is used to perform frame selection and feature extraction on the image segmentation data to obtain image feature data.

[0053] The recognition module is used to detect and recognize ships based on the image feature data using the SSD algorithm, obtain recognition data, and transmit the recognition data to the controller and tracking module.

[0054] The tracking module is used to calculate the state data of the three-axis gimbal 1 according to the identification data by adopting a feedforward pseudo-differential feedback control method.

[0055] The controller is also used to generate tracking instructions based on the identification data and the status data.

[0056] In one embodiment, the system further includes: a shore-based platform; the shore-based platform is connected to the controller; the shore-based platform is used to send control information; the control information includes: location information of a preset task area and collection instructions.

[0057] The controller is used to receive control information, issue operation instructions according to the position information of the preset task area, and control the image acquisition module to acquire image data of the preset task area in real time according to the acquisition instructions.

[0058] Specifically, the visual control system 2 also includes: a storage module; the storage module is connected to the recognition module and the shore-based platform respectively; the storage module is used to store the image corresponding to the recognition data obtained by the recognition module to obtain a stored image.

[0059] The storage module is also used to compress the stored images and transmit them to the shore-based platform.

[0060] In one embodiment, the three-axis gimbal 1 includes: three DC brush motors; each DC brush motor is provided with a reducer. Figure 2 .

[0061] Each brushed DC motor is connected to the visual control system 2 .

[0062] Specifically, the three-axis gimbal 1 is composed of three DC brush motors, each of which is equipped with a reducer to increase the torque of the DC brush motor. By calculating the external force on each DC brush motor and the angle to be adjusted, the angle that the three-axis gimbal 1 needs to be adjusted is obtained after comprehensive analysis.

[0063] In practical applications, the visual control system 2 can also be composed of an 8-megapixel camera and a jetson nano core board as the main control. The camera can be installed at the end of the three-axis gimbal 1 to scan the sea surface and track a ship or vessel after capturing it.

[0064] The Wave Glider 3 is an unmanned surface and underwater hybrid vessel used for ocean research. The Wave Glider 3 is mainly composed of three parts: a floating body, a tractor and an armored cable.

[0065] The shore-based platform is composed of Beidou communication and data receiving software. The shore-based platform can be composed of the Beidou transceiver module on the Wave Glider 3 and the transceiver module of the base station on the shore. The Jetson nano core board can transmit data to the Beidou transceiver module on the Wave Glider 3 through a network cable or other means to send data. After receiving the command, the transceiver module of the base station on the shore will display the image information in the corresponding application.

[0066] In one embodiment, the shore-based platform and the visual control system 2 are connected by signal via a Beidou communication satellite or base station.

[0067] In practical applications, the wave glider 3 is equipped with a three-axis visual tracking system and provides energy for the three-axis visual tracking system. The three-axis visual tracking system is a combination of a three-axis gimbal 1 and a visual control system 2.

[0068] The wave glider 3 is also used to travel according to the control information sent by the shore-based platform, that is, first, the visual control system 2 sends an operation instruction according to the position information of the preset task area, and then the three-axis gimbal 1 determines the travel route according to the operation instruction. Finally, the wave glider 3 travels to the preset task area according to the travel route. When the wave glider 3 travels to the preset task area, the shore-based platform sends a collection instruction, and the preset task area is photographed and monitored in real time and information is collected through the three-axis gimbal 1 and the visual control system 2, that is, the image data of the preset task area is obtained in real time.

[0069] The recognition module in the visual control system 2 performs regional segmentation on the image data to obtain the image segmentation data, and then performs frame selection and feature extraction on the image segmentation data to obtain image feature data; the recognition module uses the SSD algorithm to detect and identify ships on the image feature data to obtain recognition data.

[0070] When the recognition is successful, that is, the recognition data is obtained, the image or picture corresponding to the recognition data will be stored in the storage module, and the original image will be compressed and sent back to the base station on the shore, and finally transmitted to the shore-based platform.

[0071] Regarding the SSD algorithm, before applying it, first use the self-built data set on the workstation to train the weight file for ship recognition, then embed the SSD recognition algorithm into the Jetson Nano core board, set it to automatically start the recognition program when it is turned on, and then you can achieve automatic recognition at sea. The image preprocessing, image segmentation, and image recognition box selection in the SSD algorithm all use the SSD standard model method.

[0072] When the recognition module recognizes the ship, that is, captures the ship, it can transmit the position information of the ship to the tracking module; the tracking module adopts a feedforward pseudo-differential feedback control method to calculate the state data of the three-axis gimbal 1 according to the recognition data.

[0073] The feedforward pseudo-differential feedback control method takes into account the environmental resistance of the sea surface, wave fluctuations, and the resistance and friction experienced by each DC brush motor individually.

[0074] Specifically, the three-axis gimbal 1 is a precision machining operation platform composed of DC brushed motors with mutually perpendicular axes, and the model it conforms to is as follows:

[0075] .

[0076] in, is the position of the motor rotor of the DC brush motor; is the motor force constant of the brushed DC motor; is the motor force of the brushed DC motor; is the mass of the three-axis gimbal; is the friction coefficient; is the total disturbance force of the motor rotor of the brushed DC motor; include Friction, External disturbance and Sea wind resistance. is the current of the brushed DC motor; For time.

[0077] The position and speed of the motor of the brushed DC motor are selected as state variables. At this time, the position of the motor of the brushed DC motor as a variable is ; Movement speed is .

[0078] The calculated state equation is shown below.

[0079] .

[0080] .

[0081] in, is the instantaneous speed; is the instantaneous acceleration; is the friction coefficient; is the mass of the three-axis gimbal; is the total disturbance force of the motor rotor of the brushed DC motor; is the feedback gain.

[0082] is the input value; .

[0083] Then, the standard model of the three-axis gimbal 1 can be obtained through calculation. The combination of the three DC brush motors adjusting the angles is the motion state of the three-axis gimbal 1. The standard model of the three-axis gimbal 1 is to calculate the speed and acceleration of each DC brush motor separately.

[0084] .

[0085] in, is the acceleration of the first brushed DC motor; is the acceleration of the second brushed DC motor; is the acceleration of the third brushed DC motor; is the friction coefficient; is the mass of the three-axis gimbal; is the speed of the first brushed DC motor; is the speed of the second brushed DC motor; is the speed of the third brushed DC motor; is the total disturbance force of the motor rotor of the first brushed DC motor; is the total disturbance force of the motor rotor of the second brushed DC motor; is the total disturbance force of the motor rotor of the third DC brush motor.

[0086] is the input value of the first brushed DC motor; is the input value of the second DC brush motor; This is the input value for the third brushed DC motor.

[0087] The controller generates a tracking instruction according to the identification data and the state data obtained by the above calculation, so as to keep the ship in the center of the image at all times and achieve the purpose of tracking.

[0088] Example 2

[0089] like Figure 3 As shown, an embodiment of the present invention provides a marine visual tracking method, which adopts the system in embodiment 1 and includes:

[0090] Step 100: issuing an operation instruction according to the location information of the preset task area.

[0091] Step 200: Control the three-axis gimbal to determine the travel route according to the operation instruction.

[0092] Step 300: Acquire image data of a preset task area in real time.

[0093] Step 400: Detect and identify the image data to obtain identification data; the identification data includes: the ship and the location information of the ship.

[0094] Step 500: Calculate the status data of the three-axis gimbal according to the identification data and generate a tracking instruction; the status data includes: speed and acceleration.

[0095] Step 600: Control the three-axis gimbal to drive the wave glider according to the tracking instruction to track the ship.

[0096] The wave glider is not restricted by energy, has low cost, long endurance, large range, strong risk resistance, and can realize sea surface equipment inspection, sea surface information collection, sea surface pollution source distribution detection in low altitude range, and has the ability to approach the reconnaissance field and conduct autonomous operations at sea. When the wave glider is equipped with a visual control system to form a formation network, it can stably monitor a large area of ​​sea in real time. This not only improves the reliability of maintaining marine order, but also reduces the use of marine equipment and personnel.

[0097] At the same time, the combination of the three-axis gimbal and the visual control system can make maritime supervision automated, integrated and intelligent. Suspicious ships can be tracked purposefully to ensure the safety of maritime traffic.

[0098] The present invention adopts SSD algorithm for recognition, and the recognition rate can reach 90%. The image acquisition module uses CCD camera, and its recognition distance can reach 10km, which can fully meet the needs of ship and vessel detection.

[0099] The present invention overcomes the difficulties of difficult maritime traffic supervision and short supervision time based on the advantages of wave gliders in global positioning, long-distance communication, autonomous navigation and position keeping, and can realize long-term and large-scale navigation, thereby improving the maritime detection range of unmanned ships.

[0100] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0101] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A marine visual tracking system, It is characterized in that The system comprises: a three-axis gimbal, a visual control system and a wave glider; The three-axis gimbal and the visual control system are both arranged on the wave glider; the three-axis gimbal is connected to the wave glider; the visual control system is connected to the three-axis gimbal; The visual control system is used to issue operation instructions according to the position information of the preset task area; The three-axis gimbal is used to determine the travel route according to the operation instruction; The wave glider is used to travel to a preset mission area according to the travel route; The visual control system is also used for: Acquire image data of preset task area in real time; Detecting and identifying the image data to obtain identification data; the identification data includes: the vessel and the location information of the vessel; Calculating the state data of the three-axis gimbal according to the identification data and generating a tracking instruction; the state data includes: speed and acceleration; The three-axis gimbal is also used to drive the wave glider according to the tracking instruction to track the ship; The visual control system includes: a controller, an image acquisition module, a recognition module and a tracking module; The image acquisition module, the recognition module and the tracking module are all connected to the controller; the image acquisition module is connected to the recognition module; the recognition module is connected to the tracking module; the tracking module is connected to the three-axis gimbal; The controller is used to control the image acquisition module to acquire image data of a preset task area in real time; The identification module is used to: receiving the image data, and performing region segmentation on the image data to obtain image segmentation data; Performing frame selection and feature extraction on the image segmentation data to obtain image feature data; Using an SSD algorithm to perform ship detection and identification on the image feature data to obtain identification data, and transmitting the identification data to the controller and the tracking module; The tracking module is used to calculate the state data of the three-axis gimbal according to the identification data by using a feedforward pseudo-differential feedback control method; The feedforward pseudo-differential feedback control method takes into account the environmental resistance of the sea surface, wave fluctuations, and the resistance and friction of each DC brush motor. The position and movement speed of the motor of the brushed DC motor are selected as state variables. At this time, the position of the motor of the brushed DC motor as a variable is y(t); the movement speed is v(t); The calculated state equation is shown below; in, is the instantaneous speed; is the instantaneous acceleration; B is the friction coefficient; M is the mass of the three-axis gimbal; F sum is the total disturbance force of the motor rotor of the brushless DC motor; k f is the feedback gain; u is the input value; where u=i q ; Furthermore, the standard model of the three-axis gimbal can be obtained through calculation; the combination of the three DC brush motors adjusting the angles is the motion state of the three-axis gimbal; the standard model of the three-axis gimbal is to calculate the speed and acceleration of each DC brush motor separately; in, is the acceleration of the first brushed DC motor; is the acceleration of the second brushed DC motor; is the acceleration of the third DC brushed motor; B is the friction coefficient; M is the mass of the three-axis gimbal; is the speed of the first brushed DC motor; is the speed of the second brushed DC motor; is the speed of the third brushed DC motor; F sum1 is the total disturbance force of the motor rotor of the first brushed DC motor; F sum2 is the total disturbance force of the motor rotor of the second brushed DC motor; F sum3 is the total disturbance force of the motor rotor of the third brushed DC motor; u 1 is the input value of the first brushed DC motor; u 2 is the input value of the second DC brush motor; u 3 is the input value of the third DC brush motor; The controller is also used to generate a tracking instruction based on the identification data and the status data; The three-axis gimbal is a precision-machined operating platform composed of DC brushed motors with axes perpendicular to each other; The following models are suitable: Where, f is the position of the motor rotor of the brushed DC motor; T f F is the motor force constant of the brushed DC motor; e is the motor force of the brushless DC motor; M is the mass of the three-axis gimbal; B is the friction coefficient; F sum is the total disturbance force of the motor rotor of the brushless DC motor; F sum Including F sf Friction, F of External disturbance force and F wf Sea wind resistance; i q is the current of the brushed DC motor; t is the time; The three brushed DC motors adjust the angles and the combination creates the motion state of the three-axis gimbal.

2. The marine visual tracking system according to claim 1, It is characterized in that The system further comprises: a shore-based platform; The shore-based platform is connected to the controller; The shore-based platform is used to send control information; the control information includes: location information of a preset mission area and collection instructions; The controller is used to receive the control information, issue an operation instruction according to the position information of the preset task area, and control the image acquisition module to acquire image data of the preset task area in real time according to the acquisition instruction.

3. The marine visual tracking system according to claim 2, It is characterized in that The visual control system further includes: a storage module; The storage module is connected to the identification module and the shore-based platform respectively; The storage module is used for: The image corresponding to the recognition data obtained by the recognition module is stored to obtain a stored image; The stored image is compressed and transmitted to the shore-based platform.

4. The marine visual tracking system according to claim 1, It is characterized in that The three-axis gimbal comprises: three DC brush motors; each of the DC brush motors is provided with a reducer; Each of the DC brushed motors is connected to the visual control system.

5. The marine visual tracking system according to claim 1, It is characterized in that The wave glider comprises: a floating vessel, a tractor and an armored cable; The traction machine and the armored cable are both arranged on the floating vessel; the traction machine is connected to the three-axis pan-tilt head.

6. The marine visual tracking system according to claim 1, It is characterized in that The image acquisition module adopts a CCD camera.

7. The marine visual tracking system according to claim 3, It is characterized in that The shore-based platform and the visual control system are connected by signals via Beidou communication satellites or base stations.

8. A method for visual tracking at sea, It is characterized in that The method adopts the system according to any one of claims 1 to 7, and the method comprises: Issue operation instructions according to the location information of the preset task area; Controlling the three-axis gimbal to determine a travel route according to the operation instructions; Acquire image data of preset task area in real time; Detecting and identifying the image data to obtain identification data; the identification data includes: the vessel and the location information of the vessel; Calculate the state data of the three-axis gimbal according to the identification data and generate a tracking instruction; The state data includes: speed and acceleration; The three-axis gimbal is controlled to drive the wave glider according to the tracking instruction to track the ship.

Images