Illegal fishing detection method and system based on unmanned aerial vehicle

Abstract

The invention discloses an illegal fishing detection method and system based on an unmanned aerial vehicle, and relates to the technical field of illegal fishing detection, and the method comprises the following steps: enabling the unmanned aerial vehicle to be equipped with a high-definition camera, and controlling a remote controller to enable the unmanned aerial vehicle to fly to a preset height to cover a target water surface; the unmanned aerial vehicle obtains water surface video data through a preset route or real-time control; extracting real-time ship data from the unmanned aerial vehicle video, wherein the real-time ship data comprises a ship ID, a position coordinate, a speed and a direction; after ship data extraction, a data acquisition module is used for preprocessing a video stream, including image denoising, stabilization and enhancement; calculating the relative position and distance between each ship and the personnel on the ship by using a position calculation module; determining the relative displacement of the person on the ship, and calculating whether the person is in a static state or a continuous moving state on the ship. According to the invention, accurate analysis is carried out by effectively utilizing position data of the ship; the fishing behavior can be detected in real time in the water surface high-altitude environment, and the efficiency of attacking illegal fishing is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of illegal fishing detection, and in particular to an illegal fishing detection method and system based on a drone. Background Art

[0002] The monitoring system is one of the most widely used systems in the security system. The most suitable construction site monitoring system on the market is the handheld video communication device. Video monitoring is now the mainstream. From the earliest analog monitoring to the popular digital monitoring in the past few years to the emerging network video monitoring, earth-shaking changes have taken place.

[0003] Existing monitoring systems mainly rely on fixed cameras and manual monitoring when identifying illegal fishing, which is difficult to adapt to large water surface environments. Drones have flexible maneuverability and a wide field of view, and can provide more comprehensive water surface monitoring data. However, when identifying illegal fishing and other situations that mainly rely on manual judgment or simple rule matching, automatic identification will be affected by the drone shooting environment, and it is impossible to effectively use the ship's location data for accurate analysis through traditional methods, and the crackdown on illegal fishing is poor, so there is room for improvement. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and propose an illegal fishing detection method based on drones. Its advantages are that it can effectively use the position data of the ship for accurate analysis; it can detect fishing behavior in real time in a high-altitude environment on the water surface, and improve the efficiency of combating illegal fishing.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The illegal fishing detection method based on drone includes the following steps:

[0007] Step 1: Equip the drone with a high-definition camera and control the remote control to fly it to a predetermined height to cover the target water surface. The drone obtains water surface video data through a preset route or real-time control.

[0008] Step 2: Extract real-time ship data from the drone video, including ship ID, location coordinates, speed and direction; After the ship data is extracted, the video stream is preprocessed using the data acquisition module, including image denoising, stabilization and enhancement;

[0009] Step 3: Use the position calculation module to calculate the relative position and distance between each ship and the people on it; determine the relative displacement of the people on the ship, and calculate whether the people on the ship are stationary or moving; by analyzing the posture trajectory of the human body, determine whether the human action is related to fishing;

[0010] Step 4: Use the behavior recognition module to identify the human posture. If the position of the human body relative to the boat does not change much in a short period of time, and the posture change is consistent with fishing characteristics, it is determined to be illegal fishing behavior. Combined with historical data and fishing types, the rationality of illegal fishing behavior is further verified;

[0011] Step 5: Use the communication module to transmit the identification results to the surface management system. The identification results include the ID, location, time and related video clips of the illegal fishing vessel, and the results are stored in the database.

[0012] The present invention is further configured such that the image is denoised using median filtering, a neighborhood within a certain range around a certain pixel point is counted, and all pixel values ​​in the neighborhood are counted, a median value in the pixel value set is taken, the value of the pixel point is replaced by the median value, and image enhancement is performed using histogram equalization.

[0013] The present invention is further configured such that the pixel values ​​between the pixels are independent of each other, so each pixel is processed separately without affecting each other; the separate pixel processing requires parameter initialization as follows:

[0014]

[0015] Among them, p(X t ) represents the probability of the observed pixel value X occurring at time t, η(x,μ i,t ,∑ i,t ) represents the Gaussian distribution of the pixel at time t, μ i,t is the mean, ∑ i,t As its variance, each pixel in the image can be modeled by Gaussian distribution superposition according to a certain weight, then ω i,t represents the weight of the i-th Gaussian distribution, and the sum of the weights is 1, that is, in:

[0016]

[0017] Where n is the dimension of pixel X.

[0018] The present invention is further configured that the ship ID is extracted from the photographed picture using picture text recognition technology, comprising the following steps:

[0019] Step 1: Character segmentation: Separate the text in the image into individual characters or words. This process analyzes the spacing and shape between characters, and the system effectively separates the characters.

[0020] Step 2: Feature extraction: The system extracts the features of each character, including strokes, shapes, and outlines; these features are used to train the recognition model to enable it to distinguish between different characters;

[0021] Step 3: Character Recognition: Using machine learning algorithms, the system compares the extracted features with known characters to identify the text in the image.

[0022] The present invention is further configured such that the ship position coordinates are (x 1 ,y 1 ,z 1 ), the position coordinates of the personnel are marked as (x 2 ,y 2 ,z 2 ), R represents the position of the personnel relative to the ship; the calculation formula is: R = (x 2 -x 1 ,y 2 -y 1 ,z 2 -z 1 ).

[0023] The present invention is further configured such that the straight-line distance between the ship and the personnel is d, and the calculation formula of the straight-line distance is:

[0024] The present invention is further configured such that the calculation formula for the ship speed is: v represents the speed of the ship, s represents the distance the ship moves per unit time, and t represents time.

[0025] The present invention is further configured as follows: the human posture recognition includes environmental feature extraction, bone sequence extraction and bone feature extraction. The bone feature extraction utilizes the OpenPose model, extracts the features of the input image by a neural network, and enters different branches to predict the positions of key points of the human skeleton and the connection information between each key point of the skeleton; after extracting the human skeleton features, the human skeleton data is input into the convolutional network in the order of the time axis to retain the time characteristics; when utilizing the selective input of the LSTM network input gate, redundant feature data is filtered out, and through the combination of CNN and LSTM, the complete time and space data of the bone feature sequence is obtained, the bone features are combined with the environmental features for feature-level fusion and decision-level fusion, and finally the human posture recognition result is output.

[0026] The present invention is further configured that the human body posture recognition neural network needs to be trained to find the optimal parameters; Softmax is a classifier commonly used in neural network classification training, and the calculation formula of the Softmax function is:

[0027] Among them, z i represents the score value of the i-th category, K represents the total number of categories; the Softmax function calculates the original score z for each category iAfter exponential processing, the results are normalized to obtain the probability of each category.

[0028] The present invention is further configured that the human posture recognition neural network uses the loss function as a benchmark during the learning and training process, and seeks the parameters that minimize the loss function as the optimal parameters; a cross entropy loss function is designed, and the cross entropy uses the spacing between two probability distributions to represent the deviation between the detection value and the true value, and the expression is as follows: Among them, y true is the detection output value, y pred represents the true value;

[0029] The smaller the cross entropy, the closer the detection value is to the true value. The calculation of the cross entropy loss function can be simplified to summing the probability values ​​of each category and taking its negative logarithm, which represents the negative logarithm of the predicted probability of the category corresponding to the true label. By minimizing the cross entropy loss function, the prediction result of the model is made as close to the true label as possible, thereby achieving model training and optimization.

[0030] According to the present invention, an illegal fishing detection system based on a drone is provided, comprising:

[0031] Step 1: Equip the drone with a high-definition camera and control the remote control to fly it to a predetermined height to cover the target water surface. The drone obtains water surface video data through a preset route or real-time control.

[0032] Module 2: Extract real-time ship data from drone videos, including ship ID, location coordinates, speed and direction; After the ship data is extracted, the video stream is preprocessed using the data acquisition module, including image denoising, stabilization and enhancement;

[0033] Module 3: Use the position calculation module to calculate the relative position and distance between each ship and the people on it; determine the relative displacement of the people on the ship, and calculate whether the people on the ship are stationary or moving; by analyzing the posture trajectory of the human body, determine whether the human action is related to fishing;

[0034] Module 4: Use the behavior recognition module to identify human posture. If the position of the human body relative to the boat does not change much in a short period of time, and the posture change is consistent with fishing characteristics, it is judged as illegal fishing. Combined with historical data and fishing types, the rationality of illegal fishing behavior is further verified;

[0035] Module 5: Use the communication module to transmit the identification results to the surface management system. The identification results include the ID, location, time and related video clips of the illegal fishing vessel, and the results are stored in the database.

[0036] The beneficial effects of the present invention are as follows: the illegal fishing detection method based on drones obtains water surface video data through drones, analyzes identified and tracked ship data, calculates the relative surface positions of people and ships, and identifies qualified fishing personnel and behaviors; effectively utilizes the position data of ships for precise analysis; and can detect fishing behaviors in real time in a high-altitude environment over the water, thereby improving the efficiency of combating illegal fishing. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of the detection process of the illegal fishing detection method based on drone proposed in the present invention;

[0038] Figure 2 This is a schematic diagram of the human posture recognition process of the illegal fishing detection method based on drone proposed in the present invention. DETAILED DESCRIPTION

[0039] The technical solution of this patent is further described in detail below in conjunction with specific implementation methods.

[0040] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0041] Reference Figure 1 , a method for detecting illegal fishing based on drones, comprising the following steps:

[0042] Step 1: Equip the drone with a high-definition camera and control the remote control to fly it to a predetermined height to cover the target water surface. The drone obtains water surface video data through a preset route or real-time control.

[0043] Step 2: Extract real-time ship data from the drone video, including ship ID, location coordinates, speed and direction; After the ship data is extracted, the video stream is preprocessed using the data acquisition module, including image denoising, stabilization and enhancement;

[0044] Step 3: Use the position calculation module to calculate the relative position and distance between each ship and the people on it; determine the relative displacement of the people on the ship, and calculate whether the people on the ship are stationary or moving; by analyzing the posture trajectory of the human body, determine whether the human action is related to fishing;

[0045] Step 4: Use the behavior recognition module to identify the human posture. If the position of the human body relative to the boat does not change much in a short period of time, and the posture change is consistent with fishing characteristics, it is determined to be illegal fishing behavior. Combined with historical data and fishing types, the rationality of illegal fishing behavior is further verified;

[0046] Step 5: Use the communication module to transmit the identification results to the surface management system. The identification results include the ID, location, time and related video clips of the illegal fishing vessel, and the results are stored in the database.

[0047] In this embodiment, median filtering is used for image denoising, which can not only eliminate noise but also protect the detail information in the image to a certain extent; the neighborhood within a certain range around a certain pixel point is counted, and by counting all pixel values ​​in the neighborhood, the median value in the pixel value set is taken, and the value of the pixel point is replaced by the median value, so that isolated noise can be effectively eliminated; histogram equalization is used for image enhancement;

[0048] Identification of background model: Assume that the pixel values ​​between pixels are unrelated, so each pixel can be processed separately without affecting each other; parameter initialization is as follows:

[0049]

[0050] Among them, p(X t ) represents the probability of the observed pixel value X occurring at time t, η(x,μ i,t ,∑ i,t ) represents the Gaussian distribution of the pixel at time t, μ i,t is the mean, ∑ i,t As its variance, each pixel in the image can be modeled by Gaussian distribution superposition according to a certain weight, then ω i,t represents the weight of the i-th Gaussian distribution, and the sum of the weights is 1, that is, in:

[0051]

[0052] Where n is the dimension of pixel X.

[0053] Furthermore, the ship ID is extracted from the captured image using image text recognition technology, including the following steps:

[0054] Step 1: Character segmentation: Separate the text in the image into individual characters or words. This process analyzes the spacing and shape between characters, and the system effectively separates the characters.

[0055] Step 2: Feature extraction: The system extracts the features of each character, including strokes, shapes, and outlines; these features are used to train the recognition model to enable it to distinguish between different characters;

[0056] Step 3: Character Recognition: Using machine learning algorithms, the system compares the extracted features with known characters to identify the text in the image.

[0057] The ship's position coordinates are marked as (x 1 ,y 1 ,z 1 ), the position coordinates of the personnel are marked as (x 2 ,y 2 ,z 2 ), R represents the position of the personnel relative to the ship; the calculation formula is: R = (x 2 -x 1 ,y 2 -y 1 ,z 2 -z 1 ) The straight-line distance between the ship and the personnel is d, and the calculation formula of the straight-line distance is: The calculation formula for ship speed is: v represents the speed of the ship, s represents the distance the ship moves per unit time, and t represents time.

[0058] Reference Figure 2 ,Human posture recognition includes environmental feature extraction, bone sequence extraction and bone feature extraction. The bone feature extraction uses the OpenPose model. The neural network extracts the features of the input image and enters different branches to predict the position of the key points of the human skeleton and the connection information between each key point of the skeleton. This part is consistent with the network structure of the OpenPose model. After the two prediction branches are processed in parallel, a data file in JASON format is obtained, which stores the information of the key points. Convert the JASON file format to xt text format to obtain a text file of the skeleton key points.

[0059] After extracting the human skeleton features, the human skeleton data is input into the convolutional network in the order of the time axis to retain the time characteristics; when using the selective input of the LSTM network input gate, the redundant feature data is filtered out, and the complete time and space data of the skeleton feature sequence is obtained through the combination of CNN and LSTM. The skeleton features are combined with environmental features for feature-level fusion and decision-level fusion, and finally the human posture recognition results are output.

[0060] The human posture recognition neural network needs to be trained to find the best parameters; Softmax is a classifier commonly used in neural network classification training. The basic idea is to convert the predicted output values ​​obtained at all forward propagation moments into probability distributions, calculate the expected probability of each possible result, and finally calculate the maximum mean category, which is the classification recognition result; the calculation formula of the Softmax function is:

[0061] Among them, z i represents the score value of the i-th category, K represents the total number of categories; the Softmax function calculates the original score z for each categoryi After exponential processing, the results are normalized to obtain the probability of each category.

[0062] The human posture recognition neural network uses the loss function as a benchmark during the learning and training process, and seeks the parameters that minimize the loss function as the optimal parameters; the cross entropy loss function is designed. The cross entropy uses the distance between two probability distributions to represent the deviation between the detection value and the true value. The expression is as follows: Among them, y true is the detection output value, y pred represents the true value;

[0063] The smaller the cross entropy, the closer the detection value is to the true value. The calculation of the cross entropy loss function can be simplified to summing the probability values ​​of each category and taking its negative logarithm, which represents the negative logarithm of the predicted probability of the category corresponding to the true label. By minimizing the cross entropy loss function, the prediction result of the model is made as close to the true label as possible, thereby achieving model training and optimization.

[0064] The present invention also provides an illegal fishing detection system based on a drone, which can be implemented by executing the process steps of the illegal fishing detection method based on a drone, that is, those skilled in the art can understand the illegal fishing detection method based on a drone as a preferred implementation of the illegal fishing detection system based on a drone. The system includes:

[0065] Step 1: Equip the drone with a high-definition camera and control the remote control to fly it to a predetermined height to cover the target water surface. The drone obtains water surface video data through a preset route or real-time control.

[0066] Module 2: Extract real-time ship data from drone videos, including ship ID, location coordinates, speed and direction; After the ship data is extracted, the video stream is preprocessed using the data acquisition module, including image denoising, stabilization and enhancement;

[0067] Module 3: Use the position calculation module to calculate the relative position and distance between each ship and the people on it; determine the relative displacement of the people on the ship, and calculate whether the people on the ship are stationary or moving; by analyzing the posture trajectory of the human body, determine whether the human action is related to fishing;

[0068] Module 4: Use the behavior recognition module to identify human posture. If the position of the human body relative to the boat does not change much in a short period of time, and the posture change is consistent with fishing characteristics, it is judged as illegal fishing. Combined with historical data and fishing types, the rationality of illegal fishing behavior is further verified;

[0069] Module 5: Use the communication module to transmit the identification results to the surface management system. The identification results include the ID, location, time and related video clips of the illegal fishing vessel, and the results are stored in the database.

[0070] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0071] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for detecting illegal fishing based on drones, characterized in that: The following steps are involved: Step 1: Equip the drone with a high-definition camera and use the remote control to fly it to a predetermined height to cover the target water surface; The drone acquires water surface video data through a preset route or real-time control; Step 2: Extract real-time ship data from the drone video, including ship ID, location coordinates, speed and direction; After the ship data is extracted, the video stream is preprocessed using the data acquisition module, including image denoising, stabilization and enhancement; Step 3: Use the position calculation module to calculate the relative position and distance between each ship and the people on it; determine the relative displacement of the people on the ship, and calculate whether the people on the ship are stationary or moving; by analyzing the posture trajectory of the human body, determine whether the human action is related to fishing; Step 4: Use the behavior recognition module to identify the human posture. If the position of the human body relative to the boat does not change much in a short period of time, and the posture change conforms to the characteristics of fishing, it is determined to be illegal fishing. Combined with historical data and fishing types, the rationality of illegal fishing is further verified. Step 5: Use the communication module to transmit the identification results to the surface management system. The identification results include the ID, location, time and related video clips of the illegal fishing vessel, and the results are stored in the database.

2. The illegal fishing detection method based on drone according to claim 1 is characterized in that: The image is denoised by using median filtering, counting the neighborhood within a certain range around a certain pixel point, and by counting all pixel values ​​in the neighborhood, taking the median of the pixel value set, using this median to replace the value of the pixel point, and using histogram equalization to perform image enhancement.

3. The illegal fishing detection method based on drone according to claim 2 is characterized in that: The pixel values ​​between the pixels are independent of each other, so each pixel is processed separately without affecting each other; the individual pixel processing requires parameter initialization: as follows: Among them, p(X t ) represents the probability of the observed pixel value X occurring at time t, η(x,μ i,t ,∑ i,t ) represents the Gaussian distribution of the pixel at time t, μ i,t is the mean, ∑ i,t As its variance, each pixel in the image can be modeled by Gaussian distribution superposition according to a certain weight, then ω i,t represents the weight of the i-th Gaussian distribution, and the sum of the weights is 1, that is, in: Where n is the dimension of pixel X.

4. The illegal fishing detection method based on drone according to claim 1 is characterized in that: The ship ID is extracted from the captured image using image text recognition technology, including the following steps: Step 1: Character segmentation: Separate the text in the image into individual characters or words. This process analyzes the spacing and shape between characters, and the system effectively separates the characters. Step 2: Feature extraction: The system extracts the features of each character, including strokes, shapes, and outlines; these features are used to train the recognition model to enable it to distinguish between different characters; Step 3: Character Recognition: Using machine learning algorithms, the system compares the extracted features with known characters to identify the text in the image.

5. The illegal fishing detection method based on drone according to claim 1 is characterized in that: The ship position coordinates are marked as (x1, y1, z1), the personnel position coordinates are marked as (x2, y2, z2), and R represents the position of the personnel relative to the ship; the calculation formula is: R = (x2-x1, y2-y1, z2-z1); The straight-line distance between the ship and the personnel is d, and the calculation formula of the straight-line distance is:

6. The illegal fishing detection method based on drone according to claim 1 is characterized in that: The calculation formula of the ship speed is: v represents the speed of the ship, s represents the distance the ship moves per unit time, and t represents time.

7. The illegal fishing detection method based on drone according to claim 1 is characterized in that: The human posture recognition includes environmental feature extraction, bone sequence extraction and bone feature extraction. The bone feature extraction uses the OpenPose model, extracts the features of the input image by a neural network, and enters different branches to predict the positions of key points of the human skeleton and the connection information between each bone key point. After extracting the human skeleton features, the human skeleton data is input into the convolutional network in the order of the time axis to retain the time characteristics. When the selective input of the LSTM network input gate is used, redundant feature data is filtered out, and the complete time and space data of the bone feature sequence is obtained through the combination of CNN and LSTM. The bone features are combined with environmental features for feature-level fusion and decision-level fusion, and finally the human posture recognition result is output.

8. The method for detecting illegal fishing based on drone according to claim 7, characterized in that: The human posture recognition neural network needs to be trained to find the best parameters; Softmax is a classifier commonly used in neural network classification training, and the calculation formula of the Softmax function is: Among them, z i represents the score value of the i-th category, K represents the total number of categories; the Softmax function calculates the original score z for each category i After exponential processing, the results are normalized to obtain the probability of each category.

9. The method for detecting illegal fishing based on a drone according to claim 8, characterized in that: The human posture recognition neural network uses the loss function as a benchmark during the learning and training process, and seeks the parameters that minimize the loss function as the optimal parameters; a cross entropy loss function is designed, and the cross entropy uses the spacing between two probability distributions to represent the deviation between the detection value and the true value. The expression is as follows: Among them, y true is the detection output value, y pred represents the true value; The smaller the cross entropy, the closer the detection value is to the true value. The calculation of the cross entropy loss function can be simplified to summing the probability values ​​of each category and taking its negative logarithm, which represents the negative logarithm of the predicted probability of the category corresponding to the true label. By minimizing the cross entropy loss function, the prediction result of the model is made as close to the true label as possible, thereby achieving model training and optimization.

10. An illegal fishing detection system based on drones, characterized in that: include: Step 1: Equip the drone with a high-definition camera and use the remote control to fly it to a predetermined height to cover the target water surface; The drone acquires water surface video data through a preset route or real-time control; Module 2: Extract real-time ship data from drone videos, including ship ID, location coordinates, speed and direction; After the ship data is extracted, the video stream is preprocessed using the data acquisition module, including image denoising, stabilization and enhancement; Module 3: Use the position calculation module to calculate the relative position and distance between each ship and the people on it; determine the relative displacement of the people on the ship, and calculate whether the people on the ship are stationary or moving; by analyzing the posture trajectory of the human body, determine whether the human action is related to fishing; Module 4: Use the behavior recognition module to identify human posture. If the position of the human body relative to the boat does not change much in a short period of time, and the posture change is consistent with fishing characteristics, it is judged as illegal fishing. Combined with historical data and fishing types, the rationality of illegal fishing behavior is further verified; Module 5: Use the communication module to transmit the identification results to the surface management system. The identification results include the ID, location, time and related video clips of the illegal fishing vessel, and the results are stored in the database.

Images