Light-attracted fish detection system and method

Abstract

The application discloses a fish detection system and method based on light attraction. The fish detection system is based on an underwater unmanned submarine, and a fish gathering lamp system, an imaging system and a navigation positioning system are arranged on the underwater unmanned submarine. The fish detection method comprises the following steps: the fish gathering lamp system attracts fish to the vicinity, and the imaging system acquires a video image of the periphery; an image recognition technology is used to recognize fish bodies in the video image, a target fish species in the image is found out, and the fish detection system is controlled to track the target fish species. The fish detection system and method can quickly and accurately locate a target fish species in a relatively wide water area, are favorable for accurate positioning and fishing of a specific fish species, and thus greatly improve operation efficiency, effectively reduce time cost and labor cost, and improve economic benefits.

Description

Technical Field

[0001] This invention relates to the field of marine engineering equipment, and in particular to a fish detection system and method based on light-attracting. Background Technology

[0002] With the development of science and technology, human beings' ability to fish in distant waters is constantly improving. However, current distant water fishing operations still face some problems that need to be solved.

[0003] Currently, accurately locating target fish schools in vast waters remains a major challenge for the deep-sea fishing industry. At present, the main method relies on experienced crew members to determine the location of fish schools based on their experience. This experience-based approach has low accuracy and makes it difficult to pinpoint the target fish schools, resulting in low fishing efficiency, high time and labor costs, and consequently reduced economic benefits. Summary of the Invention

[0004] The purpose of this invention is to provide a fish detection system and method based on light attraction, which can quickly and accurately locate target fish species and schools in a relatively wide body of water.

[0005] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0006] A fish detection system based on light attraction, wherein the fish detection system is based on an underwater unmanned submersible, and the underwater unmanned submersible is equipped with a fish-attracting light system, an imaging system and a navigation and positioning system;

[0007] The fish-attracting light system is used to lure fish together.

[0008] The imaging system is used to acquire video images of the surrounding area;

[0009] The navigation and positioning system is used to obtain positioning information.

[0010] A method for detecting fish, comprising:

[0011] The fish detection system described above is used;

[0012] Deploy the fish detection system into the water and turn on the fish-attracting light system and imaging system of the fish detection system.

[0013] The fish-attracting light system lures fish to the vicinity, and the imaging system acquires video images of the surrounding area;

[0014] Image recognition technology is used to identify fish in video images and pinpoint the target fish species in the images;

[0015] The identified target fish species are used as the tracking target, and the fish detection system is controlled to track the target fish species.

[0016] Furthermore, after turning on the fish-attracting light system, the light emitted by the system can be adjusted according to the preferences of the target fish species.

[0017] Furthermore, during the process of acquiring surrounding video images by the imaging system, the area where the target fish species is located is designated as the region of interest, and target tracking is performed only on key information within the region of interest.

[0018] Furthermore, the process of the imaging system acquiring surrounding video images also includes:

[0019] Camera calibration was performed using a camera calibration method based on planar checkerboard markers.

[0020]

[0021] Where A is the camera's intrinsic parameter matrix, s is the scale factor, R and t are the extrinsic parameter matrices describing the relationship between the camera and the global coordinate system, i.e., the camera's extrinsic parameter matrices, and r i Let R represent the i-th column vector, where u and v are pixel coordinates;

[0022] Binocular correction is used to correct two non-coplanar aligned images to achieve coplanar alignment by eliminating distortion.

[0023] Binocular matching performs stereo matching on images to create a stereo model of the images;

[0024] Depth is calculated by determining the depth information of pixels based on the matching results. The stereo vision equation is as follows:

[0025]

[0026]

[0027]

[0028] Where f is the camera focal length, x L x R Let d be the horizontal position of the target in the imaging plane of the left and right cameras, b be the parallax, X be the horizontal distance of the target, Y be the height of the target, and Z be the vertical distance of the target.

[0029] Furthermore, the image recognition technology used to identify fish in the video image and pinpoint the target fish species in the image is as follows:

[0030] A fish species identification module is used to identify fish in video images in order to identify the target fish species in the images;

[0031] The fish species identification module is an image recognition module built based on a deep learning model.

[0032] Furthermore, the specific process of identifying fish bodies in video images includes:

[0033] The image is preprocessed to obtain a series of candidate regions containing the target fish body;

[0034] For each candidate region, a deep learning model is used to extract the feature vector of the candidate region, and the feature vector is converted into a numerical representation of a certain dimension.

[0035] Use a classifier to determine whether each candidate region belongs to the area where the target fish species exists, and output the probability value of the target fish species existing.

[0036] For candidate regions identified as containing the target fish species, a regression algorithm is used to calculate the precise location and size of the target fish species.

[0037] Overlapping bounding boxes are deduplicated, and bounding boxes with low confidence are filtered out to obtain the final target fish species detection results.

[0038] Furthermore, after identifying and locating the target fish species, the navigation and positioning system obtains its current spatial coordinates and superimposes them with the relative coordinates of the target fish species obtained by the imaging system to obtain the position coordinate information of the target fish species. Based on the position coordinates of the target fish species, the system tracks the target fish species.

[0039] Furthermore, the fish control detection system tracks the target fish species based on a target tracking algorithm, namely the Kalman algorithm.

[0040] Furthermore, the fish control detection system tracks the target fish species, and the specific process includes:

[0041] State initialization: Define the target state vector, including position and velocity information, and set the initial covariance matrix;

[0042] State prediction: Using the state vector from the previous moment and combining it with the motion model, predict the target's position at the current moment;

[0043] Predictive covariance update: Predict the state covariance matrix at the current moment using the motion model;

[0044] Observation input: Obtain the observation information at the current moment;

[0045] Estimating the filter gain: The estimated filter gain matrix is ​​calculated using the prediction covariance and the observation noise covariance;

[0046] State correction: Based on the observed values ​​and the estimated filter gain matrix, the target state estimate at the current time is obtained;

[0047] Variance correction: Correct the state covariance matrix at the current time step based on the estimated filter gain matrix.

[0048] In the fish detection system and method of the present invention, a fish-attracting lamp system is used to lure fish to the vicinity, an imaging system is used to acquire video images of the surrounding fish schools, and image recognition technology is used to identify the fish in the video images to identify the target fish species in the images. The identified target fish species is used as the tracking target, and the location information of the target fish species is obtained based on a navigation and positioning system to track the target fish species. This allows for the rapid location of the target fish species and the implementation of operations (commercial fishing or scientific research) for the target fish species and the fish schools.

[0049] Compared with the prior art, the fish detection system and method of the present invention have the following advantages: they can quickly and accurately locate target fish species and schools in a relatively wide body of water, which is conducive to the precise location and fishing of a specific fish species, thereby greatly improving operational efficiency, effectively reducing time and labor costs, and improving economic benefits. Attached Figure Description

[0050] Figure 1 A flowchart of the fish detection method according to the present invention;

[0051] Figure 2 This is a schematic diagram of the detection principle based on an imaging system.

[0052] Figure 3 This is a flowchart illustrating the process of using a target tracking algorithm to track a target fish species. Detailed Implementation

[0053] The present invention will be further illustrated below with specific embodiments:

[0054] This embodiment provides a fish detection system based on light-attracting, which is essentially built upon existing underwater unmanned submersible technology.

[0055] Those skilled in the art will understand that conventional underwater unmanned vehicles are equipped with propulsion devices. In this embodiment, a non-propeller vector water jet propulsion device is used. Pectoral fins are provided on both sides of the vehicle's chest (which can be understood as the lower front part), i.e., two pectoral fins.

[0056] Specifically

[0057] The fish detection system of this embodiment is based on an underwater unmanned submersible and includes an imaging system, a navigation and positioning system, and a fish-attracting light system. In addition, a control module is also provided to achieve overall coordinated control of the entire fish detection system.

[0058] The imaging system is designed to capture and acquire video image data of the surrounding environment.

[0059] The imaging system specifically includes a front-facing camera and a rear-facing camera. The front-facing camera is located on the back of the submersible, and the rear-facing camera is located on the belly of the submersible. The images captured by the front-facing camera and the rear-facing camera are combined to achieve 360-degree surrounding image acquisition.

[0060] It should be noted that, in this embodiment, the imaging system is a three-dimensional stereo imaging system, in which both the front-facing camera and the rear-facing camera are binocular cameras, thereby enabling stereo imaging. In this way, in conjunction with the navigation and positioning system, the three-dimensional spatial positioning of the target object (target fish species) in the image can be achieved.

[0061] The function of the fish-attracting light system is to use light to lure fish to the vicinity of the fish detection system, which is referred to as "luring".

[0062] The fish-attracting lamp system specifically includes two fish-attracting lamps and a spectral measurement subsystem.

[0063] The two fish-attracting lights are a forward-facing fish-attracting light and a backward-facing fish-attracting light, respectively.

[0064] The forward-facing fish-attracting light is located on the back of the submersible. Specifically, a forward-facing circular hole is provided on the back of the submersible, and the forward-facing fish-attracting light is located in the circular hole.

[0065] The rearward fish-attracting light is located on the belly of the submersible. Specifically, a rearward circular hole is provided on the belly of the submersible, and the rearward fish-attracting light is installed in the circular hole.

[0066] The spectral measurement subsystem consists of two spectral measurement sensors, which are fixed inside the pectoral fins on both sides of the submersible by clamps, and these sensors are capable of contacting the water. Based on these spectral measurement sensors, the spectral measurement subsystem can measure spectral data. In this embodiment, the spectral measurement subsystem is used to measure the spectral data of a fish-attracting lamp underwater.

[0067] The fish-attracting light system "uses light to lure fish to gather in a concentrated area." Its specific principle is as follows: The fish-attracting light is turned on, and a spectral measurement subsystem detects the frequency of the light emitted by the light. Then, based on the frequency detected by the spectral measurement subsystem, the light emission frequency of the fish-attracting light is adjusted until the desired target frequency is reached to attract the fish. The fish will then gather around the light, thus achieving the effect of attracting a school of fish.

[0068] The navigation and positioning system is used to acquire satellite positioning information to determine the location (spatial coordinate data information) of the fish detection system and the target fish species.

[0069] The underwater unmanned submersible, imaging system, navigation and positioning system, and fish-attracting light system mentioned above are all connected to the control module via electrical circuits. The control module can receive electrical signals sent by all electrical components and can control all electrical components to operate in coordination.

[0070] It should be noted that the underwater unmanned submersible, imaging system, navigation and positioning system, and fish-attracting light system mentioned earlier all use existing technology devices or equipment.

[0071] The fish detection system in this embodiment is usually installed on a mother ship. When the fish detection system needs to be used for underwater operations, the staff releases the fish detection system from the mother ship into the water. The mother ship and the fish detection system are connected by an extremely thin communication cable and a safety rope. The mother ship is equipped with a dedicated computer system, which is called the "master computer". Through the communication cable, the fish detection system and the master computer on the mother ship can realize data information exchange (i.e., communication).

[0072] The main function of the fish detection system in this embodiment is to locate and track target fish species underwater until the target fish species is found.

[0073] To achieve the above functions, this embodiment also provides a fish detection method, which specifically includes the following steps S1 to S4.

[0074] S1. When operations are required, the staff on the mother ship will deploy the fish detection system into the target waters. Once the fish detection system is in the water, the staff will control the fish-attracting light system on the fish detection system to be turned on via the main control computer. Furthermore, the staff will adjust the light emitted by the fish-attracting light system according to the pre-determined light preferences of the target fish species, so that the light matches the preferences of the target fish species, thereby attracting the target fish species to the vicinity through the light.

[0075] The phrase "adjusting the illumination emitted by the fish-attracting lamp system" specifically refers to adjusting various illumination parameters such as the frequency, intensity, and color of the light.

[0076] Additional note: Most fish species exhibit phototaxis, meaning they are attracted to light with specific spectral characteristics. However, different species display different phototaxis behaviors. By aligning the lighting emitted by the fish-attracting light system with the preferences of the target fish species, they will be more easily lured to the area.

[0077] It should be noted that using a fish-attracting light system to lure target fish species is existing technology, and its specific implementation method and process are common knowledge known to those skilled in the art.

[0078] S2, activate the imaging system to acquire video image data of the area surrounding the fish detection system, especially image data of the surrounding fish; the acquired video image data is transmitted to the main control computer on the mother ship via the communication cable between the fish detection system and the mother ship.

[0079] In this embodiment, during the process of acquiring image data of surrounding fish, the imaging system designates the area where the target is located as the region of interest and tracks the target only for key information within the region of interest, thereby reducing the amount of data processed and effectively improving tracking accuracy.

[0080] In addition, to ensure the quality of the acquired images, the imaging system also includes the following steps S21 to S24 during the image acquisition process.

[0081] S21, Camera calibration, using a camera calibration method based on planar checkerboard markers.

[0082]

[0083] Where A is the camera intrinsic parameter matrix, s is the scale factor, R and t are the extrinsic parameter matrices describing the relationship between the camera and the global coordinate system, i.e., the camera's extrinsic parameter matrices, r_i represents the i-th column vector of R, and u and v are pixel coordinate systems.

[0084] S22, binocular correction, performs distortion removal operation on two images that are not actually coplanar aligned, and corrects them to be coplanar aligned.

[0085] S23, Binocular matching, performs stereo matching on the images to establish a stereo model of the images.

[0086] S24, Calculate depth: Based on the matching results, calculate the depth information of the pixels. The stereo vision equation is as follows:

[0087]

[0088]

[0089]

[0090] Where f is the camera focal length, x_L and x_R are the lateral positions of the target in the imaging planes of the left and right cameras, d is the parallax, b is the baseline, X is the lateral distance of the target, Y is the height of the target, and Z is the longitudinal distance of the target.

[0091] S3 uses existing image recognition technology to continuously identify fish in video images in order to identify the target fish species in the images.

[0092] Specifically

[0093] The main control computer on the mother ship is pre-configured with a fish species identification module (software module), and the target fish species to be identified are set for the fish species identification module. After the main control computer receives the video image data transmitted by the fish detection system, the fish species identification module continuously identifies the fish in the video image in order to identify the target fish species in the image.

[0094] It should be noted that, in this embodiment, the fish species identification module is essentially an image recognition module built on a deep learning model.

[0095] More specifically, the process of identifying fish bodies in video images includes the following steps S31 to S35.

[0096] S31, use existing image processing methods to preprocess the image to obtain a series of candidate regions containing the target fish body.

[0097] The image processing techniques employed include image pyramids, sliding windows, and so on.

[0098] S32. For each candidate region, a deep learning model is used to extract its feature vector and transform it into a numerical representation of a certain dimension.

[0099] S33: Use a classifier to determine whether each candidate region belongs to the area where the target fish species exists, and output the probability value of the existence of the target fish species.

[0100] In this embodiment, the classifier used is the softmax classifier.

[0101] S34. For candidate regions that are identified as containing the target fish species, a regression algorithm is used to calculate the precise location and size of the target fish species.

[0102] The regression algorithm used can be linear regression, multinomial regression, etc.

[0103] S35. For those overlapping bounding boxes or bounding boxes with low confidence, deduplication or filtering is performed to obtain the final target fish species detection results.

[0104] In addition, after identifying the target fish species, the navigation and positioning system obtains the current spatial coordinates of the fish detection system and superimposes them with the relative coordinates of the target fish species obtained by the imaging system. This allows the system to obtain the position information of the target fish species in three-dimensional space, which is beneficial for subsequent tracking of the target fish species.

[0105] S4. After the target fish species is identified, it is used as the tracking target. Existing target tracking technology is used to control the fish detection system to track the target fish species until the fish school where the target fish species is located is found.

[0106] During the tracking of the target fish species, the lens of the imaging system is always pointed towards the target fish species being tracked, so that the target fish species is always in the central area of ​​the video image.

[0107] In this embodiment, an existing target tracking algorithm is used to track the target fish species. This target tracking algorithm defines each window as a state variable of the current frame and achieves motion tracking of the 3D object by predicting the next state variable.

[0108] In this embodiment, the target tracking algorithm used is the Kalman algorithm.

[0109] See Figure 3 In this embodiment, the process of tracking the target fish species using the target tracking algorithm specifically includes the following steps S41 to S47.

[0110] S41, State Initialization: Define the target state vector, including information such as position and velocity, and set the initial covariance matrix.

[0111] It should be noted that the target mentioned here refers to the fish in the image.

[0112] S42, State Prediction: Predict the target's position at the current moment by combining the state vector from the previous moment with the motion model.

[0113] S43, Predicting Covariance Update: Predicting the state covariance matrix at the current moment using the motion model.

[0114] S44, Observation Input: Obtain the observation information at the current moment.

[0115] S45, Estimating Filter Gain: Calculate the estimated filter gain matrix using the prediction covariance and the observation noise covariance.

[0116] S46, State Correction: Based on the observed values ​​and the estimated filter gain matrix, obtain the target state estimate at the current time.

[0117] S47, Variance Correction: Correct the state covariance matrix at the current time based on the estimated filter gain matrix.

[0118] See Figure 2 It should be noted that the deep learning-based target tracking algorithm achieves 3D target detection by establishing a connection between the image and the world coordinate system, and obtains the target's category and true size. In the implementation process, the image is divided into multiple grid regions, and a small stride window is established in each grid for continuous convolution.

[0119] In this embodiment, the position information of the moving object in the world is provided by combining the aforementioned binocular camera with a satellite navigation system. The satellite navigation system is used to obtain the current position of the instrument, and combined with the calculated object coordinates, the position information of the object in the world coordinate system is obtained. The conversion from camera coordinates to world coordinates is as follows:

[0120]

[0121] Among them, (X) c ,Y c Z c (X) is the camera coordinate system. W ,Y W Z W () is the world coordinate system.

[0122] It should be noted that the "target fish species" mentioned in this article refers to the fish species targeted for capture in fishing operations, or the fish species targeted for scientific research.

[0123] The fish detection system in this embodiment includes a fish-attracting lamp system, an imaging system, and a navigation and positioning system. In the fish detection method of this embodiment, the fish-attracting lamp system is used to lure fish to the vicinity, the imaging system is used to acquire video images of the surrounding fish schools, and image recognition technology is used to identify the fish in the video images to identify the target fish species. The identified target fish species is used as the tracking target, and the location information of the target fish species is obtained based on the navigation and positioning system to track the target fish species. This allows for the rapid location of the target fish species and the implementation of operations (commercial fishing or scientific research) targeting the target fish species and school.

[0124] The fish detection system and method of this embodiment can quickly and accurately locate target fish species and schools, which is beneficial for the precise location and fishing of a specific fish species, thereby greatly improving operational efficiency, effectively reducing time and labor costs, and enhancing economic benefits.

[0125] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A fish detection system based on light-attracting, characterized in that: The fish detection system is based on an underwater unmanned submersible, which is equipped with a fish-attracting light system, an imaging system, and a navigation and positioning system. The fish-attracting light system is used to lure fish together. The imaging system is used to acquire video images of the surrounding area; The navigation and positioning system is used to acquire positioning information; The fish detection system is used to locate target fish species and schools, and its specific implementation method includes: Deploy the fish detection system into the water and turn on the fish-attracting light system and imaging system of the fish detection system. The fish-attracting light system lures fish to the vicinity, and the imaging system acquires video images of the surrounding area; Image recognition technology is used to identify fish in video images and pinpoint the target fish species in the images; The identified target fish species are used as the tracking target, and the fish detection system is controlled to track the target fish species. The method of using image recognition technology to identify fish in video images and find the target fish species in the images is as follows: the image recognition technology used is: using a fish species identification module to identify fish in video images in order to identify the target fish species in the images. The fish species identification module is an image recognition module built based on a deep learning model; The specific process of identifying fish in video images includes: The image is preprocessed to obtain a series of candidate regions containing the target fish body; For each candidate region, a deep learning model is used to extract the feature vector of the candidate region, and the feature vector is converted into a numerical representation of a certain dimension. Use a classifier to determine whether each candidate region belongs to the area where the target fish species exists, and output the probability value of the target fish species existing. For candidate regions identified as containing the target fish species, a regression algorithm is used to calculate the precise location and size of the target fish species. Overlapping bounding boxes are deduplicated, and bounding boxes with low confidence are filtered out to obtain the final target fish species detection results.

2. A method for detecting fish, characterized in that: include: A fish detection system is employed, which is based on an underwater unmanned submersible. The underwater unmanned submersible is equipped with a fish-attracting light system, an imaging system, and a navigation and positioning system. The fish-attracting light system is used to lure fish to gather together; the imaging system is used to acquire video images of the surrounding area. The navigation and positioning system is used to acquire positioning information; Deploy the fish detection system into the water and turn on the fish-attracting light system and imaging system of the fish detection system. The fish-attracting light system lures fish to the vicinity, and the imaging system acquires video images of the surrounding area; Image recognition technology is used to identify fish in video images and pinpoint the target fish species in the images; The identified target fish species are used as the tracking target, and the fish detection system is controlled to track the target fish species. The method of using image recognition technology to identify fish in video images and find the target fish species in the image is as follows: A fish species identification module is used to identify fish in video images in order to identify the target fish species in the images; The fish species identification module is an image recognition module built based on a deep learning model; The specific process of identifying fish in video images includes: The image is preprocessed to obtain a series of candidate regions containing the target fish body; For each candidate region, a deep learning model is used to extract the feature vector of the candidate region, and the feature vector is converted into a numerical representation of a certain dimension. Use a classifier to determine whether each candidate region belongs to the area where the target fish species exists, and output the probability value of the target fish species existing. For candidate regions identified as containing the target fish species, a regression algorithm is used to calculate the precise location and size of the target fish species. Overlapping bounding boxes are deduplicated, and bounding boxes with low confidence are filtered out to obtain the final target fish species detection results.

3. The fish detection method according to claim 2, characterized in that: After turning on the fish-attracting light system, adjust the light emitted by the system according to the preferences of the target fish species.

4. The fish detection method according to claim 2, characterized in that: During the process of acquiring surrounding video images by the imaging system, the area where the target fish species is located is set as the region of interest, and target tracking is performed only on key information within the region of interest.

5. The fish detection method according to claim 2, characterized in that: After identifying and locating the target fish species, the navigation and positioning system obtains its current spatial coordinates and superimposes them with the relative coordinates of the target fish species obtained by the imaging system to obtain the position coordinate information of the target fish species. Based on the position coordinates of the target fish species, the system tracks the target fish species.

6. The fish detection method according to claim 2, characterized in that: The fish control detection system tracks the target fish species based on a target tracking algorithm, namely the Kalman algorithm.

7. The fish detection method according to claim 6, characterized in that: The fish control detection system tracks the target fish species, and the specific process includes: State initialization: Define the target state vector, including position and velocity information, and set the initial covariance matrix; State prediction: Using the state vector from the previous moment and combining it with the motion model, predict the target's position at the current moment; Predictive covariance update: Predict the state covariance matrix at the current moment using the motion model; Observation input: Obtain the observation information at the current moment; Estimating the filter gain: The estimated filter gain matrix is ​​calculated using the prediction covariance and the observation noise covariance; State correction: Based on the observed values ​​and the estimated filter gain matrix, the target state estimate at the current time is obtained; Variance correction: Correct the state covariance matrix at the current time step based on the estimated filter gain matrix.

Images