A method and system for dynamically collecting in-situ squid fishing behavior of fish by fusing space-time environmental factors

Abstract

The application discloses a kind of fusion space-time environmental factors and system of squid fishing fish behavior in situ dynamic acquisition method.The method comprises the following steps: obtaining in situ microscale hydrological data through observation platform;Hydrological data and historical distribution data are input into space-time distribution prediction model, and probability distribution map is output to drive observation platform to navigate and seek independently;After visual module captures target, the temporal pattern change characteristics of motion video stream are extracted using deep neural network on edge computing node and behavior classification label is output;A multidimensional state matrix is constructed, and a random forest regression model is input to predict the three-dimensional spatial position offset of the target;According to the offset, the pose compensation parameters are calculated and the shooting angle and focal length are adjusted to achieve continuous center shooting.The application realizes automatic observation of fish behavior through multi-scale environmental perception and double-layer seeking, edge computing behavior recognition and trajectory prediction constrained by environmental fluid, and solves the problem of difficult continuous tracking of fish in complex underwater environment.

Description

Technical Field

[0001] This invention belongs to the field of marine fishery resource observation and intelligent sensing technology, specifically involving an in-situ dynamic acquisition method and system for squid and angfish behavior that integrates spatiotemporal environmental factors. It is particularly suitable for automatic identification and dynamic visual tracking of fish underwater behavior using multi-source sensors and edge computing. Background Technology

[0002] In existing technologies, the observation of the behavior of marine fish such as squid mainly relies on macroscopic fishing ground prediction models (such as Chinese patent CN106157162A), which predict the approximate range and abundance of central fishing grounds by analyzing large-scale environmental factors such as sea surface temperature (SST). Although such macroscopic models can indicate the potential distribution area of ​​fish schools, they are difficult to capture and track the real-time behavior of individual fish at the microscopic scale.

[0003] Meanwhile, some existing underwater in-situ observation equipment has achieved in-situ image acquisition at the hardware level, but its data processing and behavior analysis mostly rely on manual interpretation or offline analysis. It lacks a closed-loop system that can integrate micro-environmental data in real time, autonomously identify fish behavior, and dynamically adjust visual acquisition strategies. This leads to prominent problems of "difficulty in finding fish and losing track of fish" in complex marine environments, making it difficult to obtain high-quality in-situ continuous behavioral data. Summary of the Invention

[0004] This invention was developed to address the aforementioned problems and aims to provide a method and system for in-situ dynamic acquisition of squid fishing behavior that integrates spatiotemporal environmental factors. It seamlessly connects macroscopic spatiotemporal distribution ecological models with microscopic deep learning behavior prediction, realizing a fully automated in-situ observation method for fish behavior "from finding fish to photographing fish".

[0005] This invention provides an in-situ dynamic acquisition method for squid fishing behavior based on spatiotemporal environmental factors, comprising the following steps:

[0006] Step S1: In-situ microscale hydrological data is acquired in real time through environmental sensors mounted on the observation platform. The in-situ microscale hydrological data includes the three-dimensional temperature and salinity gradient field and microflow field vector data of the current water area.

[0007] Step S2: Input the in-situ microscale hydrological data and historical distribution data into the preset spatiotemporal distribution prediction model, output the local microenvironmental spatial probability distribution map of the target fish population, and drive the observation platform to autonomously search along the path of probability gradient increase in the probability distribution map.

[0008] Step S3: After the visual module of the observation platform captures the target fish, it continuously collects the motion video stream of the target fish, and uses a deep neural network deployed on the edge computing node to extract the temporal morphological change features of the target fish in the motion video stream, and outputs typical behavior classification labels that represent the current behavioral state.

[0009] Step S4: Construct a multidimensional state matrix from typical behavior classification labels, historical movement trajectories of the target fish, and current microfluidic field vector data, and input the multidimensional state matrix into a pre-trained random forest regression prediction model to predict the three-dimensional spatial position offset of the target fish in the next time window in real time.

[0010] Step S5: Based on the three-dimensional spatial position offset, the pose compensation parameters of the visual imaging mechanism of the observation platform are calculated in reverse, and the shooting tracking angle and focal length are adjusted in advance to achieve continuous centered shooting of the dynamic behavior of the target fish.

[0011] Preferably, in step S1, in-situ microscale hydrological data is acquired in real time by a micro temperature, salinity, depth meter and an acoustic Doppler current profiler mounted on the observation platform, and is used to construct a three-dimensional microscale gradient field containing temperature, salinity and water flow vectors.

[0012] Preferably, in step S2, the spatiotemporal distribution prediction model is a generalized additive model (GAM) or a maximum entropy model (MaxEnt).

[0013] Preferably, in step S2, the autonomous search includes: global layer search, which determines macro hotspot areas based on macro historical spatiotemporal data (SST), salinity data, and spatiotemporal distribution prediction models; and local layer search, in which the observation platform calculates the spatial derivative of the three-dimensional temperature-salinity gradient field within the macro hotspot areas, and sets the hydrological strata and microscale eddy edges as high-weight areas for priority detection.

[0014] Preferably, in step S3, the deep neural network is a spatiotemporal three-dimensional convolutional neural network or a long short-term memory network containing spatial convolutional layers and temporal memory units; the extracted temporal morphological change features include at least the tail wagging frequency features and pectoral fin deflection angle features of the target fish.

[0015] Preferably, in step S4, when the random forest regression prediction model builds the decision tree, it uses the microfluidic field vector data as a weight correction factor for node splitting; when the behavior classification label is escape or feeding behavior, the model increases the feature importance weight of the microfluidic field vector data in predicting the position offset of the next time window.

[0016] This invention provides an in-situ dynamic acquisition system for squid fishing behavior that integrates spatiotemporal environmental factors, comprising: a multi-scale environmental perception module for acquiring in-situ micro-scale hydrological data in real time through environmental sensors mounted on an observation platform, including three-dimensional temperature-salinity gradient field and microfluidic field vector data of the current water area; an autonomous navigation and homing module for inputting the in-situ micro-scale hydrological data and historical distribution data into a pre-set spatiotemporal distribution prediction model, outputting a local micro-environmental spatial probability distribution map of the target fish group, and driving the observation platform to autonomously hom to the water area where the target fish is located along the probability distribution map; and a behavior recognition and classification module for continuously acquiring motion videos of the target fish after the visual module of the observation platform captures the target fish. The system analyzes the flow of data and utilizes deep neural networks deployed on edge computing nodes to extract temporal morphological change features from the motion video stream, outputting typical behavior classification labels representing the current behavioral state. A trajectory prediction calculation module constructs a multidimensional state matrix from the typical behavior classification labels, the target fish's historical movement trajectory, and the current microfluidic field vector data. This multidimensional state matrix is ​​then input into a pre-trained random forest regression prediction model to predict the target fish's three-dimensional spatial position offset in the next time window in real time. A dynamic tracking and shooting module calculates the pose compensation parameters of the observation platform's visual shooting mechanism based on the three-dimensional spatial position offset, and adjusts the shooting tracking angle and focal length in advance to achieve continuous, centered shooting of the target fish's dynamic behavior.

[0017] The present invention provides an electronic device, including a memory and a processor. The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the method described above.

[0018] The present invention provides a computer storage medium storing computer-executable instructions thereon, which, when executed by a processor, implement the steps of the method described above.

[0019] Technical effect

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] 1. Multi-scale environmental perception and autonomous homing: By constructing a micro-scale three-dimensional temperature and salinity gradient field and micro-flow field map, the macro-fishing ground prediction is combined with micro-environment gradient navigation, realizing the observation platform's refined autonomous homing in high-probability waters, effectively solving the problem that macro-prediction is difficult to locate specific fish schools.

[0022] 2. Edge computing and real-time behavior recognition: Deploying deep neural networks on edge computing nodes enables real-time processing and behavior classification (such as feeding and escaping) of fish movement video streams directly at the observation end, avoiding bandwidth consumption and processing delays caused by transmitting large amounts of raw video data back, thus improving response speed.

[0023] 3. Trajectory prediction and dynamic tracking under environmental fluid constraints: By fusing behavioral classification labels, historical motion trajectories and microfluidic field vector data into a multidimensional state matrix, the matrix is ​​input into a random forest model to predict position offset. The vision mechanism is controlled in advance to adjust the focal length and angle, enabling the prediction of sudden behaviors (such as escape) and continuous centered shooting, thus solving the problem of "losing the fish" in complex underwater environments. Attached Figure Description

[0024] Figure 1 This is a flowchart of the in-situ dynamic acquisition method for squid fishing behavior that integrates spatiotemporal environmental factors in Embodiment 1 of the present invention;

[0025] Figure 2 This is a block diagram of the in-situ dynamic acquisition system for squid fishing behavior that integrates spatiotemporal environmental factors in Embodiment 2 of the present invention. Detailed Implementation

[0026] To make the technical means, creative features, objectives and effects of this invention easier to understand, the following embodiments, in conjunction with the accompanying drawings, specifically illustrate a method and system for in-situ dynamic acquisition of squid fishing behavior that integrates spatiotemporal environmental factors.

[0027] Example 1

[0028] This embodiment provides an in-situ dynamic acquisition method for squid fishing behavior that integrates spatiotemporal environmental factors. The core steps of the method in this embodiment are:

[0029] Real-time in-situ hydrological data (such as microscale temperature and salinity gradients) are collected and used as input parameters. Combined with historical spatiotemporal distribution characteristics, the probability distribution of the occurrence of the target fish population is initially predicted using the maximum entropy model (MaxEnt) or the generalized additive model (GAM).

[0030] The observation platform autonomously navigates to the high-probability water area based on the distribution map and initiates visual pre-scanning.

[0031] Once a fish target is captured, the deep neural network (DNN) deployed on the edge computing node is immediately invoked to extract and classify the temporal features of typical fish behaviors (such as feeding, escaping, and courtship).

[0032] By combining the Random Forest algorithm, the motion trajectory of the next second is predicted in real time based on the preceding behavior sequence, and the shooting focal length and tracking angle are adjusted in advance accordingly.

[0033] Figure 1 This is a flowchart of the in-situ dynamic acquisition method for squid fishing behavior that integrates spatiotemporal environmental factors in an embodiment of the present invention.

[0034] like Figure 1 As shown, specifically, the in-situ dynamic acquisition method for squid fishing behavior based on the fusion of spatiotemporal environmental factors in this embodiment includes the following steps:

[0035] Step S1, Multi-scale environmental perception and micro-environment map construction: In-situ micro-scale hydrological data are acquired in real time through environmental sensors mounted on the observation platform. The in-situ micro-scale hydrological data includes the three-dimensional temperature and salinity gradient field and micro-flow field vector data of the current water area.

[0036] In this embodiment, in addition to collecting sea surface temperature (SST) or macroscopic salinity, the observation platform needs to be equipped with a high-frequency miniature CTD (conductivity, temperature, depth) and an acoustic Doppler current profiler (ADCP). During the cruise, a "microscale three-dimensional temperature and salinity gradient field" and a "micro-flow field" map are constructed in real time.

[0037] Step S2, a two-layer search strategy based on environmental gradient (to solve the difficulty of finding fish): input in-situ microscale hydrological data and historical distribution data into a pre-set spatiotemporal distribution prediction model, output a local microenvironmental spatial probability distribution map of the target fish group, and drive the observation platform to autonomously search for the water area where the target fish is located along the probability distribution map.

[0038] In this embodiment, at the global level, an improved generalized additive model (GAM) or maximum entropy model (MaxEnt) is used, combined with macroscopic SST and salinity data, to output macroscopic hotspot areas at the 10-kilometer level. At the local level, upon entering a hotspot area, the algorithm switches to a "micro-environment sensitive mode." Since fish often congregate at the edges of temperature gradients or micro-scale eddies, the system calculates the temperature-salinity gradient vector in real time, guiding the observation platform to autonomously cruise along the steepest gradient or the edge of a micro-eddy.

[0039] Step S3, deep sequence classification for behavior perception (solving the difficulty of losing track): After the visual module of the observation platform captures the target fish, it continuously collects the motion video stream of the target fish, and uses a deep neural network deployed on the edge computing node to extract the temporal morphological change features of the motion video stream, and outputs typical behavior classification labels that represent the current behavior state.

[0040] In this embodiment, a spatiotemporal three-dimensional convolutional neural network (such as 3D-CNN or combined with LSTM) is used to process a continuous video stream. It not only identifies what the fish is doing (feeding, escaping, swimming), but more importantly, it extracts the deformation features of its body posture (such as tail wagging frequency and pectoral fin angle) to generate a "high-frequency behavior feature vector".

[0041] Furthermore, by combining the speed, color, and direction of the squid hook with the squid's behavioral judgment mechanism regarding the hook, we can predict the squid's predation and escape behavior.

[0042] Step S4: Construct a multidimensional state matrix from typical behavior classification labels, historical movement trajectories of the target fish, and current microfluidic field vector data. Input the multidimensional state matrix into a pre-trained random forest regression prediction model to predict the three-dimensional spatial position offset of the target fish in the next time window in real time.

[0043] To address the shortcomings of traditional prediction methods that only consider the fish's own movement history, this embodiment combines local water flow vectors (measured by ADCP), microenvironmental gradient distribution, and current behavioral characteristics (DNN output) into a feature matrix, which is then input into the RandomForest regression model.

[0044] Step S5: Based on the three-dimensional spatial position offset, calculate the pose compensation parameters of the visual imaging mechanism of the observation platform, and adjust the shooting tracking angle and focal length in advance to achieve continuous centered shooting of the target fish's dynamic behavior.

[0045] In this embodiment, if the DNN determines that the fish is in a "startled and fleeing" state, and there is a strong current on one side, the RF model will combine fluid dynamics priors to predict that the fish will most likely move downstream or lunge into deeper water. Based on this, the system will drive the gimbal or platform to turn 0.5-1 seconds in advance, perfectly keeping the fish in the center of the frame.

[0046] Example 2

[0047] This embodiment provides an in-situ dynamic acquisition system for squid fishing behavior that integrates spatiotemporal environmental factors.

[0048] Figure 2 This is a block diagram of the in-situ dynamic acquisition system for squid fishing behavior that integrates spatiotemporal environmental factors in this embodiment.

[0049] like Figure 2 As shown, the in-situ dynamic acquisition system for squid fishing behavior that integrates spatiotemporal environmental factors in this embodiment includes a multi-scale environmental perception module 10, an autonomous navigation and homing module 20, a behavior recognition and classification module 30, a trajectory prediction and calculation module 40, and a dynamic tracking and shooting module 50.

[0050] The multi-scale environment perception module 10 is used to execute the process in step S1 of embodiment 1.

[0051] The autonomous navigation module 20 is used to execute the process in step S2 of embodiment 1.

[0052] The behavior recognition and classification module 30 is used to execute the process in step S3 of embodiment 1.

[0053] The trajectory prediction calculation module 40 is used to execute the process in step S4 of embodiment 1.

[0054] The dynamic tracking and shooting module 50 is used to execute the process in step S5 of embodiment 1.

[0055] The above embodiments are preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention.

Claims

1. A method for in-situ dynamic acquisition of squid fishing behavior by integrating spatiotemporal environmental factors, characterized in that, Includes the following steps: Step S1: In-situ microscale hydrological data is acquired in real time through environmental sensors mounted on the observation platform. The in-situ microscale hydrological data includes the three-dimensional temperature and salinity gradient field and microflow field vector data of the current water area. Step S2: Input the in-situ microscale hydrological data and historical distribution data into the preset spatiotemporal distribution prediction model, output the local microenvironmental spatial probability distribution map of the target fish group, and drive the observation platform to autonomously find the water area where the target fish is located along the probability distribution map; Step S3: After the visual module of the observation platform captures the target fish, it continuously collects the motion video stream of the target fish, and uses a deep neural network deployed on the edge computing node to extract the temporal morphological change features of the motion video stream, and outputs typical behavior classification labels that represent the current behavior state. Step S4: Construct a multidimensional state matrix from the typical behavior classification labels, the historical movement trajectory of the target fish, and the current micro-flow field vector data, and input the multidimensional state matrix into a pre-trained random forest regression prediction model to predict the three-dimensional spatial position offset of the target fish in the next time window in real time. Step S5: Based on the three-dimensional spatial position offset, calculate the pose compensation parameters of the visual imaging mechanism of the observation platform, and adjust the shooting tracking angle and focal length in advance to achieve continuous centered shooting of the dynamic behavior of the target fish.

2. The in-situ dynamic acquisition method for squid fishing behavior based on spatiotemporal environmental factors according to claim 1, characterized in that: In step S1, the in-situ microscale hydrological data is acquired in real time by the micro temperature, salinity and depth meter and the acoustic Doppler current profiler mounted on the observation platform, and is used to construct a three-dimensional microscale gradient field containing temperature, salinity and water flow vectors.

3. The in-situ dynamic acquisition method for squid fishing behavior based on spatiotemporal environmental factors according to claim 1, characterized in that: In step S2, the spatiotemporal distribution prediction model is a generalized additive model or a maximum entropy model.

4. The in-situ dynamic acquisition method for squid fishing behavior based on spatiotemporal environmental factors according to claim 1, characterized in that: In step S2, the autonomous search includes: The global layer search identifies macro-hotspot areas based on macro-historical spatiotemporal data, salinity data, and the aforementioned spatiotemporal distribution prediction model. In the local layer-finding process, within the macroscopic hotspot region, the observation platform calculates the spatial derivative of the three-dimensional temperature-salinity gradient field and sets the hydrological strata and the edge of the microscale eddy as high-weight regions for priority detection.

5. The in-situ dynamic acquisition method for squid fishing behavior based on spatiotemporal environmental factors according to claim 1, characterized in that: In step S3, the deep neural network is a spatiotemporal three-dimensional convolutional neural network or a long short-term memory network containing spatial convolutional layers and temporal memory units; the extracted temporal morphological change features include at least the tail wagging frequency features and pectoral fin deflection angle features of the target fish.

6. The in-situ dynamic acquisition method for squid fishing behavior based on spatiotemporal environmental factors according to claim 1, characterized in that: In step S4, when the random forest regression prediction model builds the decision tree, it uses the microfluidic field vector data as a weight correction factor for node splitting; when the behavior classification label is escape or feeding behavior, the model increases the feature importance weight of the microfluidic field vector data in predicting the position offset of the next time window.

7. A system for in-situ dynamic acquisition of squid fishing behavior by integrating spatiotemporal environmental factors, characterized in that, include: The multi-scale environmental perception module is used to acquire in-situ micro-scale hydrological data in real time through environmental sensors mounted on the observation platform. The in-situ micro-scale hydrological data includes the three-dimensional temperature and salinity gradient field and micro-flow field vector data of the current water area. The autonomous navigation module is used to input the in-situ microscale hydrological data and historical distribution data into a preset spatiotemporal distribution prediction model, output a local microenvironmental spatial probability distribution map of the target fish group, and drive the observation platform to autonomously navigate to the waters where the target fish are located along the probability distribution map. The behavior recognition and classification module is used to continuously acquire motion video streams of target fish after the vision module of the observation platform captures the target fish, and to extract the temporal morphological change features of the motion video stream using a deep neural network deployed on edge computing nodes, and output typical behavior classification labels that characterize the current behavior state. The trajectory prediction calculation module is used to construct a multi-dimensional state matrix from the typical behavior classification labels, the historical movement trajectory of the target fish and the current micro-flow field vector data, and input the multi-dimensional state matrix into a pre-trained random forest regression prediction model to predict the three-dimensional spatial position offset of the target fish in the next time window in real time. The dynamic tracking and shooting module is used to calculate the pose compensation parameters of the visual shooting mechanism of the observation platform based on the three-dimensional spatial position offset, and adjust the shooting tracking angle and focal length in advance to achieve continuous centered shooting of the dynamic behavior of the target fish.

8. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method as described in any one of claims 1-6.

9. A computer-readable storage medium storing computer-executable instructions thereon, characterized in that: When the computer-executable instructions are executed by a processor, they implement the steps of the method as described in any one of claims 1-6.

Images