Intelligent all-time sturgeon and guinea pig monitoring method and system

Abstract

The invention relates to the technical field of fish monitoring, and discloses an intelligent all-time sturgeon and guinea pig monitoring method and system, and the method comprises the steps: carrying out the video stream preprocessing of the video stream data of sturgeons and guinea pigs, and obtaining the fluid video images of sturgeons and guinea pigs; inputting the fluid video pictures of the sturgeons and the guinea pigs into target detection models of the sturgeons and the guinea pigs to obtain target detection results of the sturgeons and the guinea pigs; tracking sturgeon and guinea pig targets based on the sturgeon and guinea pig target detection results to obtain sturgeon and guinea pig target tracking results; performing quantity statistics on the sturgeon and guinea pig targets based on the target detection results of the sturgeons and guinea pigs to obtain target statistical results of the sturgeons and guinea pigs; and determining sturgeon and guinea pig monitoring results based on the sturgeon and guinea pig target detection results, the sturgeon and guinea pig target tracking results and the sturgeon and guinea pig target statistical results. The sturgeon and guinea pig monitoring system improves the sturgeon and guinea pig monitoring efficiency, real-time performance and range, and reduces the monitoring cost.

Description

Technical Field

[0001] The present invention relates to the technical field of fish monitoring, and in particular to an intelligent full-time sturgeon and dolphin monitoring method and system. Background Art

[0002] Sturgeons and dolphins are precious aquatic organisms on Earth, with extremely high scientific research, ecological and cultural value. However, due to factors such as overfishing, water pollution, and water conservancy project construction, the populations of sturgeons and dolphins have decreased sharply and are in danger of extinction. Therefore, real-time monitoring of sturgeons and dolphins, as well as the rational management and protection of fish resources and the avoidance of overexploitation, are of great significance for maintaining the balance of aquatic ecosystems and achieving the sustainable development and utilization of fish resources.

[0003] Currently, sturgeons and dolphins are monitored through manual patrols, but manual patrols have problems such as low efficiency, poor real-time performance, high cost and small coverage, resulting in poor monitoring results. Summary of the Invention

[0004] In view of this, the present invention provides an intelligent full-time sturgeon and dolphin monitoring method and system to solve the problem of poor monitoring effect on sturgeons and dolphins.

[0005] In a first aspect, the present invention provides an intelligent full-time sturgeon and dolphin monitoring method, the method comprising:

[0006] Obtaining video stream data of sturgeons and dolphins in the target waters, performing video stream preprocessing on the video stream data of sturgeons and dolphins, and obtaining fluid video images of the sturgeons and dolphins;

[0007] Inputting sturgeon and dolphin fluid video images into the sturgeon and dolphin target detection model to obtain sturgeon and dolphin target detection results; wherein the sturgeon and dolphin target detection model includes a YOLOv10 layer, a flattening layer, and a DeTr layer;

[0008] Based on the target detection results of sturgeons and dolphins, the sturgeons and dolphins are tracked to obtain the target tracking results of the sturgeons and dolphins;

[0009] Based on the target detection results of sturgeons and dolphins, the number of sturgeons and dolphins targets is counted to obtain the target statistical results of sturgeons and dolphins;

[0010] The monitoring results of sturgeons and dolphins are determined based on the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistical results of sturgeons and dolphins.

[0011] The intelligent full-time sturgeon and dolphin monitoring method provided in this embodiment obtains video stream data of sturgeons and dolphins in the target waters, performs video stream preprocessing on the video stream data of sturgeons and dolphins, and obtains fluid video images of sturgeons and dolphins; inputs the fluid video images of sturgeons and dolphins into the target detection model of sturgeons and dolphins to obtain target detection results of sturgeons and dolphins; wherein the target detection model of sturgeons and dolphins includes a YOLOv10 layer, a flattening layer, and a DeTr layer; based on the target detection results of sturgeons and dolphins, the sturgeon and dolphin targets are tracked to obtain target tracking results of sturgeons and dolphins; based on the target detection results of sturgeons and dolphins, the target detection results of sturgeons and dolphins are obtained. The number of sturgeon and dolphin targets is counted to obtain target statistical results of sturgeons and dolphins; the monitoring results of sturgeons and dolphins are determined based on the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistical results of sturgeons and dolphins; the target detection results of sturgeons and dolphins are obtained by inputting fluid video images of sturgeons and dolphins into the target detection model of sturgeons and dolphins, and the sturgeon and dolphin targets are tracked and counted based on the target detection results of sturgeons and dolphins, thereby achieving accurate monitoring of sturgeons and dolphins in the target waters, improving the monitoring efficiency, real-timeness and scope of sturgeons and dolphins, and reducing the monitoring costs of sturgeons and dolphins.

[0012] In an optional embodiment, video stream preprocessing is performed on the video stream data of sturgeons and dolphins to obtain fluid video images of sturgeons and dolphins, including:

[0013] Obtaining video stream data of sturgeons and dolphins in the target waters, and performing fluidization processing on the video stream data of sturgeons and dolphins to obtain initial video images of sturgeons and dolphins;

[0014] Based on the initial video images of sturgeons and dolphins, the adversarial generative model is used to perform image deblurring to obtain fluid video images of sturgeons and dolphins.

[0015] The intelligent, full-time sturgeon and dolphin monitoring method provided in this embodiment obtains video stream data of sturgeons and dolphins in target waters, performs fluidization processing on the sturgeon and dolphin video stream data to obtain initial sturgeon and dolphin video images, and then uses a generative adversarial model to deblur the initial sturgeon and dolphin video images. This generates high-definition, high-resolution fluid video images of sturgeons and dolphins with clear target boundaries. This eliminates image blur, color distortion, and fish posture changes caused by insufficient underwater imaging conditions, turbid water, and disturbances, thereby improving the accuracy of sturgeon and dolphin target detection results.

[0016] In an optional embodiment, the sturgeon and dolphin fluid video images are input into the sturgeon and dolphin target detection model to obtain the sturgeon and dolphin target detection results, including:

[0017] Based on the fluid video images of sturgeons and dolphins, feature extraction is performed using the YOLOv10 layer to obtain the resulting feature map;

[0018] Use the flatten layer to convert the resulting feature map into sequence data;

[0019] Based on the sequence data, the DeTr layer is used for target detection, and the target detection results of sturgeons and dolphins are obtained.

[0020] The intelligent, full-time sturgeon and dolphin monitoring method provided in this embodiment uses the YOLOv10 layer to extract features from fluid video images of sturgeons and dolphins, obtaining a resulting feature map that effectively extracts the features of the sturgeon and dolphin targets. The resulting feature map is converted into sequence data using a flattening layer, and target detection is performed using a DeTr layer based on the sequence data, achieving efficient detection of sturgeons and dolphins and improving the accuracy of the target detection results for sturgeons and dolphins.

[0021] In an optional embodiment, before inputting the fluid video images of sturgeons and dolphins into the target detection model of sturgeons and dolphins to obtain the target detection results of sturgeons and dolphins, the method further includes:

[0022] Target samples of sturgeons and dolphins are obtained, and an initial target detection model is trained using the target samples of sturgeons and dolphins to obtain target detection models of sturgeons and dolphins.

[0023] The intelligent, full-time sturgeon and dolphin monitoring method provided in this embodiment obtains target samples of sturgeons and dolphins and uses these target samples to train an initial target detection model, thereby constructing a target detection model for sturgeons and dolphins, and laying the foundation for obtaining target detection results for sturgeons and dolphins.

[0024] In an optional embodiment, the initial target detection model is trained using target samples of sturgeons and dolphins to obtain target detection models of sturgeons and dolphins, including:

[0025] Input the target samples of sturgeon and dolphin into the initial target detection model to obtain target prediction data of sturgeon and dolphin;

[0026] Based on the target prediction data of sturgeon and dolphin and the target annotation data of sturgeon and dolphin, the target positioning loss value, sequence loss value and target bounding box sharpness loss value are calculated respectively;

[0027] The initial target detection model was iteratively adjusted using the target localization loss, sequence loss, and target bounding box sharpness loss to obtain the target detection models for sturgeons and dolphins.

[0028] The intelligent full-time sturgeon and dolphin monitoring method provided in this embodiment calculates the target positioning loss value, sequence loss value, and target bounding box sharpness loss value based on the target prediction data of sturgeons and dolphins and the target annotation data of sturgeons and dolphins, and uses the target positioning loss value, sequence loss value, and target bounding box sharpness loss value to iteratively adjust the initial target detection model, thereby realizing the training of the target detection model for sturgeons and dolphins, improving the target detection capability of the target detection model for sturgeons and dolphins, and laying the foundation for accurately obtaining target detection results for sturgeons and dolphins.

[0029] In an optional embodiment, tracking the sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain the sturgeon and dolphin target tracking results includes:

[0030] Based on the target detection results of sturgeons and dolphins, the target tracking algorithm is used to determine the predicted locations of each type of sturgeons and dolphins;

[0031] Based on the predicted positions of various types of sturgeons and dolphins, a retractable rotating multi-mode camera was used to track the targets of various types of sturgeons and dolphins, and the target tracking results of sturgeons and dolphins were obtained.

[0032] The intelligent, full-time sturgeon and dolphin monitoring method provided in this embodiment uses a target tracking algorithm based on the target detection results of sturgeons and dolphins to determine the predicted positions of various types of sturgeons and dolphins, thereby accurately predicting the positions of target sturgeons and dolphins. Furthermore, a retractable, rotating multi-mode camera is used to track various types of sturgeons and dolphins, thereby achieving dynamic monitoring and tracking of various types of sturgeons and dolphins and accurately obtaining target tracking results for sturgeons and dolphins.

[0033] In a second aspect, the present invention provides an intelligent full-time sturgeon and dolphin monitoring system, the system comprising:

[0034] A video stream preprocessing module is used to obtain video stream data of sturgeons and dolphins in the target waters, perform video stream preprocessing on the video stream data of sturgeons and dolphins, and obtain fluid video images of sturgeons and dolphins;

[0035] A target detection module is used to input the sturgeon and dolphin fluid video images into the sturgeon and dolphin target detection model to obtain the sturgeon and dolphin target detection results; wherein the sturgeon and dolphin target detection model includes a YOLOv10 layer, a flattening layer, and a DeTr layer;

[0036] A tracking module is used to track sturgeon and dolphin targets based on the target detection results of sturgeon and dolphin, and obtain target tracking results of sturgeon and dolphin;

[0037] A statistics module is used to count the number of sturgeons and dolphins based on the target detection results of sturgeons and dolphins, and obtain the target statistics results of sturgeons and dolphins;

[0038] The determination module is used to determine the monitoring results of sturgeons and dolphins based on the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistical results of sturgeons and dolphins.

[0039] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to thereby execute the intelligent full-time sturgeon and dolphin monitoring method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0040] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the intelligent full-time sturgeon and dolphin monitoring method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0041] In a fifth aspect, the present invention provides a computer program product comprising computer instructions for causing a computer to execute the intelligent full-time sturgeon and dolphin monitoring method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 1 is a flow chart of an intelligent full-time sturgeon and dolphin monitoring method according to an embodiment of the present invention;

[0044] Figure 2 is a structural diagram of a retractable and rotatable multi-mode camera according to an embodiment of the present invention;

[0045] Figure 3 1 is a flow chart of another intelligent full-time sturgeon and dolphin monitoring method according to an embodiment of the present invention;

[0046] Figure 4 2 is a flow chart of another intelligent full-time sturgeon and dolphin monitoring method according to an embodiment of the present invention;

[0047] Figure 51 is a flow chart of another intelligent full-time sturgeon and dolphin monitoring method according to an embodiment of the present invention;

[0048] Figure 6 is a structural block diagram of an intelligent full-time sturgeon and dolphin monitoring system according to an embodiment of the present invention;

[0049] Figure 7 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0051] Sturgeons and dolphins are both precious aquatic organisms on Earth, with extremely high scientific, ecological and cultural value. Due to their long evolutionary history, sturgeons are often regarded as "living fossils" of the biological world. Sturgeons belong to the order Acipenser, class of bony fishes, while dolphins belong to the order Cetacea, class of Mammalia. Despite differences in species attributes, sturgeons and dolphins occupy key ecological niches in their respective ecosystems. As important components of freshwater and marine ecosystems, sturgeons and dolphins play the roles of top predators and high-level consumers in the food chain, respectively. The dynamic changes in their populations are of great significance to maintaining the structural stability and functional integrity of aquatic ecosystems.

[0052] In recent years, due to the intensification of human activities, the population of sturgeons and dolphins has declined sharply, and their survival is facing severe challenges. Overfishing has hindered the replenishment of their populations, and the deterioration of water quality caused by industrial wastewater, domestic sewage discharge and agricultural non-point source pollution has seriously damaged the habitat environment of sturgeons and dolphins. At the same time, water conservancy project construction, channel dredging and other activities have not only severed their migration channels, but also changed the hydrological conditions, further compressing their living space.

[0053] Sturgeons and dolphins have distinctive features such as unique appearance, morphology and movement. Adult sturgeons and dolphins can reach more than 3 meters in length and weigh more than 500 kilograms, making them giants among freshwater fish. They have wide heads, transversely split mouths located at the bottom of the heads, and four tentacles. Their bodies are covered with five rows of hard bone plates (called bony scales), their backs are usually dark gray or black, and their abdomens are lighter in color. Sturgeons and dolphins have streamlined bodies, which are suitable for fast swimming in the water. The above characteristics can be used as a basis for monitoring sturgeons and dolphins in target waters.

[0054] However, the monitoring and protection of sturgeons and dolphins is mainly achieved through manual patrols, which have disadvantages such as low efficiency, lack of real-time performance, high monitoring costs and limited coverage, and cannot meet the needs of modern biodiversity conservation. Therefore, exploring efficient and accurate monitoring technologies and building a scientific and reasonable resource management and protection system are of great practical significance for achieving the sustainable utilization of sturgeon and dolphin resources and maintaining the health of aquatic ecosystems. How to break through technical bottlenecks and improve monitoring effects has become a key issue that needs to be urgently addressed in the field of biological conservation.

[0055] To solve the above technical problems, the embodiments of the present invention provide an intelligent full-time sturgeon and dolphin monitoring method. It should be noted that the sturgeon and dolphin monitoring method provided by the embodiments of the present invention can be executed by an intelligent full-time sturgeon and dolphin monitoring system. The intelligent full-time sturgeon and dolphin monitoring system can be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware. The electronic device can be a server or a terminal. The server in the embodiments of the present application can be a single server or a server cluster composed of multiple servers. The terminal in the embodiments of the present application can be a smart phone, personal computer, tablet computer, wearable device, smart robot or other smart hardware device. In the following method embodiments, the execution subject is an electronic device as an example for explanation.

[0056] According to an embodiment of the present invention, an embodiment of an intelligent full-time sturgeon and dolphin monitoring method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0057] In this embodiment, an intelligent full-time sturgeon and dolphin monitoring method is provided, which can be used in the above-mentioned electronic equipment. Figure 1 FIG. 1 is a flow chart of an intelligent full-time sturgeon and dolphin monitoring method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0058] Step S101 , obtaining video stream data of sturgeons and dolphins in a target water area, performing video stream preprocessing on the video stream data of sturgeons and dolphins, and obtaining fluid video images of sturgeons and dolphins.

[0059] Specifically, if Figure 2As shown, a retractable and rotatable multi-mode camera is used to shoot sturgeons and dolphins in a target water area to obtain video stream data of the sturgeons and dolphins. The retractable and rotatable multi-mode camera includes: a fixing plate, a rectangular box cover, an L-shaped carrier plate, a spherical camera, a card reader sensor, a temperature sensor switch, a connecting base plate, a deep screw hole, a long screw rod, a placement slot, a micro motor, a central processing unit, a PCB (Printed Circuit Board), a communication module, an electric control switch module, an information recognition module, and an information database module.

[0060] Furthermore, the retractable and rotatable multi-mode camera has retractable and rotatable functions, which can be used to shoot sturgeons and dolphins at different angles and track targets. At the same time, the retractable and rotatable multi-mode camera has a multi-mode imaging function and can shoot video data in different modes; the retractable and rotatable multi-mode camera will not damage the ecological environment or affect the normal life of fish.

[0061] Furthermore, after the retractable rotating multi-mode camera shoots the video stream data of sturgeons and dolphins, it is sent to the sturgeon and dolphin monitoring device through the streaming media transmission protocol, wherein the streaming media transmission protocol adopts the RTSP (Real Time Streaming Protocol) protocol.

[0062] In step S102 , the fluid video images of sturgeons and dolphins are input into a target detection model of sturgeons and dolphins to obtain target detection results of sturgeons and dolphins. The target detection model of sturgeons and dolphins includes a YOLOv10 layer, a flattening layer, and a DeTr layer.

[0063] Specifically, the target detection model for sturgeons and dolphins uses the YOLOv10-DeTr model. The YOLOv10-DeTr model is a model that combines the YOLOv10 (You Only Look Once version 10, a target detection algorithm based on a deep neural network) model with the DeTr (Detection Transformer, a target detection algorithm based on a deep neural network) model through a flattening layer. It enhances sharp features at the rotation boundary to adapt to the special appearance characteristics of sturgeons and dolphins. The YOLOv10-DeTr model combines the advantages of the YOLO model and the DeTr model and uses a series of network layers to achieve efficient target detection.

[0064] Step S103 : Tracking the sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain sturgeon and dolphin target tracking results.

[0065] Step S104 , counting the number of sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain sturgeon and dolphin target statistical results.

[0066] Specifically, the target detection results of sturgeons and dolphins provide information such as the predicted bounding boxes of sturgeons and dolphins, their categories, and confidence levels. Each sturgeon and dolphin target in each video frame is numbered, and Faster R-CNN (Faster Region-based Convolutional Neural Network) is used to further accurately classify and locate the sturgeons and dolphins in the current frame, thereby improving the accuracy of sturgeon and dolphin target recognition. The detected targets are tracked using a target tracking algorithm, and the same sturgeon and dolphin targets are numbered and matched based on their motion information (such as position change and speed) in consecutive frames. At the same time, a target ID (Identification) list of sturgeons and dolphins is maintained in each frame. The above list is used to track and count active sturgeons and dolphins in the target waters.

[0067] Step S105 , determining the monitoring results of sturgeons and dolphins based on the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistical results of sturgeons and dolphins.

[0068] Specifically, the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistical results of sturgeons and dolphins are stored in the back-end server and database as the monitoring results of sturgeons and dolphins, and the corresponding data storage model and index structure are established based on this, and the monitoring results of sturgeons and dolphins are visually displayed to users through the WEB (World Wide Web) front-end.

[0069] Furthermore, the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistical results of sturgeons and dolphins are stored in the form of files in the back-end server and database, and are organized into a hierarchical structure with directories and sub-directories according to the corresponding attribute information, so as to establish a corresponding data storage model and index structure to facilitate retrieval and access of the stored data, establish corresponding discrimination rules, and provide a corresponding API interface (Application Programming Interface) for WEB front-end scheduling and management; the stored data is retrieved and displayed through the WEB front-end, and various results are visually displayed to users.

[0070] The intelligent full-time sturgeon and dolphin monitoring method provided in this embodiment obtains video stream data of sturgeons and dolphins in the target waters, performs video stream preprocessing on the video stream data of sturgeons and dolphins, and obtains fluid video images of sturgeons and dolphins; inputs the fluid video images of sturgeons and dolphins into the target detection model of sturgeons and dolphins to obtain target detection results of sturgeons and dolphins; wherein the target detection model of sturgeons and dolphins includes a YOLOv10 layer, a flattening layer, and a DeTr layer; based on the target detection results of sturgeons and dolphins, the sturgeon and dolphin targets are tracked to obtain target tracking results of sturgeons and dolphins; based on the target detection results of sturgeons and dolphins, the target detection results of sturgeons and dolphins are obtained. The number of sturgeon and dolphin targets is counted to obtain target statistical results of sturgeons and dolphins; the monitoring results of sturgeons and dolphins are determined based on the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistical results of sturgeons and dolphins; the target detection results of sturgeons and dolphins are obtained by inputting fluid video images of sturgeons and dolphins into the target detection model of sturgeons and dolphins, and the sturgeon and dolphin targets are tracked and counted based on the target detection results of sturgeons and dolphins, thereby achieving accurate monitoring of sturgeons and dolphins in the target waters, improving the monitoring efficiency, real-timeness and scope of sturgeons and dolphins, and reducing the monitoring costs of sturgeons and dolphins.

[0071] In this embodiment, an intelligent full-time sturgeon and dolphin monitoring method is provided, which can be used in the above-mentioned electronic equipment. Figure 3 FIG. 1 is a flow chart of an intelligent full-time sturgeon and dolphin monitoring method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0072] Step S301 : obtaining video stream data of sturgeons and dolphins in a target water area, performing video stream preprocessing on the video stream data of sturgeons and dolphins, and obtaining fluid video images of sturgeons and dolphins.

[0073] Specifically, the above step S301 includes:

[0074] Step S3011: obtaining video stream data of sturgeons and dolphins in the target waters, and performing fluidization processing on the video stream data of sturgeons and dolphins to obtain initial video images of sturgeons and dolphins.

[0075] Specifically, the fluidization processing includes: video stream data decoding, video stream data format conversion, video stream data noise reduction and video stream data enhancement.

[0076] Step S3012: Based on the initial video images of sturgeons and dolphins, an adversarial generative model is used to perform image deblurring processing to obtain fluid video images of sturgeons and dolphins.

[0077] Specifically, even after the initial sturgeon and dolphin video images are obtained through fluidization processing such as video stream data denoising and video stream data enhancement, there may still be problems such as image blur, color distortion, and changeable fish postures caused by uneven underwater lighting and turbid water. Therefore, an adversarial generative model is used for image deblurring. The adversarial generative model adopts DeblurGAN (Deblurring Generative Adversarial Network). The DeblurGAN model is used to generate high-definition, noise-suppressed, and well-defined fluid video images of sturgeons and dolphins.

[0078] Furthermore, DeblurGAN optimizes the generator and discriminator through adversarial training to restore underwater blurred images caused by insufficient imaging conditions, motion, or defocus. The generator extracts features from the initial sturgeon and dolphin video images through multiple layers of convolution, activation functions, and pooling operations, and ultimately generates deblurred initial sturgeon and dolphin video images. The generator's goal is to learn the mapping from blurred images to clear images. The discriminator is also a convolutional neural network, whose function is to determine whether the input image is a real clear image or the deblurred initial sturgeon and dolphin video images generated by the generator. The discriminator's goal is to distinguish between the two types of images as accurately as possible, thereby prompting the generator to continuously improve the deblurred initial sturgeon and dolphin video images. During the training process, the generator attempts to generate deblurred initial sturgeon and dolphin video images that are as realistic as possible to deceive the discriminator, while the discriminator strives to improve its discrimination ability to distinguish between real and generated sturgeon and dolphin fluid video images. The above adversarial process enables the generator to learn a more effective deblurring strategy.

[0079] Furthermore, DeblurGAN uses content loss (measuring the pixel-level difference between the fluid video images of sturgeons and dolphins generated by the generator and the real clear images) and adversarial loss (measuring the difference in the discrimination results between the generated fluid video images of sturgeons and dolphins and the real images in the discriminator) to train the model. By minimizing the above two losses, the generator can learn how to generate high-quality deblurred fluid video images of sturgeons and dolphins while maintaining the details and texture of the fluid video images of sturgeons and dolphins.

[0080] Furthermore, the content loss is a loss function that measures the pixel-level difference between the generated sturgeon and dolphin fluid video images and the real images, which is used to ensure that the generated images have sufficient content information and structural similarity; the content loss L content The calculation formula is as follows:

[0081]

[0082] Among them, φ is the feature extractor, x i is the original image, G(z i ) is the generated clear image, N is the number of samples, z i is a sample image, and G is a generator.

[0083] Furthermore, adversarial loss is an important component of conditional adversarial networks (GANs), which is used to measure the separability of generated sturgeon and dolphin fluid video images from real images in the discriminator. Its goal is to continuously improve the generator and generate "real" images that are difficult for the discriminator to distinguish. In DeblurGAN, the role of adversarial loss is to improve the authenticity of the generated images and make the generated sturgeon and dolphin fluid video images look more natural. The adversarial loss L adv The calculation formula is as follows:

[0084]

[0085] Among them, D is the discriminator, x is the real image, and z is the input noise of the generator. For the real image x in a given probability distribution p data The expected value of (x), For random variable z under given probability distribution p z The expected value of (z).

[0086] Furthermore, in DeblurGAN, the ultimate goal is to combine content loss and adversarial loss to optimize the overall performance of the generator. The calculation formula of the model loss is as follows:

[0087] L total =αL content +βL adv (3)

[0088] Among them, α and β are weight coefficients used to balance the contribution of the two losses, L total is the model loss of DeblurGAN.

[0089] Step S302 : Tracking the sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain the sturgeon and dolphin target tracking results; wherein the sturgeon and dolphin target detection model includes a YOLOv10 layer, a flattening layer, and a DeTr layer.

[0090] Specifically, the target detection model for sturgeons and dolphins uses convolutional layers for preliminary feature extraction during the feature extraction process, and then introduces self-attention layers to process sequence data and enhance feature expression. In order to improve the detection effect of sturgeons and dolphins, object embedding layers (Object Embedding Layers) and position encoding layers (Position Encoding Layers) are added to the target detection model for sturgeons and dolphins, and multi-scale feature map input layers (Multi-Scale Feature Input Layers) are applied to extract information from different scales.

[0091] The above step S302 includes:

[0092] Step S3021: Based on the fluid video images of sturgeons and dolphins, feature extraction is performed using the YOLOv10 layer to obtain a result feature map.

[0093] Specifically, the YOLOv10 layer uses a multi-scale convolutional neural network (CNN) for feature extraction, producing feature maps at different levels, including low-level features (such as the edges and colors of sturgeons and dolphins) and high-level features (such as the limbs and overall appearance of sturgeons and dolphins). At the same time, the YOLOv10 layer applies a multi-scale feature input layer to extract features of sturgeons and dolphins at different scales.

[0094] Step S3022: Use a flattening layer to convert the resulting feature map into sequence data.

[0095] Specifically, since the DeTr layer adopts the Transformer architecture (a deep learning architecture) and mainly processes sequence data, the result feature map output by the YOLOv10 layer is a tensor, so before inputting the DeTr layer, it is necessary to use the flattening layer to convert the result feature map into sequence data; the process of using the flattening layer to convert the result feature map into sequence data includes: 1) inputting the result feature map ((C, H, W)); where C is the number of channels, H is the height, and W is the width; 2) inputting the result feature map ((C, H, W)) into the object embedding layer (Object Embedding Layers) for semantic feature embedding to obtain a semantic feature map ((C', H, W)); where C' is the number of channels of the semantic feature map; 3) inputting the semantic feature map ((C', H, W)) into the position encoding layer (Position Encoding Layers) for position encoding to obtain a semantic feature map with enhanced spatial information; 4) using the convolutional layer to extract the 8 corner point features of each pixel in the semantic feature map with enhanced spatial information; 5) using the self-attention layer (Self-Attention Layers) Layers) fuse the 8 corner features with the semantic feature map that enhances spatial information through a weighted attention mechanism to obtain a corner feature map ((C’’, H, W)); where C’’ is the number of channels of the corner feature map; 6) flatten the spatial dimension of the corner feature map, i.e., ((H, W) to H*W), and then retain the channel dimension (C’’) as the feature dimension of the sequence to obtain the flattened corner feature map ((C’’, H*W)); 7) input the flattened corner feature map ((C’’, H*W)) into the position encoding layer for position encoding to obtain sequence data; by introducing object embedding (i.e., using the object embedding layer for semantic feature embedding), position encoding (i.e., using the position encoding layer for position encoding) and adding 8 corner features to the sequence data, the sharp boundary features are highlighted, the expression effect of the sturgeon and dolphin fins is enhanced, and the target objects and target positions of different sturgeons and dolphins can be accurately represented.

[0096] Step S3023: Based on the sequence data, target detection is performed using the DeTr layer to obtain target detection results for sturgeons and dolphins.

[0097] Specifically, the DeTr layer uses a self-attention mechanism to process sequence data to obtain target detection results for sturgeons and dolphins. The target detection results are feature vectors, which include: 1) the location of sturgeons and dolphins: the feature vector contains the predicted bounding box coordinates (such as center point coordinates, width and height) used to locate sturgeons and dolphins in the image; 2) the category of sturgeons and dolphins: the feature vector provides the category information of each detected sturgeon and dolphin; 3) confidence score: the model outputs a confidence score, indicating the degree of confidence that sturgeons and dolphins exist in the predicted bounding box; 4) contextual information: since the DeTr model adopts a Transformer structure, the feature vector also contains the relationship and contextual information between different regions in the video fluid image of sturgeons and dolphins; 5) multi-scale features: through multi-level feature extraction, the feature vector contains feature information from different scales, which is conducive to detecting sturgeons and dolphins of different sizes.

[0098] Step S303: Track the sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain sturgeon and dolphin target tracking results. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0099] Step S304: Count the number of sturgeons and dolphins based on the sturgeon and dolphin target detection results to obtain the sturgeon and dolphin target statistical results. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0100] Step S305: Determine the monitoring results of sturgeons and dolphins based on the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistics results of sturgeons and dolphins. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.

[0101] The intelligent, full-time sturgeon and dolphin monitoring method provided in this embodiment obtains video stream data of sturgeons and dolphins in target waters, performs fluidization processing on the sturgeon and dolphin video stream data to obtain initial sturgeon and dolphin video images, and then uses a generative adversarial model to deblur the initial sturgeon and dolphin video images. This generates high-definition, high-resolution fluid video images of sturgeons and dolphins with clear target boundaries. This eliminates image blur, color distortion, and fish posture changes caused by insufficient underwater imaging conditions, turbid water, and disturbances, thereby improving the accuracy of sturgeon and dolphin target detection results.

[0102] In this embodiment, an intelligent full-time sturgeon and dolphin monitoring method is provided, which can be used in the above-mentioned electronic equipment. Figure 4FIG. 1 is a flow chart of an intelligent full-time sturgeon and dolphin monitoring method according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0103] Step S401: Obtain video stream data of sturgeons and dolphins in the target waters, perform video stream preprocessing on the video stream data of sturgeons and dolphins, and obtain fluid video images of sturgeons and dolphins. Figure 3 Step S301 of the illustrated embodiment will not be described in detail here.

[0104] Step S402: Obtain target samples of sturgeons and dolphins, and use the target samples of sturgeons and dolphins to train an initial target detection model to obtain a target detection model of sturgeons and dolphins.

[0105] Specifically, the initial target detection model adopts the YOLOv10-DeTr model, and the YOLOv10-DeTr model is trained using target samples of sturgeons and dolphins to obtain target detection models of sturgeons and dolphins; wherein the above step S402 includes:

[0106] Step S4021: Input target samples of sturgeons and dolphins into the initial target detection model to obtain target prediction data of sturgeons and dolphins.

[0107] Specifically, the target samples of sturgeons and dolphins are manually selected and labeled data on sturgeons and dolphins, including features such as the appearance, morphology, and movement patterns of various sturgeons and dolphins. The initial target detection model is trained using the target samples of sturgeons and dolphins. Sturgeons and dolphins have distinctive features such as unique appearance, morphology, and movement patterns. These features include: adult sturgeons and dolphins can reach a length of more than 3 meters and a weight of more than 500 kilograms, making them giants among freshwater fish; their heads are relatively wide, with a transversely split mouth located at the bottom of the head and four tentacles; their body surfaces are covered with five rows of hard bony plates (called bony scales), with the back usually dark gray or black and the belly lighter in color; and their bodies are streamlined, suitable for fast swimming in water.

[0108] Step S4022 , based on the target prediction data of sturgeons and dolphins and the target annotation data of sturgeons and dolphins, respectively calculate the target positioning loss value, the sequence loss value, and the target bounding box sharpness loss value.

[0109] Specifically, the target samples of sturgeon and dolphin are input into the initial target detection model to obtain the target prediction data of sturgeon and dolphin, and the target positioning loss value, sequence loss value and target bounding box sharpness loss value are calculated based on the target prediction data of sturgeon and dolphin and the target annotation data of sturgeon and dolphin, respectively; the target positioning loss of the YOLOv10 layer, the sequence loss of the DeTr layer and the target bounding box sharpness loss of the flattening layer are optimized to improve the accuracy of the sturgeon and dolphin target prediction data; among which, the target positioning loss value consists of the position loss value and the confidence loss value; the sequence loss value includes the classification loss value and the position loss value; the target bounding box sharpness loss value is used to optimize the target bounding box shape, including the corner point loss value and the boundary integrity loss value.

[0110] Furthermore, the position loss value is used to measure the difference between the predicted bounding box and the true bounding box; it can be calculated using the mean squared error (MSE) or IoU (Intersection over Union); the confidence loss is used to measure the difference between the predicted value of the initial target detection model for the probability of the existence of sturgeon and dolphin targets and the true value, which can be obtained by calculating the cross entropy loss of the sturgeon and dolphin target prediction data and the sturgeon and dolphin target labeled data; the classification loss is used to evaluate the accuracy of the initial target detection model in predicting the position of sturgeons and dolphins, which can be obtained by calculating the cross entropy loss of the sturgeon and dolphin target prediction data and the sturgeon and dolphin target labeled data; the corner loss is the distance between the corner points of the predicted bounding box and the corner points of the true bounding box, which can be calculated by the mean squared error; the boundary integrity loss is used to utilize the gradient information of the boundary to promote the feature expression of the model at the boundary, ensuring that the boundary of the target is clearer and sharper, which can be obtained by performing loss calculation on the boundary area.

[0111] Furthermore, the calculation process of the mean square error includes: 1) calculating the difference between the predicted data of the sturgeon and dolphin targets and the corresponding labeled data of the sturgeon and dolphin targets, 2) squaring the difference to obtain multiple square errors, and 3) calculating the mean of the multiple square errors to obtain the mean square error; the calculation process of the intersection-over-union ratio includes: 1) determining the coordinate information of the sturgeon and dolphin target predicted bounding box and the sturgeon and dolphin target labeled bounding box, 2) calculating the intersection area and the union area of ​​the sturgeon and dolphin target predicted bounding box and the sturgeon and dolphin target labeled bounding box, 3) calculating the intersection area and the union area of ​​the sturgeon and dolphin target predicted bounding box and the sturgeon and dolphin target labeled bounding box, and 4) calculating the intersection area and the union area of ​​the sturgeon and dolphin target predicted bounding box and the sturgeon and dolphin target labeled bounding box. )The area of ​​the intersection region is divided by the area of ​​the union region to obtain the intersection-union ratio; the calculation process of the cross entropy loss is as follows: the logarithmic loss is calculated based on the true label in the sturgeon and dolphin target annotation data and the sturgeon and dolphin target prediction confidence, and then the logarithmic loss of all sturgeon and dolphin target samples is summed to obtain the cross entropy loss; the calculation process of the boundary integrity loss is as follows: the gradient amplitude of the target prediction bounding box and the target annotation bounding box is calculated, and the boundary integrity loss is calculated based on the gradient amplitude of the target prediction bounding box and the gradient amplitude of the target annotation bounding box.

[0112] In step S4023, the initial target detection model is iteratively adjusted using the target positioning loss value, the sequence loss value, and the target bounding box sharpness loss value to obtain target detection models for sturgeons and dolphins.

[0113] Specifically, if the target localization loss value, sequence loss value, and target bounding box sharpness loss value do not meet the preset iteration stop conditions, the parameters will be continuously adjusted, including learning rate, batch size, Anchor Boxes, number of queries, number of Transformer layers, and loss function weights, to improve the recognition and positioning capabilities of sturgeon and dolphin targets.

[0114] Furthermore, the performance indicators of the target detection model for sturgeons and dolphins include mAP (mean Average Precision), IoU and validation loss. If the performance indicators of the target detection model for sturgeons and dolphins do not improve significantly after 5 to 10 consecutive epochs (the number of times all training samples are forward and backward propagated once during model training), the training is stopped to obtain the initial target detection model after training, that is, the target detection model for sturgeons and dolphins.

[0115] Furthermore, the target detection model for sturgeons and dolphins is also used to automatically interpret new fluidized video images, quickly process large amounts of data, and preliminarily identify and label potential sturgeons and dolphins. Although the automated interpretation results may have some errors, the workload of manual processing is greatly reduced; subsequently, the manual fine-tuning phase will review the automated interpretation results, correct errors, supplement omissions, and refine the annotation information; the results of manual fine-tuning will be added to the target samples of sturgeons and dolphins for the next round of model training; the above iterative process continuously expands and optimizes the target samples of sturgeons and dolphins, and obtains the target sample library of sturgeons and dolphins with semi-automatic annotation and deep reinforcement learning, thereby continuously improving the performance of the target detection model for sturgeons and dolphins; when the performance indicators of the target detection model for sturgeons and dolphins meet the standards and the errors in the automated interpretation results are significantly reduced, the iterative process stops.

[0116] Furthermore, the sturgeon and dolphin target detection model performs automated interpretation, rewards and penalties, and manual correction on sturgeon and dolphin fluid video images. The processed sturgeon and dolphin fluid video images are used to continuously optimize the sturgeon and dolphin target sample library, and the sturgeon and dolphin target detection model is trained with the sturgeon and dolphin target sample library until the detection performance of the sturgeon and dolphin target detection model meets the detection performance standards. This establishes a comprehensive sturgeon and dolphin target sample library with reward and penalty feedback and an increasing sample size. The sturgeon and dolphin target sample library contains multi-source data obtained from different imaging modes, adapting to complex scenes and conditions such as uneven underwater lighting, turbid water, and uneven color, forming a multi-mode, multi-scale, and multi-angle sturgeon and dolphin target recognition knowledge set. The advantage of the semi-automatic labeling method is that it can balance efficiency and accuracy. Among them, the automated processing improves efficiency, while the manual fine-tuning ensures the quality of the results. The rich and diverse knowledge set greatly enhances the model's ability to recognize sturgeon and dolphin targets in complex scenes.

[0117] Furthermore, after the training of the target detection model for sturgeons and dolphins is completed, the recognition accuracy of the target detection model for sturgeons and dolphins can be judged. The recognition accuracy is determined by comprehensively measuring the positioning and classification performance of the target detection model for sturgeons and dolphins during the target detection process. The judgment indicators of recognition accuracy include: mAP, IoU, precision and recall; mAP is used to evaluate the overall detection performance, IoU is used to measure the degree of overlap between the predicted box and the true box, and the corner loss and boundary integrity loss of the target bounding box are used to optimize the detailed features. After the detection is completed, the accuracy of each sturgeon and dolphin fluid video image is recorded by saving the recognition results, and a penalty mechanism is used to optimize false detection and missed detection samples, and a reward mechanism is used to enhance the correct detection samples, thereby improving the robustness and detection accuracy of the model.

[0118] Furthermore, for sturgeon and dolphin fluid video images with higher recognition accuracy, corresponding rewards will be given to the above-mentioned sturgeon and dolphin fluid video images based on their contribution to improving the overall recognition performance; such rewards include but are not limited to increasing the weights of the above-mentioned sturgeon and dolphin fluid video images so that they are sampled more frequently during the model training process, or marking samples with significant contributions so that researchers can more easily identify and analyze these high-quality sturgeon and dolphin fluid video images.

[0119] Alternatively, when the recognition system has low recognition accuracy for certain images, the above-mentioned sturgeon and dolphin fluid video images and their corresponding sturgeon and dolphin target samples will be regarded as factors that contribute less to improving the recognition performance of the sturgeon and dolphin target detection model; in the above case, the penalty mechanism will intervene to reduce its weight in the training process, reduce its frequency of use in subsequent iterations, and provide feedback to researchers or developers so that the sturgeon and dolphin target detection model or the sturgeon and dolphin target samples can be manually fine-tuned, corrected and optimized.

[0120] Step S403: Input the sturgeon and dolphin fluid video images into the sturgeon and dolphin target detection model to obtain the sturgeon and dolphin target detection results; wherein the sturgeon and dolphin target detection model includes a YOLOv10 layer, a flattening layer, and a DeTr layer. Figure 3 Step S303 of the illustrated embodiment will not be described in detail here.

[0121] Step S404: Track the sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain sturgeon and dolphin target tracking results. Figure 3 Step S303 of the illustrated embodiment will not be described in detail here.

[0122] Step S405: Count the number of sturgeons and dolphins based on the sturgeon and dolphin target detection results to obtain the sturgeon and dolphin target statistical results. Figure 3 Step S304 of the illustrated embodiment will not be described in detail here.

[0123] Step S406: Determine the monitoring results of sturgeons and dolphins based on the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistics results of sturgeons and dolphins. Figure 3 Step S305 of the illustrated embodiment will not be described in detail here.

[0124] The intelligent, full-time sturgeon and dolphin monitoring method provided in this embodiment uses the YOLOv10 layer to extract features from fluid video images of sturgeons and dolphins, obtaining a resulting feature map that effectively extracts the features of sturgeon and dolphin targets. The resulting feature map is converted into sequence data using a flattening layer, and target detection is performed using a DeTr layer based on the sequence data, achieving efficient detection of sturgeons and dolphins and improving the accuracy of sturgeon and dolphin target detection results.

[0125] In this embodiment, an intelligent full-time sturgeon and dolphin monitoring method is provided, which can be used in the above-mentioned electronic equipment. Figure 5 FIG. 1 is a flow chart of an intelligent full-time sturgeon and dolphin monitoring method according to an embodiment of the present invention. Figure 5 As shown, the process includes the following steps:

[0126] Step S501: Obtain video stream data of sturgeons and dolphins in the target waters, perform video stream preprocessing on the video stream data of sturgeons and dolphins, and obtain fluid video images of sturgeons and dolphins. Figure 4 Step S401 of the illustrated embodiment will not be described in detail here.

[0127] Step S502: Input the sturgeon and dolphin fluid video images into the sturgeon and dolphin target detection model to obtain the sturgeon and dolphin target detection results; wherein the sturgeon and dolphin target detection model includes a YOLOv10 layer, a flattening layer, and a DeTr layer. Figure 4 Step S403 of the illustrated embodiment will not be described in detail here.

[0128] Step S503 : Tracking the sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain sturgeon and dolphin target tracking results.

[0129] Specifically, the above step S503 includes:

[0130] Step S5031 : Based on the target detection results of sturgeons and dolphins, a target tracking algorithm is used to determine the predicted positions of the sturgeons and dolphins of each category.

[0131] Specifically, the target tracking algorithm adopts the Deep SORT (Simple Online and Realtime Tracking) algorithm. The steps of using Deep SORT to determine the predicted positions of various categories of sturgeons and dolphins include: obtaining the predicted bounding boxes and feature vectors in the target detection results of sturgeons and dolphins, and inputting the predicted bounding boxes and feature vectors into the Deep SORT algorithm. The Deep SORT algorithm combines Kalman filtering and appearance feature similarity to achieve multi-target tracking. By analyzing the historical trajectories and appearance features of sturgeons and dolphins, DeepSORT can maintain tracking of sturgeons and dolphins in continuous video frames. At the same time, the predicted positions of various categories of sturgeons and dolphins are predicted through Kalman filtering, thereby achieving accurate dynamic monitoring of sturgeons and dolphins, and then controlling the retractable rotating multi-mode camera to track sturgeons and dolphins targets according to the predicted positions of various categories of sturgeons and dolphins.

[0132] Step S5032: Based on the predicted positions of the sturgeons and dolphins of each category, the retractable rotating multi-mode camera is used to track the sturgeons and dolphins of each category to obtain target tracking results of the sturgeons and dolphins.

[0133] Specifically, a backend server uses a retractable and rotatable multi-mode camera to track targets of various categories of sturgeons and dolphins. The steps of obtaining target tracking results of sturgeons and dolphins include: 1) adjusting the monitoring angle and direction of the retractable and rotatable multi-mode camera according to the predicted positions of various categories of sturgeons and dolphins; 2) the retractable and rotatable multi-mode camera sends the captured video data of various categories of sturgeons and dolphins to the backend server; 3) the backend server classifies and matches the video data of various categories of sturgeons and dolphins based on appearance and motion information to obtain target tracking results of sturgeons and dolphins.

[0134] Step S504: Count the number of sturgeons and dolphins based on the sturgeon and dolphin target detection results to obtain the sturgeon and dolphin target statistical results. Figure 4 Step S405 of the illustrated embodiment will not be described in detail here.

[0135] Step S505: Determine the monitoring results of sturgeons and dolphins based on the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistics results of sturgeons and dolphins. Figure 4 Step S406 of the illustrated embodiment will not be described in detail here.

[0136] The intelligent, full-time sturgeon and dolphin monitoring method provided in this embodiment uses a target tracking algorithm to determine the predicted positions of various types of sturgeons and dolphins based on the target detection results of sturgeons and dolphins, thereby accurately predicting the positions of target sturgeons and dolphins. Furthermore, a retractable, rotating multi-mode camera is used to track various types of sturgeons and dolphins, thereby achieving dynamic monitoring and tracking of various types of sturgeons and dolphins and accurately obtaining target tracking results for sturgeons and dolphins.

[0137] This embodiment also provides an intelligent, full-time sturgeon and dolphin monitoring system. This system is used to implement the above-mentioned embodiments and preferred embodiments, and the details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0138] The following describes the specific steps of an intelligent full-time sturgeon and dolphin monitoring method through a specific embodiment.

[0139] Example 1:

[0140] 1) A retractable rotating multi-mode camera is deployed in the waters where sturgeons and dolphins live to shoot the sturgeons and dolphins in the target waters to obtain video stream data of the sturgeons and dolphins;

[0141] 2) Establish a sturgeon and dolphin target sample library using semi-automatic annotation and deep reinforcement learning for precise identification of sturgeons and dolphins;

[0142] 3) The initial target detection model is trained using the target sample library of sturgeons and dolphins to obtain the target detection model of sturgeons and dolphins.

[0143] 4) The captured video stream data of sturgeons and dolphins is sent to a sturgeon and dolphin monitoring platform via a streaming media transmission protocol. The sturgeon and dolphin monitoring platform performs fluid processing on the sturgeon and dolphin video stream data and uses a generative adversarial model to perform image deblurring to generate high-definition, noise-suppressed, and well-defined fluid video images of sturgeons and dolphins;

[0144] 5) inputting the processed sturgeon and dolphin fluid video images into a sturgeon and dolphin target detection model to obtain sturgeon and dolphin target detection results;

[0145] 6) Using a retractable rotating multi-mode camera to track various types of sturgeons and dolphins, and obtaining target tracking results of sturgeons and dolphins;

[0146] 7) Counting the number of sturgeons and dolphins based on the sturgeon and dolphin target detection results to obtain sturgeon and dolphin target statistical results;

[0147] 8) Determine the monitoring results of sturgeons and dolphins based on the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistical results of sturgeons and dolphins.

[0148] This embodiment provides an intelligent full-time sturgeon and dolphin monitoring system, such as Figure 6 As shown, the system includes:

[0149] The video stream preprocessing module 601 is used to obtain video stream data of sturgeons and dolphins in the target water area, perform video stream preprocessing on the video stream data of sturgeons and dolphins, and obtain fluid video images of sturgeons and dolphins.

[0150] The target detection module 602 is used to input the fluid video images of sturgeons and dolphins into the target detection model of sturgeons and dolphins to obtain target detection results of sturgeons and dolphins; wherein the target detection model of sturgeons and dolphins includes a YOLOv10 layer, a flattening layer, and a DeTr layer.

[0151] The tracking module 603 is configured to track the sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain the sturgeon and dolphin target tracking results.

[0152] The statistics module 604 is used to count the number of sturgeons and dolphins based on the target detection results of sturgeons and dolphins, and obtain the target statistics results of sturgeons and dolphins.

[0153] The determination module 605 is configured to determine the monitoring results of sturgeons and dolphins based on the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistical results of sturgeons and dolphins.

[0154] In some optional implementations, the video stream preprocessing module 601 includes:

[0155] The fluidization processing unit is used to obtain the video stream data of sturgeons and dolphins in the target water area, perform fluidization processing on the video stream data of sturgeons and dolphins, and obtain initial video images of sturgeons and dolphins.

[0156] The deblurring processing unit is used to perform image deblurring processing based on the initial video images of sturgeons and dolphins by using the adversarial generative model to obtain fluid video images of sturgeons and dolphins.

[0157] In some optional implementations, the target detection module 602 includes:

[0158] The feature extraction unit is used to extract features based on the fluid video images of sturgeons and dolphins using the YOLOv10 layer to obtain the resulting feature map.

[0159] The conversion unit is used to convert the resulting feature map into sequence data using the flatten layer.

[0160] The target detection unit is used to perform target detection based on the sequence data using the DeTr layer to obtain target detection results for sturgeons and dolphins.

[0161] In some optional implementations, the tracking module 603 includes:

[0162] The determination unit is used to determine the predicted positions of various types of sturgeons and dolphins by using a target tracking algorithm based on the target detection results of sturgeons and dolphins.

[0163] The tracking unit is used to track the sturgeons and dolphins of each category based on the predicted positions of the sturgeons and dolphins of each category using a retractable rotating multi-mode camera to obtain target tracking results of the sturgeons and dolphins.

[0164] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0165] The intelligent full-time sturgeon and dolphin monitoring system in this embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0166] The embodiment of the present invention also provides a computer device having the above Figure 6 The intelligent full-time sturgeon and dolphin monitoring system shown.

[0167] See also Figure 7 , Figure 7 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of a GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7A processor 10 is taken as an example.

[0168] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0169] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0170] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0171] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0172] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The bus connection is taken as an example.

[0173] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0174] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0175] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0176] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An intelligent full-time sturgeon and dolphin monitoring method, characterized in that: The method comprises: Obtaining video stream data of sturgeons and dolphins in the target waters, performing video stream preprocessing on the video stream data of sturgeons and dolphins, and obtaining fluid video images of the sturgeons and dolphins; Inputting the sturgeon and dolphin fluid video images into a target detection model for sturgeons and dolphins to obtain target detection results for sturgeons and dolphins; wherein the target detection model for sturgeons and dolphins includes a YOLOv10 layer, a flattening layer, and a DeTr layer; Tracking the sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain sturgeon and dolphin target tracking results; performing a number count of sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain a target statistical result of sturgeons and dolphins; The monitoring results of sturgeons and dolphins are determined based on the target detection results of the sturgeons and dolphins, the target tracking results of the sturgeons and dolphins, and the target statistical results of the sturgeons and dolphins.

2. The method according to claim 1, characterized in that The video stream preprocessing of the video stream data of the sturgeon and dolphin to obtain fluid video images of the sturgeon and dolphin includes: Obtaining video stream data of sturgeons and dolphins in the target waters, and performing fluidization processing on the video stream data of sturgeons and dolphins to obtain initial video images of sturgeons and dolphins; Based on the initial video images of sturgeons and dolphins, an adversarial generative model is used to perform image deblurring processing to obtain fluid video images of the sturgeons and dolphins.

3. The method according to claim 1, characterized in that The step of inputting the sturgeon and dolphin fluid video images into a sturgeon and dolphin target detection model to obtain sturgeon and dolphin target detection results includes: Based on the fluid video images of the sturgeon and dolphin, feature extraction is performed using the YOLOv10 layer to obtain a result feature map; Converting the resulting feature map into sequence data using a flattening layer; Based on the sequence data, target detection is performed using the DeTr layer to obtain target detection results for the sturgeon and dolphin.

4. The method according to claim 1, wherein Before inputting the sturgeon and dolphin fluid video images into the sturgeon and dolphin target detection model to obtain the sturgeon and dolphin target detection results, the method further includes: Target samples of sturgeons and dolphins are obtained, and an initial target detection model is trained using the target samples of sturgeons and dolphins to obtain the target detection model of sturgeons and dolphins.

5. The method according to claim 4, characterized in that The method of training an initial target detection model using the target samples of sturgeons and dolphins to obtain the target detection model of sturgeons and dolphins includes: Inputting the target samples of sturgeons and dolphins into the initial target detection model to obtain target prediction data of sturgeons and dolphins; Calculating target positioning loss, sequence loss, and target bounding box sharpness loss based on the target prediction data of the sturgeon and dolphin and the target annotation data of the sturgeon and dolphin; The initial target detection model is iteratively adjusted using the target positioning loss value, the sequence loss value, and the target bounding box sharpness loss value to obtain the target detection model for sturgeons and dolphins.

6. The method according to claim 1, wherein The step of tracking the sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain the sturgeon and dolphin target tracking results includes: Based on the target detection results of the sturgeons and dolphins, a target tracking algorithm is used to determine the predicted positions of the sturgeons and dolphins of each category; Based on the predicted positions of the various categories of sturgeons and dolphins, a retractable rotating multi-mode camera is used to track the targets of the various categories of sturgeons and dolphins to obtain target tracking results of the sturgeons and dolphins.

7. An intelligent full-time sturgeon and dolphin monitoring system, characterized in that: The system comprises: A video stream preprocessing module is used to obtain video stream data of sturgeons and dolphins in the target waters, perform video stream preprocessing on the video stream data of sturgeons and dolphins, and obtain fluid video images of sturgeons and dolphins; a target detection module, configured to input the sturgeon and dolphin fluid video images into a target detection model for sturgeons and dolphins to obtain target detection results for sturgeons and dolphins; wherein the target detection model for sturgeons and dolphins comprises a YOLOv10 layer, a flattening layer, and a DeTr layer; A tracking module, configured to track the sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain sturgeon and dolphin target tracking results; a statistical module for performing a number count on the sturgeon and dolphin targets based on the sturgeon and dolphin target detection results to obtain a target statistical result of the sturgeon and dolphin targets; A determination module is used to determine the monitoring results of sturgeons and dolphins based on the target detection results of sturgeons and dolphins, the target tracking results of sturgeons and dolphins, and the target statistical results of sturgeons and dolphins.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the intelligent full-time sturgeon and dolphin monitoring method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the intelligent full-time sturgeon and dolphin monitoring method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the intelligent full-time sturgeon and dolphin monitoring method according to any one of claims 1 to 6.

Images