Multi-dimensional feature cooperative fish feeding intensity quantification identification method and model

Abstract

The present application relates to the technical field of image recognition, and more particularly to a multi-dimensional feature coordination fish feeding intensity quantification recognition method and model, which collects video data of fish feeding, extracts distance information between fish individuals and a baiting machine in a first image, extracts fish individual motion feature information in a second image, extracts fish target part feature information of fish individuals in a third image, and extracts water surface bait quantity information and water surface ripple information in a fourth image; determines fish feeding intensity recognition results according to the distance information, the fish individual motion feature information, the fish target part feature information, the water surface bait quantity information and the water surface ripple information of each fish individual; and aims to solve the problem of how to combine fish individual motion feature and spatial feature information to recognize fish feeding intensity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a multi-dimensional feature coordination fish feeding intensity quantification recognition method and model. BACKGROUND

[0002] With the continuous development of the aquaculture industry, intelligent and fine management of aquaculture has become the main trend of future development. Precise assessment of fish feeding intensity plays an increasingly important role in improving production efficiency, optimizing aquaculture management, reducing feed waste, and increasing economic benefits of aquaculture. However, as the scale of aquaculture continues to expand and the water environment becomes increasingly complex, traditional methods of assessing fish feeding intensity based on experience have gradually shown their limitations. They often lack sufficient accuracy in dealing with complex and dynamic water environments and when facing fish group occlusion.

[0003] In this context, the development of deep learning and computer vision technology provides a new technical path for intelligent assessment of fish feeding intensity. Currently, methods for assessing fish feeding behavior can be divided into two categories: overall image-based assessment methods and individual feature-based assessment methods. Overall image-based methods use deep learning models such as convolutional neural networks to extract global feature information from captured fish feeding images, achieving automated assessment of feeding intensity. They exhibit high robustness in handling complex environmental disturbances (such as water flow, bubbles, and changes in lighting), and by introducing attention mechanisms focused on fish group aggregation behavior, they improve the accuracy of feeding behavior recognition. However, the disadvantage of such methods is that they ignore the motion characteristics of individual fish feeding behavior and lack modeling of time dimension information.

[0004] Individual feature-based methods extract feature values such as fish speed, turning angle, and eccentricity from the established motion trajectory of individual fish, calculate the joint statistics of turning angle and bending tail angle to assess changes in behavior characteristics, and use an improved kinetic model to quantify swimming intensity in combination with the degree of change and turning speed. By detecting the stress behavior of fish under hunger stress, the method dynamically assesses fish feeding behavior. Its advantage is that it can effectively combine time series information to more accurately capture the dynamic behavior of fish. However, individual-based methods are mainly aimed at low-density fish group environments and focus on the motion state of individual fish. Therefore, in high-density environments, especially during the feeding process, the occlusion and overlap between fish bodies significantly reduce the tracking performance of the algorithm, leading to data loss and frequent switching of individual identities, thereby affecting the accuracy and reliability of feeding behavior assessment.

[0005] Therefore, a multi-dimensional feature cooperative fish school feeding intensity quantification recognition method combining fish individual motion characteristics and spatial feature information is needed, which can capture the motion characteristics of fish individual feeding behavior while ensuring the accuracy and reliability of feeding behavior evaluation in high fish school density environment.

[0006] The above description is only used to assist in understanding the technical solutions of the present application and does not mean that the above description is prior art. SUMMARY

[0007] The main purpose of the present application is to provide a multi-dimensional feature cooperative fish school feeding intensity quantification recognition method, which aims to solve the problem of how to combine fish individual motion characteristics and spatial feature information to recognize fish school feeding intensity.

[0008] To achieve the above purpose, the present application provides a multi-dimensional feature cooperative fish school feeding intensity quantification recognition method, which comprises:

[0009] Collecting video data of fish school feeding, and extracting a first image containing fish individuals and a bait feeder, a second image containing fish motion trajectories, a third image containing fish target parts, and a fourth image containing bait and water ripples in the video data;

[0010] Extracting distance information between the fish individuals and the bait feeder in the first image, extracting fish individual motion characteristic information in the second image, extracting fish individual target part feature information in the third image, and extracting water bait quantity information and water ripple information in the fourth image;

[0011] According to the distance information, the fish individual motion characteristic information, the fish target part feature information, the water bait quantity information and the water ripple information of each fish individual, the fish school feeding intensity recognition result is determined.

[0012] Optionally, the step of extracting the distance information between the fish individuals and the bait feeder in the first image comprises:

[0013] Inputting the first image into a Mamba-YOLO target detection model to frame the fish individual frame and the bait feeder frame in the first image through the Mamba-YOLO target detection model;

[0014] Determining the target fish individual frame with a confidence greater than a pre-set confidence threshold in each fish individual frame, and determining the pixel distance between the target fish individual frame and the bait feeder frame;

[0015] A conversion coefficient between the pixel distance and the actual distance is determined through calibration, and an actual distance is calculated according to the conversion coefficient and the pixel distance as distance information between the fish individual and the feeder.

[0016] Optionally, the second image includes a plurality of continuous RGB image frames, and the step of extracting the fish individual motion feature information in the second image includes:

[0017] The plurality of continuous RGB image frames are processed by using an optical flow method;

[0018] Pixel displacement information between adjacent RGB image frames is calculated, and motion direction information and speed component information of the fish individual are calculated according to the pixel displacement information;

[0019] Turning angle information of the fish individual is determined according to the motion direction information calculated between a plurality of adjacent RGB image frames, and acceleration information of the fish individual is calculated according to the speed component information calculated between a plurality of adjacent RGB image frames;

[0020] The speed component information, the turning angle information and the acceleration information are determined as the individual motion feature information.

[0021] Optionally, the third image includes a plurality of continuous RGB image frames, and the step of extracting the fish individual target part feature information in the third image includes:

[0022] Fish key point information in the RGB image frame is detected by using a high-resolution network, and the key point information in the RGB image frame is tracked and labeled by individual identification technology, wherein the fish key point information includes spatial coordinates corresponding to eye parts, tail parts, upper fin parts and lower fin parts of a fish individual;

[0023] Target key point information higher than a confidence threshold value in each key point information is determined;

[0024] Geometric information between target key point information is determined based on a preset morphological constraint of a fish body, and the geometric information is determined as the fish individual target part feature information.

[0025] Optionally, the step of extracting the water surface bait quantity information and the water surface ripple information in the fourth image includes:

[0026] The fourth image is preprocessed to reduce random noise in the fourth image and equalize brightness;

[0027] The DINO target detection algorithm based on the DETR framework is used to locate the bait residue area on the water surface in the preprocessed fourth image, and the water surface bait quantity information in the bait residue area is extracted.

[0028] In addition, the wave crest position and wavelength distribution representing the water surface ripples in the fourth image are extracted by image spectrum analysis technology, and the wave crest position and the wavelength distribution are analyzed by time series technology to determine the water surface ripple information.

[0029] Optionally, the step of determining the fish school feeding intensity recognition result according to the distance information, the fish individual motion feature information, the fish target part feature information, the water surface bait quantity information and the water surface ripple information comprises:

[0030] The distance information, the fish individual motion feature information, the fish target part feature information, the water surface bait quantity information and the water surface ripple information are input into a fish school feeding intensity recognition and quantification model, wherein the fish school feeding intensity recognition and quantification model comprises a space-time feature fusion module and a self-attention mechanism, and the self-attention mechanism is used to extract global and local features of each modality data and assign weights.

[0031] The fish school feeding intensity recognition result output by the fish school feeding intensity recognition and quantification model is obtained.

[0032] Optionally, the fish school feeding intensity recognition result comprises at least one of feeding frequency, feeding intensity and target feeding strategy, wherein the target feeding strategy comprises at least one of feeding amount, feeding time and feeding frequency.

[0033] In addition, to achieve the above-mentioned purpose, the present application also provides a fish school feeding intensity recognition model, which comprises:

[0034] An image and set information extraction module is configured to extract a first image containing fish individuals and baiting machines and a second image containing fish motion trajectories in video data, extract distance information between the fish individuals and the baiting machines in the first image, and extract fish individual motion feature information in the second image;

[0035] A fish individual feature and water surface information extraction module is configured to extract a third image containing fish target parts and a fourth image containing bait and water surface ripples, extract fish target part feature information of fish individuals in the third image, and extract water surface bait quantity information and water surface ripple information in the fourth image;

[0036] The fish feeding intensity recognition and quantification module is configured to determine a fish school feeding intensity recognition result according to the distance information, the fish individual motion feature information, the fish target part feature information, the water surface bait quantity information, and the water surface ripple information.

[0037] The present application has at least the following effects:

[0038] By fusing multi-dimensional information such as the motion features, spatial features, and water surface environment features of fish individuals to recognize the fish school feeding intensity, the present application realizes accurate recognition and quantification of the fish school feeding intensity, and provides real-time decision support for bait feeding, so as to dynamically adjust the feeding amount, feeding time, and feeding frequency, thereby improving the feeding efficiency, avoiding waste, and ensuring that the fish school feeding demand is accurately met. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 FIG. 1 is a flowchart of a multi-dimensional feature cooperative fish school feeding intensity quantification recognition method according to a first embodiment of the present application;

[0040] Figure 2 FIG. 5 is an architecture diagram of a fish school feeding intensity recognition model according to an embodiment of the present application;

[0041] Figure 3 FIG. 6 is an architecture diagram of a fish school feeding intensity recognition system according to an embodiment of the present application.

[0042] The implementation, functional features, and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0043] In order to better understand the above technical solutions, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. Embodiment 1

[0044] Reference Figure 1 In the present embodiment, the multi-dimensional feature cooperative fish school feeding intensity quantification recognition method includes the following steps:

[0045] Step S10, video data of fish feeding is collected, and first images containing fish individuals and bait feeding machines in the video data, second images containing fish motion trajectories, third images containing fish target parts, and fourth images containing baits and water surface ripples are extracted, respectively;

[0046] In the embodiment, the video data of fish feeding is collected by the camera, and the first image containing fish individuals and the bait feeder, the second image containing fish movement trajectory, the third image containing fish target part, and the fourth image containing bait and water surface ripples are extracted from the video data.

[0047] It should be noted that the first image to the fourth image are extracted from the video data by different rules. The video data is divided by frame as a unit, and the target image in each frame image meeting the rule is identified based on different extraction rules. The image frames meeting the preset rule in the video data are extracted and classified into the image type to which they belong. For example, the preset fish individual and bait feeder identification rule extracts the screen containing fish individuals and bait feeders from the video data as the first image, and the rest of the images are the same.

[0048] The second image is an image containing fish movement trajectory. The fish movement trajectory can be determined by at least two adjacent image frames containing fish individuals, that is, the position change of a fish between consecutive screen frames is regarded as its movement trajectory.

[0049] The third image is an image containing fish target part. The system can identify the image by pre-setting a plurality of reference detection points corresponding to the key parts of the fish of this type, and the system can capture the image frame containing the reference detection point from the video data as the third image containing the fish target part.

[0050] The fourth image is an image containing bait and water surface ripples. Similarly, the shape of the bait and water surface ripples can be pre-set as the identification rule, and the system can capture the image frame containing the bait and water surface ripples from the video data as the fourth image containing the bait and water surface ripples.

[0051] It can be understood that the same image frame in the video data can be classified into multiple image types, and the difference is that the information extracted from the image frame is different. For example, a certain frame image in the video data contains fish individuals, bait feeders, bait and water surface ripples, and the key parts on the fish individuals can be clearly identified, so it can be used as the first image containing fish individuals and bait feeders, the third image containing fish target parts, and the fourth image containing bait and water surface ripples.

[0052] In step S20, the distance information between the fish individuals and the bait feeder in the first image is extracted, the fish individual movement feature information in the second image is extracted, the fish individual target part feature information in the third image is extracted, and the water surface bait quantity information and the water surface ripple information in the fourth image are extracted.

[0053] In the embodiment, after the image classification is completed, target information in each type of image is extracted based on the information extraction rule corresponding to each image type, so as to determine the distance information between the fish individual and the feeder, the fish individual motion characteristic information, the fish target part characteristic information, the water surface bait quantity information and the water surface wave information.

[0054] In step S30, the fish school feeding intensity recognition result is determined according to the distance information, the fish individual motion characteristic information, the fish target part characteristic information, the water surface bait quantity information and the water surface wave information of each fish individual.

[0055] In the embodiment, the feeding desire of the fish school is quantified by the distance information between the fish individual and the feeder. The closer the distance between the fish individual and the feeder, the stronger the feeding desire of the fish individual. The closer the comprehensive distance between the fish school composed of the whole fish individuals and the feeder, the stronger the feeding desire of the fish school, and the feeding intensity is increased accordingly.

[0056] The fish individual motion characteristic information reflects the activity of the fish individual. The richer the fish individual motion characteristic information, the stronger the activity, and the corresponding feeding intensity is increased accordingly.

[0057] The water surface bait quantity information reflects whether the current bait quantity is sufficient. The more the bait on the water surface, the greater the degree that the bait quantity exceeds the feeding quantity of the fish school, and the weaker the feeding intensity. Conversely, the stronger the feeding intensity.

[0058] The water surface wave information reflects the activity degree of the fish school when feeding. The fish school will form waves on the water surface when feeding. The richer the water surface wave information, the greater the activity of the fish school when feeding, and the stronger the corresponding feeding intensity. Conversely, the weaker the feeding intensity.

[0059] Therefore, in the embodiment, after the above information is obtained, the feeding intensity of the fish school is comprehensively recognized according to the distance information between the fish individual and the feeder, the fish individual motion characteristic information, the fish target part characteristic information, the water surface bait quantity information and the water surface wave information, and the fish school feeding intensity recognition result is obtained.

[0060] Optionally, the fish school feeding intensity recognition result includes at least one of feeding frequency, feeding intensity and target feeding strategy. Based on the above information, the feeding frequency and the feeding intensity of the fish school are calculated, and the target feeding strategy including at least one of suitable feeding quantity, feeding time and feeding frequency is formulated according to the feeding frequency and the feeding intensity, so as to improve the feeding efficiency, avoid waste and ensure that the feeding demand of the fish school is accurately met.

[0061] In the technical scheme provided in the embodiment, the fish school feeding intensity is identified by fusing multi-dimensional information such as the motion characteristics, spatial characteristics and water surface environment characteristics of fish individuals, the accurate identification and quantification of the fish school feeding intensity are realized, and real-time decision support is provided for bait feeding, so that the feeding amount, feeding time and feeding frequency are dynamically adjusted, the feeding efficiency is improved, waste is avoided, and the feeding demand of the fish school is accurately met. Embodiment 2

[0062] Based on the first embodiment, in the embodiment, the step S20 includes the following steps:

[0063] In step S21, the first image is input into a Mamba-YOLO target detection model to frame the fish individual frame and the bait feeder frame in the first image by the Mamba-YOLO target detection model.

[0064] In step S22, the target fish individual frame with a confidence greater than a preset confidence threshold is determined in each fish individual frame, and the pixel distance between the target fish individual frame and the bait feeder frame is determined.

[0065] In step S23, a conversion coefficient between the pixel distance and the actual distance is determined by calibration, and the actual distance is calculated according to the conversion coefficient and the pixel distance as the distance information between the fish individual and the bait feeder.

[0066] Specifically, in the embodiment, the RGB image of the fish feeding scene is collected and input into the Mamba-YOLO target detection model to detect the fish individual and the bait feeder in the image, and output the coordinate information and confidence score of the detection frame. According to the center point coordinates of the detection frame, the pixel distance between the fish individual and the bait feeder is calculated by using the Euclidean distance formula; through the calibration experiment, the conversion relationship between the pixel distance and the actual distance is established, and the proportion coefficient is obtained to convert the pixel distance into the actual distance.

[0067] Optionally, the detection results of multiple image frames can be weighted and averaged to optimize the distance estimation accuracy of the fish individual and the bait feeder, and finally output the estimation result. Embodiment 3

[0068] Based on the first embodiment, in the embodiment, the second image includes a plurality of continuous RGB image frames, and the step S20 includes the following steps:

[0069] In step S24, the optical flow method is used to process a plurality of continuous RGB image frames.

[0070] Step S25, calculate the pixel displacement information between adjacent RGB image frames, and calculate the motion direction information and speed component information of the fish individual according to the pixel displacement information;

[0071] Step S26, determine the turning angle information of the fish individual according to the motion direction information calculated between multiple adjacent RGB image frames, and calculate the acceleration information of the fish individual according to the speed component information calculated between multiple adjacent RGB image frames;

[0072] Step S27, determine the speed component information, the turning angle information and the acceleration information as the individual motion characteristic information.

[0073] In this embodiment, the dense optical flow method is used to estimate the motion vector of each pixel point between adjacent image frames, to obtain the displacement amount of each pixel between two consecutive frames and generate pixel-level optical flow field information. The displacement amount contains the displacement information in the horizontal direction (x-axis) and the vertical direction (y-axis), which can reflect the motion direction and speed of the fish. Based on the pixel motion vector in the optical flow field, the motion vector of the fish individual region is integrated and analyzed by the weighted aggregation method, to calculate the instantaneous speed of the fish individual at a specific time point. Subsequently, by comparing the change trend of the speed value in the consecutive image frames, the acceleration of the fish individual is further calculated, so as to evaluate whether the fish is in the dynamic state of acceleration or deceleration.

[0074] Based on the motion vector of each pixel, the direction change of the fish individual in consecutive image frames is calculated. For each pair of adjacent image frames, the change amplitude of the direction of the pixel motion vector is analyzed, and if the fish individual turns or makes a sharp turning action, the direction of the pixel motion vector will change significantly. By calculating the direction change amount of each pixel in the adjacent two image frames, the turning angle information of the fish individual can be obtained, which is used to quantify the directional change characteristics of the fish in the feeding behavior. This information can accurately reflect the turning action of the fish and the related behavior pattern, and provide more rich dynamic characteristic dimensions for the overall motion behavior analysis of the fish school.

[0075] Combined with the instantaneous speed, acceleration and turning angle information of the fish individual, the motion state of the fish is comprehensively evaluated, and the extracted motion characteristic data is used as important input data for the subsequent fish school feeding intensity recognition model. The above method provides a data basis for real-time quantitative analysis of fish school feeding behavior and optimization of feeding strategy by extracting the motion information of the fish individual. Embodiment 4

[0076] Based on the first embodiment, in this embodiment, the third image includes multiple consecutive RGB image frames, and the step of extracting the fish target part characteristic information of the fish individual in the third image in step S20 includes:

[0077] In step S28, a high-resolution network is used to detect fish key point information in the RGB image frame, and individual identification technology is used to track and label the key point information in the RGB image frame. The fish key point information includes spatial coordinates of eye, tail, upper fin and lower fin of a fish individual.

[0078] In step S29, target key point information higher than a confidence threshold is determined from each key point information.

[0079] In step S210, geometric information between target key point information is determined based on a preset morphological constraint of a fish body, and the geometric information is determined as fish target part feature information.

[0080] In this embodiment, a high-resolution network (HRNet) is used to detect fish key points in the RGB image frame collected by the camera device, so as to accurately extract feature point information of a fish individual, including spatial coordinates of the eye, tail, upper fin and lower fin. The extraction of these key point information provides basic data support for the analysis of fish individual behavior and group interaction characteristics.

[0081] In the key point detection process, a tracking algorithm based on individual identification is introduced in this embodiment, which is used to real-time label and dynamically track the key point information of the fish group. Through this method, the spatial consistency of the key points can be maintained in the time sequence, and the features of each fish individual can be effectively distinguished, so as to ensure the accuracy and robustness of the detection result.

[0082] In order to further improve the accuracy of key point detection, a screening mechanism based on confidence score is designed in this embodiment. By setting a confidence score threshold, low-confidence key points that may exist in the detection are filtered. An interpolation algorithm and a posture optimization technology are used to correct the spatial position of the filtered key points, so as to reduce the detection error caused by the limitation of image resolution or the shielding phenomenon, thereby improving the reliability of the key point coordinate positioning.

[0083] Based on the detected key point information, the spatial distribution characteristics and geometric relationship of the key points are further analyzed in this embodiment. Combined with the morphological constraint conditions of the fish body (fish length-width ratio and fin position relationship), the shielding degree between fish individuals is quantitatively evaluated. Through this shielding analysis method, the relative position relationship and overlapping area of individuals in the fish group can be accurately determined, which lays a data foundation for the subsequent dynamic analysis and recognition of the feeding behavior pattern of the fish group.

[0084] Finally, complete key point detection and shielding analysis information is generated, and the information is used as an important input feature of the subsequent feeding intensity recognition model. Embodiment 5

[0085] Based on the first embodiment, in this embodiment, in step S20, the step of extracting the water surface bait quantity information and the water surface ripple information in the fourth image includes:

[0086] Step S211, pre-processing the fourth image to reduce random noise in the fourth image and balance the brightness;

[0087] Step S212, using a DINO target detection algorithm based on a DETR framework, locating the bait residue area on the water surface in the pre-processed fourth image, and extracting the water surface bait quantity information in the bait residue area;

[0088] Step S213, and extracting the wave peak position and wavelength distribution representing the water surface ripple in the fourth image by image spectrum analysis technology, and determining the water surface ripple information by analyzing the wave peak position and the wavelength distribution using time series technology.

[0089] In this embodiment, data preprocessing is performed on the collected RGB image frames. The preprocessing stage includes image denoising processing to effectively reduce the interference of random noise on the subsequent detection algorithm, and compensates for the image brightness unevenness caused by complex and variable lighting conditions through the light correction method, thereby significantly improving the visual quality of the image and the accuracy of the detection.

[0090] And a DINO target detection algorithm pre-trained by a special water surface bait residue detection data set is used to analyze the pre-processed image frames. The DINO model learns the feature representation ability of complex background and small targets, realizes the accurate positioning of the water surface bait residue area, and extracts the quantity, morphology and spatial distribution characteristics of the bait residue. During the pre-training process, the model optimizes the parameter configuration for specific water surface feeding scenes, enhancing its robustness and adaptability to complex environmental background and small targets.

[0091] To obtain the dynamic characteristics of the water surface environment, in this embodiment, combined with image spectrum analysis and time series analysis technology, multi-dimensional feature information of water surface ripple is extracted from consecutive frames, including wave peak position, wavelength distribution, fluctuation amplitude and change rate, etc. By capturing the key characteristics of the dynamic changes of the ripples, the present application can effectively represent the transient characteristics of the water surface fluctuation, thereby providing more detailed environmental dynamic analysis results.

[0092] To further improve the reliability of the detection and feature extraction results, in this embodiment, an adaptive confidence threshold mechanism is introduced, which dynamically adjusts the filtering threshold of the confidence score according to the specific detection scene, thereby eliminating the interference of low-confidence detection results. Post-processing optimization techniques are used to perform spatial smoothing and boundary accuracy adjustment on the detection output, ensuring that the final output of the bait residue quantity and the ripple characteristics has higher accuracy and robustness.

[0093] Finally, through the above steps, comprehensive detection of the bait residual area on the water surface and accurate extraction of water surface ripple characteristics are achieved. Example 6

[0094] Based on the first embodiment, in this embodiment, step S30 includes:

[0095] Step S31: Inputting the distance information, the individual fish motion characteristic information, the target fish part characteristic information, the water surface bait quantity information, and the water surface ripple information into a fish feeding intensity identification and quantification model, wherein the fish feeding intensity identification and quantification model includes a spatiotemporal feature fusion module and a self-attention mechanism, and the self-attention mechanism is used to extract global and local features of each modal data and assign weights;

[0096] Step S32: obtaining the fish feeding intensity identification result output by the fish feeding intensity identification and quantification model.

[0097] In this embodiment, the integrated multi-dimensional data is input into the fish feeding intensity identification and quantification model. The model includes a spatiotemporal feature fusion module and a self-attention mechanism, which are used to extract global and local information of various features and highlight key features through weight distribution. By training on the fish feeding behavior dataset, the loss function of the model is optimized to ensure its accuracy in modeling the correlation of multimodal features and predicting feeding intensity. During the real-time inference process, this embodiment analyzes the dynamic characteristics of the fish feeding behavior based on the dynamic change information of continuous image frames and combines the temporal attention mechanism to output the quantitative results of the fish feeding intensity. Ultimately, the quantitative results are presented in the form of feeding frequency and intensity level, and can be linked to the feeding system to dynamically adjust the feeding strategy according to the feeding status of the fish, thereby achieving precise feeding and resource optimization.

[0098] In addition, as an implementation solution, refer to Figure 2 This embodiment also proposes a fish feeding intensity recognition model, which includes:

[0099] An image and collection information extraction module 100 is configured to extract a first image containing both individual fish and a baitcaster and a second image containing a fish motion trajectory from the video data, extract distance information between the individual fish and the baitcaster in the first image, and extract motion feature information of the individual fish in the second image;

[0100] The fish individual feature and water surface information extraction module 200 is configured to extract a third image containing a fish target part of a fish individual and a fourth image containing a bait and water surface ripples, and extract fish target part feature information of the fish individual in the third image, and water surface bait quantity information and water surface ripple information in the fourth image.

[0101] The fish feeding intensity recognition and quantification module 300 is configured to determine a fish school feeding intensity recognition result according to the distance information, the fish individual motion feature information, the fish target part feature information, the water surface bait quantity information, and the water surface ripple information.

[0102] As an implementation solution, Figure 3 The fish school feeding intensity recognition system related to the embodiment of the present application is shown in a schematic diagram of a hardware running environment architecture.

[0103] As Figure 3 shown, the fish school feeding intensity recognition system can include a processor 1001 such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. The communication bus 1002 is configured to realize the connection and communication between the components. The user interface 1003 can include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 can further include a standard wired interface and a wireless interface. The network interface 1004 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory) such as a magnetic disk memory. The memory 1005 can optionally be a storage device independent of the aforementioned processor 1001.

[0104] Those skilled in the art can understand, Figure 3 The fish school feeding intensity recognition system architecture shown in the foregoing embodiment of the present application does not constitute a limitation on the fish school feeding intensity recognition system, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0105] As Figure 3 shown, the memory 1005 as a storage medium can include an operating system, a network communication module, a user interface module, and a fish school feeding intensity recognition program. The operating system is a program for managing and controlling the hardware and software resources of the fish school feeding intensity recognition system, and the running of the fish school feeding intensity recognition program and other software or programs.

[0106] In Figure 3The user interface 1003 is mainly used for connecting the terminal and communicating data with the terminal; the network interface 1004 is mainly used for the background server and communicating data with the background server; the processor 1001 can be used to call the fish school feeding intensity identification program stored in the memory 1005.

[0107] In the embodiment, the fish school feeding intensity identification system comprises a memory 1005, a processor 1001 and a fish school feeding intensity identification program stored in the memory and executable on the processor.

[0108] When the processor 1001 calls the fish school feeding intensity identification program stored in the memory 1005, the following operations are performed:

[0109] Collecting video data of fish school feeding, and extracting a first image containing fish individuals and a bait feeder, a second image containing fish movement trajectories, a third image containing fish target parts and a fourth image containing bait and water ripples in the video data respectively;

[0110] Extracting distance information between the fish individuals and the bait feeder in the first image, fish individual movement feature information in the second image, fish individual target part feature information in the third image, and water bait quantity information and water ripple information in the fourth image;

[0111] According to the distance information, the fish individual movement feature information, the fish individual target part feature information, the water bait quantity information and the water ripple information of each fish individual, a fish school feeding intensity identification result is determined.

[0112] When the processor 1001 calls the fish school feeding intensity identification program stored in the memory 1005, the following operations are performed:

[0113] Inputting the first image into the Mamba-YOLO target detection model to frame the fish individual frame and the bait feeder frame in the first image through the Mamba-YOLO target detection model;

[0114] Determining a target fish individual frame with a confidence greater than a preset confidence threshold in each fish individual frame, and determining the pixel distance between the target fish individual frame and the bait feeder frame;

[0115] Determining the conversion coefficient between the pixel distance and the actual distance through calibration, and calculating the actual distance according to the conversion coefficient and the pixel distance as the distance information between the fish individual and the bait feeder.

[0116] When the processor 1001 invokes the fish school feeding intensity recognition program stored in the memory 1005, the following operations are performed:

[0117] A plurality of consecutive RGB image frames are processed using an optical flow method;

[0118] Pixel displacement information between adjacent RGB image frames is calculated, and the motion direction information and the velocity component information of the fish individuals are calculated based on the pixel displacement information;

[0119] The turning angle information of the fish individuals is determined based on the motion direction information calculated between a plurality of adjacent RGB image frames, and the acceleration information of the fish individuals is calculated based on the velocity component information calculated between a plurality of adjacent RGB image frames;

[0120] The velocity component information, the turning angle information, and the acceleration information are determined as the individual motion feature information.

[0121] When the processor 1001 invokes the fish school feeding intensity recognition program stored in the memory 1005, the following operations are performed:

[0122] Fish key point information in the RGB image frame is detected using a high-resolution network, and the key point information in the RGB image frame is tracked and labeled through individual identification technology, wherein the fish key point information includes spatial coordinates corresponding to the eye, tail, upper fin, and lower fin of the fish individual;

[0123] Target key point information higher than a confidence threshold is determined in each of the key point information;

[0124] Geometric information between target key point information is determined based on a preset morphological constraint of the fish body, and the geometric information is determined as the fish target part feature information.

[0125] When the processor 1001 invokes the fish school feeding intensity recognition program stored in the memory 1005, the following operations are performed:

[0126] The fourth image is preprocessed to reduce random noise in the fourth image and equalize brightness;

[0127] A DINO target detection algorithm based on a DETR framework is used to locate a bait residue area on the water surface in the preprocessed fourth image, and water bait quantity information in the bait residue area is extracted;

[0128] In addition, a wave crest position and a wavelength distribution representing water surface ripples in the fourth image are extracted through image spectrum analysis technology, and the wave crest position and the wavelength distribution are analyzed using time series technology to determine the water surface ripple information.

[0129] When the processor 1001 invokes the fish school feeding intensity recognition program stored in the memory 1005, the following operations are performed:

[0130] The distance information, the fish individual motion feature information, the fish target part feature information, the water surface bait quantity information, and the water surface ripple information are input into a fish school feeding intensity recognition and quantification model, wherein the fish school feeding intensity recognition and quantification model comprises a space-time feature fusion module and a self-attention mechanism, and the self-attention mechanism is used to extract global and local features of each modality data and assign weights.

[0131] The fish school feeding intensity recognition result output by the fish school feeding intensity recognition and quantification model is obtained.

[0132] In addition, those skilled in the art can understand that all or part of the processes in the method for implementing the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program comprises program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the fish school feeding intensity recognition system to implement the process steps of the above-mentioned method embodiments.

[0133] Therefore, the present application also provides a computer-readable storage medium storing a fish school feeding intensity recognition program, wherein the fish school feeding intensity recognition program is executed by a processor to implement the steps of the multi-dimensional feature cooperative fish school feeding intensity quantification and recognition method according to the above-mentioned embodiments.

[0134] The computer-readable storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk, an optical disk, or various computer-readable storage media that can store program codes.

[0135] It should be noted that the storage medium provided by the present application is a storage medium used to implement the method of the present application, and therefore the specific structure and modifications of the storage medium can be understood based on the method introduced in the present application, and therefore will not be described here. Any storage medium used in the method of the present application belongs to the scope of protection of the present application.

[0136] Those skilled in the art will appreciate that embodiments of the application can be readily used as a method, a system or a computer program product. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0137] The application is described in relation to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It is understood that each flow and / or block in the flow diagrams and / or block diagrams, and combinations of flows and / or blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the function specified by the flow or flows and / or block or blocks.

[0138] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams flow or flows and / or block or blocks. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the function specified by the flow or flows and / or block or blocks.

[0139] The computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams flow or flows and / or block or blocks. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the function specified by the flow or flows and / or block or blocks.

[0140] It should be noted that the word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding the citation of elements does not exclude the presence of a plurality of such elements. It is appreciated that the application can be implemented by means of both hardware and software, and that the means recited in any claim are intended to cover both alternatives. The person skilled in the art will readily recognize that the application can be implemented by means of a hardware-only or software-only solution, or by means of a combination of hardware and software. Any reference signs in the claims should not be construed as limiting the scope of the claims. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding the citation of an element does not exclude the presence of a plurality of such elements. It is appreciated that the application can be implemented by means of both hardware and software, and that the means recited in any claim are intended to cover both alternatives. The person skilled in the art will readily recognize that the application can be implemented by means of a hardware-only or software-only solution, or by means of a combination of hardware and software. Any reference signs in the claims should not be construed as limiting the scope of the claims.

[0141] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the preferred embodiments by those of skill in the art once given the benefit of the present disclosure without departing from the spirit and scope of the application. Therefore, it is intended that such additional variations and modifications be included within the scope of the application. The application is to be limited only by the claims.

[0142] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A multi-dimensional feature coordination method for quantifying and identifying the feeding intensity of a fish school, characterized in that, The method comprises the following steps: Collecting video data of fish feeding, and extracting first images containing fish individuals and feeding machines, second images containing fish trajectories, third images containing fish target parts, and fourth images containing baits and water ripples in the video data respectively; Extracting distance information between the fish individuals and the feeding machines in the first images, fish individual motion feature information in the second images, fish individual target part feature information in the third images, and water bait quantity information and water ripple information in the fourth images; Determining fish feeding intensity recognition results according to the distance information, the fish individual motion feature information, the fish individual target part feature information, the water bait quantity information, and the water ripple information of each fish individual; The second image comprises a plurality of continuous RGB image frames, and the step of extracting fish individual motion feature information in the second image comprises: Processing a plurality of continuous RGB image frames by using an optical flow method; Calculating pixel displacement information between adjacent RGB image frames, and calculating motion direction information and velocity component information of the fish individuals according to the pixel displacement information; Determining turning angle information of the fish individuals according to the motion direction information calculated between a plurality of adjacent RGB image frames, and calculating acceleration information of the fish individuals according to the velocity component information calculated between a plurality of adjacent RGB image frames; Determining the velocity component information, the turning angle information, and the acceleration information as the individual motion feature information; The step of extracting water bait quantity information and water ripple information in the fourth image comprises: Preprocessing the fourth image to reduce random noise and balance brightness in the fourth image; Using a DINO target detection algorithm based on a DETR framework to locate bait residue areas on the water surface in the preprocessed fourth image, and extracting water bait quantity information in the bait residue areas; And extracting wave peak positions and wavelength distributions representing water ripples in the fourth image by using an image spectrum analysis technique, and analyzing the wave peak positions and the wavelength distributions by using a time series technique to determine the water ripple information; The step of determining fish feeding intensity recognition results according to the distance information, the fish individual motion feature information, the fish individual target part feature information, the water bait quantity information, and the water ripple information comprises: Inputting the distance information, the fish individual motion feature information, the fish individual target part feature information, the water bait quantity information, and the water ripple information into a fish feeding intensity recognition and quantification model, wherein the fish feeding intensity recognition and quantification model comprises a spatio-temporal feature fusion module and a self-attention mechanism, the self-attention mechanism is used to extract global and local features of each modality data and assign weights; Obtaining fish feeding intensity recognition results output by the fish feeding intensity recognition and quantification model.

2. The method of claim 1, wherein, The step of extracting the distance information between the fish individuals and the feeder in the first image comprises: inputting the first image into a Mamba-YOLO target detection model to frame the fish individual frames and the feeder frame in the first image through the Mamba-YOLO target detection model; determining a target fish individual frame with a confidence greater than a preset confidence threshold in each fish individual frame, and determining the pixel distance between the target fish individual frame and the feeder frame; determining the conversion coefficient between the pixel distance and the actual distance through calibration, and calculating the actual distance as the distance information between the fish individuals and the feeder according to the conversion coefficient and the pixel distance.

3. The method of claim 1, wherein, The third image comprises a plurality of continuous RGB image frames, and the step of extracting the fish target part feature information of the fish individuals in the third image comprises: detecting fish key point information in the RGB image frames using a high-resolution network, and tracking and labeling the key point information in the RGB image frames through individual identification technology, wherein the fish key point information comprises spatial coordinates representing the eyes, tail, upper fin and lower fin of the fish individuals; determining target key point information higher than a confidence threshold in each key point information; determining the geometric information between the target key point information based on the preset morphological constraints of the fish body, and determining the geometric information as the fish target part feature information.

4. The method of claim 1, wherein, The fish school feeding intensity recognition result comprises at least one of feeding frequency, feeding intensity and target feeding strategy, wherein the target feeding strategy comprises at least one of feeding amount, feeding time and feeding frequency.

5. A fish school feeding intensity identification model, used for implementing the multi-dimensional feature coordinated fish school feeding intensity quantitative identification method of any one of claims 1-4, characterized in that, The fish school feeding intensity recognition model comprises: an image and set information extraction module for extracting a first image containing fish individuals and a feeder and a second image containing fish movement trajectories in video data, and extracting distance information between the fish individuals and the feeder in the first image, and extracting fish individual movement feature information in the second image; a fish individual feature and water surface information extraction module for extracting a third image containing fish target parts and a fourth image containing bait and water surface ripples, and extracting fish target part feature information of fish individuals in the third image, and water surface bait quantity information and water surface ripple information in the fourth image; a fish feeding intensity recognition and quantification module for determining fish school feeding intensity recognition results according to the distance information, the fish individual movement feature information, the fish target part feature information, the water surface bait quantity information and the water surface ripple information.

Images