Multi-dimensional feature collaborative quantitative recognition method and model for fish ingestion intensity

Abstract

The invention relates to the technical field of image recognition, in particular to a multi-dimensional feature collaborative fish school ingestion intensity quantitative recognition method and model, and the method comprises the steps: collecting the ingestion video data of a fish school, extracting the distance information between a fish individual and a bait casting machine in a first image, and obtaining the distance information between the fish individual and the bait casting machine; extracting fish individual motion feature information in the second image, extracting fish target part feature information of fish individuals in the third image, and extracting water surface bait quantity information and water surface ripple information in the fourth image; according to the distance information, the fish individual motion characteristic information, the fish target part characteristic information, the water surface bait quantity information and the water surface ripple information of each fish individual, determining a fish school feeding intensity identification result; the objective of the invention is to solve the problem of how to combine fish individual motion features and spatial feature information to identify the ingestion intensity of fish schools.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method and model for quantitatively identifying the feeding intensity of fish schools using multi-dimensional feature collaboration. Background Art

[0002] With the continuous development of the aquaculture industry, intelligent and refined aquaculture has become the main trend of future development. Accurate assessment of fish feeding intensity plays an increasingly important role in improving production efficiency, optimizing aquaculture management, reducing feed waste and increasing the economic benefits of aquaculture. However, with the continuous expansion of aquaculture scale and the increasing complexity of the water environment, the traditional experience-based fish feeding intensity assessment method has gradually revealed its limitations. It usually lacks sufficient accuracy when dealing with complex and dynamic water environments and when facing fish occlusion.

[0003] In this context, the development of deep learning and computer vision technology has provided a new technical path for the intelligent assessment of fish feeding intensity. At present, the evaluation methods of fish feeding behavior are mainly divided into two categories: evaluation methods based on overall images and evaluation methods based on individual features. The overall image-based method uses deep learning models such as convolutional neural networks to extract global feature information from captured fish feeding images to achieve automatic evaluation of feeding intensity. It shows high robustness in dealing with complex environmental interference (such as water flow, bubbles, light changes, etc.), and improves the accuracy of feeding behavior recognition by introducing an attention mechanism that focuses on fish aggregation behavior. However, the shortcomings of this method are that it ignores the motion characteristics of individual fish feeding behavior and lacks modeling of time dimension information.

[0004] The method based on individual characteristics extracts the characteristic values ​​of fish speed, turning angle, eccentricity, etc. from the established motion trajectory of individual fish, calculates the joint statistics of turning angle and tail bending angle to evaluate the changes in behavioral characteristics, and uses the improved kinetic energy model to combine the degree of change and turning speed to quantify the swimming intensity. The stress behavior of fish under hunger stress is detected by swimming intensity, and then the feeding behavior of fish is dynamically evaluated. Its advantage is that it can effectively combine time series information to more accurately capture the dynamic behavior of fish. However, the individual-based method mainly targets low-density fish environments and focuses on the movement state of individual fish. Therefore, in high-density environments, especially during feeding, the occlusion and overlap problems between fish bodies will significantly reduce the tracking performance of the algorithm, resulting in data loss and frequent individual identity switching, thereby affecting the accuracy and reliability of feeding behavior evaluation.

[0005] In view of this, a method for quantitatively identifying fish feeding intensity that combines the multi-dimensional features of individual fish motion characteristics and spatial feature information is needed, which can capture the motion characteristics of individual fish feeding behavior while ensuring the accuracy and reliability of feeding behavior assessment in high fish density environments.

[0006] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0007] The main purpose of this application is to provide a method for quantitatively identifying the feeding intensity of fish schools with coordinated multi-dimensional features, aiming to solve the problem of how to combine the movement characteristics of individual fish with spatial feature information to identify the feeding intensity of fish schools.

[0008] To achieve the above objectives, the present application provides a method for quantitatively identifying the feeding intensity of fish schools using multi-dimensional feature collaboration, the method comprising: Collecting video data of fish feeding, and extracting a first image containing both fish individuals and a baitcasting machine, a second image containing a fish movement track, a third image containing a target part of the fish, and a fourth image containing bait and water surface ripples from the video data; Extracting the distance information between the individual fish and the bait casting machine in the first image, extracting the motion feature information of the individual fish in the second image, extracting the target fish part feature information of the individual fish in the third image, and extracting the water surface bait quantity information and water surface ripple information in the fourth image; The fish feeding intensity recognition result is determined based on the distance information of each individual fish, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information.

[0009] Optionally, the step of extracting the distance information between the individual fish in the first image and the bait caster comprises: Inputting the first image into a Mamba-YOLO target detection model, so as to select a fish individual frame and a bait casting machine frame in the first image through the Mamba-YOLO target detection model; Determine, among the individual fish frames, a target fish frame whose confidence is greater than a preset confidence threshold, and determine a pixel distance between the target fish frame and the bait casting machine frame; The conversion coefficient between the pixel distance and the actual distance is determined by calibration, and the actual distance is calculated based on the conversion coefficient and the pixel distance as the distance information between the individual fish and the bait caster.

[0010] Optionally, the second image includes a plurality of continuous RGB image frames, and the step of extracting the individual fish movement feature information in the second image includes: Processing a plurality of continuous RGB image frames using an optical flow method; Calculating pixel displacement information between adjacent RGB image frames, and calculating movement direction information and velocity component information of the individual fish according to the pixel displacement information; Determine the turning angle information of the individual fish according to the motion direction information calculated between the plurality of adjacent RGB image frames, and calculate the acceleration information of the individual fish according to the velocity component information calculated between the plurality of adjacent RGB image frames; The velocity component information, the rotation angle information and the acceleration information are determined as the individual motion feature information.

[0011] Optionally, the third image includes a plurality of continuous RGB image frames, and the step of extracting characteristic information of target fish parts of individual fish in the third image includes: Using a high-resolution network to detect the key point information of the fish in the RGB image frame, and tracking and marking the key point information in the RGB image frame through individual identification technology, wherein the key point information of the fish includes the spatial coordinates corresponding to the eyes, tail, upper fin and lower fin of the fish individual; Determine target key point information above a confidence threshold among each of the key point information; The geometric information between the target key point information is determined based on the preset morphological constraints of the fish body, and the geometric information is determined as the characteristic information of the target part of the fish.

[0012] Optionally, the step of extracting water surface bait quantity information and water surface ripple information in the fourth image includes: Preprocessing the fourth image to reduce random noise in the fourth image and equalize brightness; Using the DINO target detection algorithm based on the DETR framework, locating the bait residue area on the water surface in the fourth image after preprocessing, and extracting the water surface bait quantity information in the bait residue area; Furthermore, the peak position and wavelength distribution representing the water surface ripples in the fourth image are extracted by image spectrum analysis technology, and the peak position and the wavelength distribution are analyzed by time series technology to determine the water surface ripple information.

[0013] Optionally, the step of determining the result of identifying the feeding intensity of a school of fish according to the distance information, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information comprises: Inputting the distance information, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information into a fish school feeding intensity identification and quantification model, wherein the fish school feeding intensity identification and quantification model includes a spatiotemporal feature fusion module and a self-attention mechanism, and the self-attention mechanism is used to extract the global and local features of each modal data and assign weights; Obtain the fish feeding intensity identification result output by the fish feeding intensity identification and quantification model.

[0014] Optionally, the fish school feeding intensity identification result includes at least one of feeding frequency, feeding intensity and target feeding strategy, wherein the target feeding strategy includes at least one of feeding amount, feeding time and feeding frequency.

[0015] In addition, to achieve the above purpose, the present application also provides a fish feeding intensity recognition model, the fish feeding intensity recognition model comprising: An image and set information extraction module, used to extract a first image containing both individual fish and a baitcasting machine and a second image containing a fish motion track from the video data, and to extract distance information between the individual fish and the baitcasting machine in the first image, and to extract motion feature information of the individual fish in the second image; A fish individual feature and water surface information extraction module, used to extract a third image containing a target fish part and a fourth image containing bait and water surface ripples, and extract fish target part feature information of individual fish in the third image, and water surface bait quantity information and water surface ripple information in the fourth image; The fish feeding intensity identification and quantification module is used to determine the fish feeding intensity identification result based on the distance information, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information.

[0016] This application has at least the following effects: By integrating multi-dimensional information such as the movement characteristics, spatial characteristics and water surface environment characteristics of individual fish, the feeding intensity of fish schools can be identified, and accurate identification and quantification of fish feeding intensity can be achieved. Real-time decision support can be provided for feeding, so as to dynamically adjust the feeding amount, feeding time and feeding frequency, thereby improving feeding efficiency, avoiding waste, and ensuring that the feeding needs of fish schools are accurately met. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of the first embodiment of the method for quantitatively identifying the feeding intensity of fish schools based on multi-dimensional feature collaboration of the present application; Figure 2This is a schematic diagram of the architecture of a fish feeding intensity recognition model according to an embodiment of the present application; Figure 3 Schematic diagram of the architecture of the fish feeding intensity identification system involved in the embodiment of the present application.

[0018] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0019] In order to better understand the above technical solution, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. Example 1

[0020] Reference Figure 1 In this embodiment, the multi-dimensional feature-coordinated quantitative identification method of fish feeding intensity includes the following steps: Step S10, collecting video data of fish feeding, and extracting from the video data a first image containing both fish individuals and a baitcaster, a second image containing a fish movement track, a third image containing a target part of the fish, and a fourth image containing bait and water surface ripples; In this embodiment, video data of fish feeding is collected by a camera, and a first image including individual fish and a bait caster, a second image including the movement trajectory of the fish, a third image including the target part of the fish, and a fourth image including bait and water surface ripples are extracted from the video data.

[0021] It should be noted that the first image to the fourth image are extracted from the video data using different rules respectively, the video data is divided into frames, and the target image that meets the rules in each frame is identified based on different extraction rules. The image frames that meet the preset rules in the video data are extracted and classified into the image types to which they belong. For example, the recognition rules for individual fish and bait casters are preset, and the pictures containing both individual fish and bait casters in the video data are extracted as the first image, and the same applies to the remaining images.

[0022] The second image is an image containing a fish motion track, and the fish motion track can be determined by at least two adjacent image frames containing individual fish, that is, the position change of a fish between consecutive image frames can be regarded as its motion track; The third image is an image containing the target part of the fish. The system can identify this type of image by pre-setting multiple reference detection points corresponding to the key parts of this type of fish. The system captures the image frames containing the reference detection points from the video data as the third image containing the target part of the fish.

[0023] The fourth image is an image containing bait and water ripples. Similarly, the shapes of the bait and water ripples can be pre-set as recognition rules, and the system can capture image frames containing bait and water ripples from the video data as the fourth image containing bait and water ripples.

[0024] It is understandable that the same image frame in the video data can be classified into multiple image types, and the difference is that as image frames of different classifications, the information subsequently extracted from the image frames is different. For example, a certain frame of image in the video data contains fish individuals, bait casting machine, bait and water surface ripples at the same time, and can clearly identify the key parts of the fish individuals, then it can be used as the first image containing fish individuals and bait casting machine, the third image containing the target part of the fish, and the fourth image containing bait and water surface ripples.

[0025] Step S20, extracting the distance information between the individual fish and the bait caster in the first image, extracting the motion feature information of the individual fish in the second image, extracting the target fish part feature information of the individual fish in the third image, and extracting the water surface bait quantity information and water surface ripple information in the fourth image; In this embodiment, after the image classification is completed, the target information in each type of image is extracted based on the information extraction rules corresponding to each image type, so as to determine the distance information between the individual fish and the bait caster, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information.

[0026] Step S30, determining the result of identifying the feeding intensity of the school of fish based on the distance information of each of the individual fish, the movement characteristic information of the individual fish, the characteristic information of the target part of the fish, the amount of bait on the water surface and the water surface ripple information.

[0027] In this embodiment, the feeding desire of the fish school is quantified by the distance information between the individual fish and the bait caster. The closer the distance between the individual fish and the bait caster, the stronger the feeding desire of the individual fish. The closer the combined distance between the entire school of fish individuals and the bait caster, the stronger the feeding desire of the fish school, and the higher the feeding intensity.

[0028] The motion characteristic information of individual fish reflects the activity of the individual fish. The richer the motion characteristic information of individual fish, the stronger the activity, and the corresponding feeding intensity increases accordingly. The information on the amount of bait on the water surface reflects whether the current amount of bait invested is sufficient. The more bait on the water surface, the greater the extent to which the amount of bait invested exceeds the food intake of the fish school, and the weaker the feeding intensity; otherwise, the stronger it is.

[0029] The water surface ripple information reflects the level of activity of the fish when eating. When the fish are eating, they will splash water on the water surface to form ripples. The richer the water surface ripple information means the more active the fish are when eating, and the corresponding feeding intensity will be stronger; otherwise, the weaker it will be.

[0030] Therefore, in this embodiment, after obtaining the above information, the feeding intensity of the fish school is comprehensively identified based on the distance information between the individual fish and the bait caster, the movement characteristic information of the individual fish, the characteristic information of the target part of the fish, the amount of bait on the water surface and the water surface ripple information to obtain the feeding intensity identification result of the fish school.

[0031] Optionally, the fish school feeding intensity identification result includes at least one of feeding frequency, feeding intensity and target feeding strategy. Based on the above information, the feeding frequency and feeding intensity of the fish school are calculated, and a target feeding strategy including at least one of appropriate feeding amount, feeding time and feeding frequency is formulated according to the feeding frequency and feeding intensity, so as to improve feeding efficiency, avoid waste, and ensure that the feeding needs of the fish school are accurately met.

[0032] In the technical solution provided in this embodiment, the feeding intensity of fish schools is identified by integrating multi-dimensional information such as the movement characteristics, spatial characteristics and water surface environment characteristics of individual fish, so as to achieve accurate identification and quantification of the feeding intensity of fish schools, and provide real-time decision support for feeding bait, so as to dynamically adjust the feeding amount, feeding time and feeding frequency, etc., thereby improving feeding efficiency, avoiding waste, and ensuring that the feeding needs of fish schools are accurately met. Example 2

[0033] Based on the first embodiment, in this embodiment, in step S20, the step of extracting the distance information between the individual fish in the first image and the bait caster includes: Step S21, inputting the first image into a Mamba-YOLO target detection model, so as to select a fish individual frame and a bait casting machine frame in the first image through the Mamba-YOLO target detection model; Step S22, determining, among the individual fish frames, a target fish frame whose confidence is greater than a preset confidence threshold, and determining a pixel distance between the target fish frame and the bait casting machine frame; Step S23, determining the conversion coefficient between the pixel distance and the actual distance through calibration, and calculating the actual distance according to the conversion coefficient and the pixel distance as the distance information between the individual fish and the bait caster.

[0034] Specifically, in this embodiment, an RGB image of a fish feeding scene is collected and input into the Mamba-YOLO target detection model to detect individual fish and baitcasters in the image, and output the coordinate information of the detection frame and the confidence score. According to the coordinates of the center point of the detection frame, the pixel distance between the individual fish and the baitcaster is calculated using the Euclidean distance formula; through a calibration experiment, a conversion relationship between pixel distance and actual distance is established, and a proportional coefficient is obtained to convert the pixel distance into actual distance.

[0035] Optionally, the detection results of multiple frames of images may be processed by weighted averaging to optimize the distance estimation accuracy between individual fish and the baitcasting machine, and finally output the estimation result. Example 3

[0036] Based on the first embodiment, in this embodiment, the second image includes a plurality of continuous RGB image frames, and in step S20, the step of extracting the individual fish motion feature information in the second image includes: Step S24, using an optical flow method to process a plurality of continuous RGB image frames; Step S25, calculating pixel displacement information between adjacent RGB image frames, and calculating movement direction information and velocity component information of the individual fish according to the pixel displacement information; Step S26, determining the turning angle information of the individual fish according to the motion direction information calculated between the plurality of adjacent RGB image frames, and calculating the acceleration information of the individual fish according to the velocity component information calculated between the plurality of adjacent RGB image frames; Step S27: determining the velocity component information, the rotation angle information and the acceleration information as the individual motion feature information.

[0037] In this embodiment, a dense optical flow method is used to estimate the motion vector of each pixel between adjacent image frames, obtain the displacement of each pixel between two consecutive frames, and generate pixel-level optical flow field information. The displacement includes displacement information in the horizontal direction (x-axis) and the vertical direction (y-axis), which can reflect the movement direction and speed of the fish. Based on the pixel motion vector in the optical flow field, the motion vector of the individual fish area is integrated and analyzed by the weighted aggregation method to calculate the instantaneous speed of the individual fish at a specific time point. Subsequently, by comparing the changing trend of the speed value in the consecutive image frames, the acceleration of the individual fish is further calculated to evaluate whether the fish is in a dynamic state of acceleration or deceleration.

[0038] Based on the motion vector of each pixel, the direction change of the individual fish in the continuous image frames is calculated. For each pair of adjacent image frames, the change amplitude of the pixel's motion vector direction is analyzed. If the individual fish turns or turns sharply, the direction of the pixel's motion vector will change significantly. By calculating the direction change of each pixel in two adjacent frames, the turning angle information of the individual fish can be obtained, which is used to quantify the directional change characteristics of the fish in feeding behavior. This information can accurately reflect the turning action and related behavior patterns of the fish, providing a richer dynamic feature dimension for the overall movement behavior analysis of the fish school.

[0039] Combined with the instantaneous speed, acceleration and rotation angle information of individual fish, the movement state of fish is comprehensively evaluated, and the extracted movement feature data is used as important input data for the subsequent fish feeding intensity recognition model. The above method extracts the movement information of individual fish, providing a data basis for the real-time quantitative analysis of fish feeding behavior and the optimization of feeding strategies. Example 4

[0040] Based on the first embodiment, in this embodiment, the third image includes a plurality of continuous RGB image frames, and in step S20, the step of extracting characteristic information of the target fish part of the individual fish in the third image includes: Step S28, using a high-resolution network to detect the key point information of the fish in the RGB image frame, and tracking and marking the key point information in the RGB image frame through individual identification technology, wherein the key point information of the fish includes the spatial coordinates corresponding to the eyes, tail, upper fin and lower fin of the fish individual; Step S29, determining target key point information above the confidence threshold among the key point information; Step S210, determining geometric information between target key point information based on preset morphological constraints of the fish body, and determining the geometric information as characteristic information of the target part of the fish.

[0041] In this embodiment, a high-resolution network (HRNet) is used to detect fish key points on the RGB image frames collected by the camera device to accurately extract the characteristic point information of individual fish, including the spatial coordinates of the eyes, tail, upper fins and lower fins. The extraction of these key point information provides basic data support for the analysis of individual fish behavior and group interaction characteristics.

[0042] In the key point detection process, this embodiment introduces a tracking algorithm based on individual identification to perform real-time labeling and dynamic tracking of key point information of fish schools and fish. This method can maintain the spatial consistency of key points in the time series and effectively distinguish the characteristics of each individual fish, ensuring the accuracy and robustness of the detection results.

[0043] In order to further improve the accuracy of key point detection, this embodiment designs a screening mechanism based on confidence score, and filters low-confidence key points that may exist in the detection by setting the confidence score threshold. Interpolation algorithm and posture optimization technology are used to correct the spatial position of the key points that pass the screening, so as to reduce the detection error caused by image resolution limitation or occlusion, thereby improving the reliability of key point coordinate positioning.

[0044] Based on the detected key point information, this embodiment further analyzes the spatial distribution characteristics of the key points and their geometric relationships. Combined with the morphological constraints of the fish body (the length-to-width ratio of the fish body and the position relationship of the fins), the degree of occlusion between individual fish is quantitatively evaluated. Through this occlusion analysis method, the relative position relationship and overlapping area of ​​individuals in the fish school can be accurately determined, laying a data foundation for the subsequent dynamic analysis and identification of the feeding behavior pattern of the fish school.

[0045] Finally, complete key point detection and occlusion analysis information is generated, and this information is used as an important input feature of the subsequent feeding intensity recognition model. Example 5

[0046] Based on the first embodiment, in this embodiment, in step S20, the step of extracting the water surface bait quantity information and water surface ripple information in the fourth image includes: Step S211, preprocessing the fourth image to reduce random noise in the fourth image and balance brightness; Step S212, using the DINO target detection algorithm based on the DETR framework to locate the bait residue area on the water surface in the preprocessed fourth image, and extract the water surface bait quantity information in the bait residue area; Step S213, and, extracting the peak position and wavelength distribution characterizing the water surface ripples in the fourth image by using image spectrum analysis technology, and analyzing the peak position and the wavelength distribution by using time series technology to determine the water surface ripple information.

[0047] In this embodiment, the collected RGB image frames are subjected to data preprocessing. The preprocessing stage includes image denoising to effectively reduce the interference of random noise on the subsequent detection algorithm, and compensates for the uneven image brightness caused by complex and changeable lighting conditions through a lighting correction method, thereby significantly improving the visual quality of the image and the accuracy of detection.

[0048] The DINO target detection algorithm, which has been pre-trained with a dedicated surface bait residue detection dataset, is used to analyze the pre-processed image frames. The DINO model accurately locates the surface bait residue area by learning the feature representation capabilities of complex backgrounds and small targets, while extracting the quantity, morphology, and spatial distribution characteristics of the bait residue. During the pre-training process, the model optimizes parameter configuration for specific surface feeding scenarios, enhancing its robustness and adaptability to complex environmental backgrounds and small targets.

[0049] In order to obtain the dynamic characteristics of the water surface environment, in this embodiment, the multi-dimensional characteristic information of water surface ripples is extracted from continuous frames by combining image spectrum analysis and time series analysis technology, including parameters such as peak position, wavelength distribution, fluctuation amplitude and change rate. By capturing the key characteristics of the dynamic changes of ripples, the present invention can effectively characterize the transient characteristics of water surface fluctuations, thereby providing more detailed environmental dynamic analysis results.

[0050] In order to further improve the reliability of detection and feature extraction results, in this embodiment, an adaptive confidence threshold mechanism is introduced to dynamically adjust the filter threshold of the confidence score according to the specific detection scenario, thereby eliminating the interference of low-confidence detection results. Post-processing optimization technology is used to perform spatial smoothing and precise boundary adjustment on the detection output to ensure that the final output of the bait residue quantity and ripple characteristics has higher accuracy and robustness.

[0051] Finally, through the above steps, comprehensive detection of the residual bait area on the water surface and accurate extraction of water surface ripple characteristics can be achieved. Example 6

[0052] Based on the first embodiment, in this embodiment, step S30 includes: Step S31, inputting the distance information, the individual fish motion characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information into a fish school feeding intensity recognition and quantification model, wherein the fish school feeding intensity recognition and quantification model includes a spatiotemporal feature fusion module and a self-attention mechanism, and the self-attention mechanism is used to extract the global and local features of each modal data and assign weights; Step S32, obtaining the fish feeding intensity identification result output by the fish feeding intensity identification and quantification model.

[0053] In this embodiment, the integrated multi-dimensional data is input into the fish feeding intensity recognition and quantification model. The model includes a spatiotemporal feature fusion module and a self-attention mechanism, which are used to extract global and local information of various features and highlight key features through weight distribution. By training on the fish feeding behavior data set, the loss function of the model is optimized to ensure the accuracy of its correlation modeling of multimodal features and feeding intensity prediction. In the real-time reasoning process, based on the dynamic change information of continuous image frames, this embodiment combines the temporal attention mechanism to analyze the dynamic characteristics of the fish feeding behavior and output the quantitative results of the fish feeding intensity. Finally, the quantitative results are presented in the form of feeding frequency and intensity level, and the feeding system can be linked to dynamically adjust the feeding strategy according to the feeding status of the fish, so as to achieve precise feeding and resource optimization.

[0054] In addition, as an implementation solution, refer to Figure 2 This embodiment also proposes a fish feeding intensity recognition model, and the fish feeding intensity recognition model includes: The image and set information extraction module 100 is used to extract a first image containing both individual fish and a baitcasting machine and a second image containing a fish motion track from the video data, and to extract distance information between the individual fish and the baitcasting machine in the first image, and to extract motion feature information of the individual fish in the second image; The fish individual feature and water surface information extraction module 200 is used to extract the third image containing the target fish part and the fourth image containing the bait and the water surface ripples, and extract the fish target part feature information of the individual fish in the third image, and the water surface bait quantity information and the water surface ripple information in the fourth image; The fish feeding intensity identification and quantification module 300 is used to determine the fish feeding intensity identification result based on the distance information, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information.

[0055] As an implementation plan, Figure 3 This is a schematic diagram of the architecture of the hardware operating environment of the fish feeding intensity identification system involved in the embodiment of the present application.

[0056] like Figure 3As shown, the fish feeding intensity identification system may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0057] Those skilled in the art will understand that Figure 3 The fish feeding intensity identification system architecture shown in the figure does not constitute a limitation of the fish feeding intensity identification system, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0058] like Figure 3 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a fish feeding intensity recognition program. The operating system is a program that manages and controls the hardware and software resources of the fish feeding intensity recognition system, the operation of the fish feeding intensity recognition program and other software or programs.

[0059] exist Figure 3 In the fish feeding intensity identification system shown, the user interface 1003 is mainly used to connect to the terminal and communicate data with the terminal; the network interface 1004 is mainly used for the background server and communicates data with the background server; the processor 1001 can be used to call the fish feeding intensity identification program stored in the memory 1005.

[0060] In this embodiment, the fish feeding intensity recognition system includes: a memory 1005, a processor 1001, and a fish feeding intensity recognition program stored in the memory and executable on the processor, wherein: When the processor 1001 calls the fish school feeding intensity identification program stored in the memory 1005, the following operations are performed: Collecting video data of fish feeding, and extracting a first image containing both fish individuals and a baitcasting machine, a second image containing a fish movement track, a third image containing a target part of the fish, and a fourth image containing bait and water surface ripples from the video data; Extracting the distance information between the individual fish and the bait casting machine in the first image, extracting the motion feature information of the individual fish in the second image, extracting the target fish part feature information of the individual fish in the third image, and extracting the water surface bait quantity information and water surface ripple information in the fourth image; The fish feeding intensity recognition result is determined based on the distance information of each individual fish, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information.

[0061] When the processor 1001 calls the fish school feeding intensity identification program stored in the memory 1005, the following operations are performed: Inputting the first image into a Mamba-YOLO target detection model, so as to select a fish individual frame and a bait casting machine frame in the first image through the Mamba-YOLO target detection model; Determine, among the individual fish frames, a target fish frame whose confidence is greater than a preset confidence threshold, and determine a pixel distance between the target fish frame and the bait casting machine frame; The conversion coefficient between the pixel distance and the actual distance is determined by calibration, and the actual distance is calculated based on the conversion coefficient and the pixel distance as the distance information between the individual fish and the bait caster.

[0062] When the processor 1001 calls the fish school feeding intensity identification program stored in the memory 1005, the following operations are performed: Processing a plurality of continuous RGB image frames using an optical flow method; Calculating pixel displacement information between adjacent RGB image frames, and calculating movement direction information and velocity component information of the individual fish according to the pixel displacement information; Determine the turning angle information of the individual fish according to the motion direction information calculated between the plurality of adjacent RGB image frames, and calculate the acceleration information of the individual fish according to the velocity component information calculated between the plurality of adjacent RGB image frames; The velocity component information, the rotation angle information and the acceleration information are determined as the individual motion feature information.

[0063] When the processor 1001 calls the fish school feeding intensity identification program stored in the memory 1005, the following operations are performed: Using a high-resolution network to detect the key point information of the fish in the RGB image frame, and tracking and marking the key point information in the RGB image frame through individual identification technology, wherein the key point information of the fish includes the spatial coordinates corresponding to the eyes, tail, upper fin and lower fin of the fish individual; Determine target key point information above a confidence threshold among each of the key point information; The geometric information between the target key point information is determined based on the preset morphological constraints of the fish body, and the geometric information is determined as the characteristic information of the target part of the fish.

[0064] When the processor 1001 calls the fish school feeding intensity identification program stored in the memory 1005, the following operations are performed: Preprocessing the fourth image to reduce random noise in the fourth image and equalize brightness; Using the DINO target detection algorithm based on the DETR framework, locating the bait residue area on the water surface in the fourth image after preprocessing, and extracting the water surface bait quantity information in the bait residue area; Furthermore, the peak position and wavelength distribution representing the water surface ripples in the fourth image are extracted by image spectrum analysis technology, and the peak position and the wavelength distribution are analyzed by time series technology to determine the water surface ripple information.

[0065] When the processor 1001 calls the fish school feeding intensity identification program stored in the memory 1005, the following operations are performed: Inputting the distance information, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information into a fish school feeding intensity identification and quantification model, wherein the fish school feeding intensity identification and quantification model includes a spatiotemporal feature fusion module and a self-attention mechanism, and the self-attention mechanism is used to extract the global and local features of each modal data and assign weights; Obtain the fish feeding intensity identification result output by the fish feeding intensity identification and quantification model.

[0066] In addition, it can be understood by a person skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the fish feeding intensity identification system to implement the process steps of the embodiment of the above method.

[0067] Therefore, the present application also provides a computer-readable storage medium, which stores a fish feeding intensity identification program. When the fish feeding intensity identification program is executed by a processor, it implements the various steps of the multi-dimensional feature coordinated fish feeding intensity quantitative identification method described in the above embodiment.

[0068] The computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0069] It should be noted that since the storage medium provided in the embodiment of the present application is the storage medium used to implement the method of the embodiment of the present application, based on the method introduced in the embodiment of the present application, the person skilled in the art can understand the specific structure and deformation of the storage medium, so it is not repeated here. All storage media used in the method of the embodiment of the present application belong to the scope of protection of this application.

[0070] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0071] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0072] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0074] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0075] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0076] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for quantitatively identifying the feeding intensity of fish schools based on multi-dimensional feature collaboration, characterized in that: The method comprises the following steps: Collecting video data of fish feeding, and extracting a first image containing both fish individuals and a baitcasting machine, a second image containing a fish movement track, a third image containing a target part of the fish, and a fourth image containing bait and water surface ripples from the video data; Extracting the distance information between the individual fish and the bait casting machine in the first image, extracting the motion feature information of the individual fish in the second image, extracting the target fish part feature information of the individual fish in the third image, and extracting the water surface bait quantity information and water surface ripple information in the fourth image; The fish feeding intensity recognition result is determined based on the distance information of each individual fish, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information.

2. The method according to claim 1, characterized in that The step of extracting the distance information between the individual fish in the first image and the baitcasting machine comprises: Inputting the first image into a Mamba-YOLO target detection model, so as to select a fish individual frame and a bait casting machine frame in the first image through the Mamba-YOLO target detection model; Determine, among the individual fish frames, a target fish frame whose confidence is greater than a preset confidence threshold, and determine a pixel distance between the target fish frame and the bait casting machine frame; The conversion coefficient between the pixel distance and the actual distance is determined by calibration, and the actual distance is calculated based on the conversion coefficient and the pixel distance as the distance information between the individual fish and the bait caster.

3. The method according to claim 1, characterized in that The second image includes a plurality of continuous RGB image frames, and the step of extracting the individual fish movement feature information in the second image includes: Processing a plurality of continuous RGB image frames using an optical flow method; Calculating pixel displacement information between adjacent RGB image frames, and calculating movement direction information and velocity component information of the individual fish according to the pixel displacement information; Determine the turning angle information of the individual fish according to the motion direction information calculated between the plurality of adjacent RGB image frames, and calculate the acceleration information of the individual fish according to the velocity component information calculated between the plurality of adjacent RGB image frames; The velocity component information, the rotation angle information and the acceleration information are determined as the individual motion feature information.

4. The method according to claim 1, characterized in that The third image includes a plurality of continuous RGB image frames, and the step of extracting characteristic information of target fish parts of individual fish in the third image includes: Using a high-resolution network to detect the key point information of the fish in the RGB image frame, and tracking and marking the key point information in the RGB image frame through individual identification technology, wherein the key point information of the fish includes the spatial coordinates corresponding to the eyes, tail, upper fin and lower fin of the fish individual; Determine target key point information above a confidence threshold among each of the key point information; The geometric information between the target key point information is determined based on the preset morphological constraints of the fish body, and the geometric information is determined as the characteristic information of the target part of the fish.

5. The method according to claim 1, characterized in that The step of extracting the water surface bait quantity information and water surface ripple information in the fourth image comprises: Preprocessing the fourth image to reduce random noise in the fourth image and equalize brightness; Using the DINO target detection algorithm based on the DETR framework, locating the bait residue area on the water surface in the fourth image after preprocessing, and extracting the water surface bait quantity information in the bait residue area; Furthermore, the peak position and wavelength distribution representing the water surface ripples in the fourth image are extracted by image spectrum analysis technology, and the peak position and the wavelength distribution are analyzed by time series technology to determine the water surface ripple information.

6. The method according to claim 1, characterized in that The step of determining the result of the fish school feeding intensity recognition according to the distance information, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information comprises: Inputting the distance information, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information into a fish school feeding intensity identification and quantification model, wherein the fish school feeding intensity identification and quantification model includes a spatiotemporal feature fusion module and a self-attention mechanism, and the self-attention mechanism is used to extract the global and local features of each modal data and assign weights; Obtain the fish feeding intensity identification result output by the fish feeding intensity identification and quantification model.

7. The method according to claim 1 or 6, characterized in that The fish school feeding intensity identification result includes at least one of feeding frequency, feeding intensity and target feeding strategy, wherein the target feeding strategy includes at least one of feeding amount, feeding time and feeding frequency.

8. A fish feeding intensity identification model, characterized in that: The fish school feeding intensity identification model includes: An image and set information extraction module, used to extract a first image containing both individual fish and a baitcasting machine and a second image containing a fish motion track from the video data, and to extract distance information between the individual fish and the baitcasting machine in the first image, and to extract motion feature information of the individual fish in the second image; A fish individual feature and water surface information extraction module, used to extract a third image containing a target fish part and a fourth image containing bait and water surface ripples, and extract fish target part feature information of individual fish in the third image, and water surface bait quantity information and water surface ripple information in the fourth image; The fish feeding intensity identification and quantification module is used to determine the fish feeding intensity identification result based on the distance information, the individual fish movement characteristic information, the target fish part characteristic information, the water surface bait quantity information and the water surface ripple information.

Images