AI visual fish metering method and system based on event triggering

Abstract

The invention provides an AI visual fish metering method and system based on event triggering, relates to the technical field of fishing tackles, and solves the technical problems of resource waste, data redundancy and short service life of an existing visual metering method. The technical method comprises the following steps: monitoring the quality of a metering area; when the quality change of the metering area in the set time period exceeds a preset threshold value, exiting the dormant state; acquiring continuous image data of the metering area; inputting the continuous image data into a pre-trained deep learning model to obtain a fish target detection and tracking result; on the basis of the fish target detection and tracking result, fish metering is carried out according to a preset metering rule; and after a preset stop condition is met, recovering the dormant state. The event-triggered awakening mode is adopted to prevent the device from being in a high-power-consumption state for a long time, waste of a large amount of resources is avoided, and the service life of the device is prolonged.

Description

Technical Field

[0001] This invention relates to the field of fishing tackle technology, and in particular to an event-triggered AI visual fish measurement method and system. Background Technology

[0002] In aquaculture and commercial fishing grounds, accurate measurement of fish weight and quantity is a crucial management aspect. For example, fish farms need to count fry and estimate the weight of adult fish to determine feeding amounts and harvesting times; fishing grounds require billing based on the species and total weight of fish caught. Existing automated visual measurement solutions mostly rely on continuous camera operation, leading to the following new problems: 1. Resource waste and high power consumption: Continuous recording and calculation without fish present results in significant waste of electricity and computing power, a fatal flaw for outdoor scenarios powered by solar or batteries. 2. Data redundancy: Generates a large amount of invalid video data, increasing storage costs and the burden of subsequent data filtering. 3. Shortened lifespan: Continuous high-load operation affects the equipment's lifespan.

[0003] Therefore, there is an urgent need to design a metering method that uses an event-triggered mechanism to achieve intelligent on-demand activation. Summary of the Invention

[0004] The purpose of this invention is to provide an event-triggered AI visual fish measurement method and system, which solves the technical problems of resource waste, data redundancy, and short lifespan in existing visual measurement methods. The various technical effects of the preferred solutions among the many technical solutions provided by this invention are detailed below.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The event-triggered AI visual fish measurement method provided by this invention includes the following steps: Monitor the quality of the measurement area; When the quality change of the measurement area exceeds a preset threshold within a set time period, exit the sleep state and acquire continuous image data of the measurement area. The continuous image data is input into a pre-trained deep learning model to obtain fish target detection and tracking results; Based on the fish target detection and tracking results, fish are quantified according to preset measurement rules; After the preset stop conditions are met, the system resumes hibernation.

[0006] Preferably, after acquiring the image data of the measurement area, the method further includes: A lightweight model is used to verify the target based on the acquired image data; If no valid fish target is detected, the process is interrupted and the system enters the dormant state.

[0007] Preferably, the continuous image data is input into a pre-trained deep learning model to obtain fish target detection and tracking results, including: The initial velocity of the detected target in the first frame of image data is preset; Tracks with a duration of less than 5 frames are filtered out.

[0008] Preferably, inputting the continuous image data into a pre-trained deep learning model to obtain fish target detection and tracking results further includes: Perform forward reasoning on each frame of image information to output the bounding box and confidence score of the fish. Output the coordinates of key points on the fish's body; When the bounding boxes overlap significantly or the confidence level is low, generate a pixel-level precise mask for each fish body.

[0009] Preferably, inputting the continuous image data into a pre-trained deep learning model to obtain fish target detection and tracking results further includes: For the trajectory of each fish that already exists in the previous frame, predict the most likely location of each fish in the current frame based on its historical speed and position; Match all bounding boxes detected in this frame with all predicted trajectory locations; Optimal matching is achieved using a fusion cost matrix, which includes motion cost and appearance cost. The motion cost and appearance cost are fused to form a total cost matrix. Each detection box is assigned an optimal existing trajectory ID, or a new trajectory is created, or an old trajectory is terminated. The system maintains at least 5 frames of valid trajectories.

[0010] Preferably, the measurement rules include: Pre-define one or more virtual measurement lines or polygonal measurement areas in the image coordinate system; When the fish's movement trajectory crosses the measuring line or polygonal area, a measuring action is performed; Double-counting treatment is applied to the already measured fish.

[0011] Preferably, the anti-duplication measurement process includes: Assign a unique ID to the movement trajectory of each fish and set a "measured" status flag; Once the fish has been measured, the “measured” status flag is set to True; To prevent the fish from being weighed again before the weighing sensor is reset.

[0012] Preferably, the stopping condition includes: the quality of the metering area recovers to below a threshold and remains below a preset time.

[0013] The system provided by this invention, which applies the above-mentioned event-triggered AI visual fish measurement method, includes a protective shell, a first flap, a pallet pod, a weighing sensor, a drive unit, a second flap, and a camera. The first flap is provided at the upper inlet of the protective shell, and the pallet pod is provided below the first flap. The pallet pod is connected to the protective shell through a weighing sensor. The bottom wall of the pallet pod is the second flap, which is connected to the drive end of the drive unit. The camera is provided at the top of the pallet pod.

[0014] Preferably, it also includes a proximity sensor disposed on the protective shell, the proximity sensor being disposed on one side of the first flap in the closed state.

[0015] The application employs the above technical solution and has at least the following beneficial effects: By monitoring the quality of the metering area and determining whether to exit the sleep state based on its quality changes, the device can avoid being in a high-power state for a long time, reducing power and computing costs. It can also avoid excessive data acquisition, avoiding resource waste, storage costs, and subsequent screening burdens, indirectly extending the device's lifespan.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of an event-triggered AI visual fish measurement method provided in an embodiment of the present invention; Figure 2 This is a cross-sectional structural diagram of the event-triggered AI visual fish measurement system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the main view structure of the event-triggered AI visual fish measurement system provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the left-side structure of the event-triggered AI visual fish measurement system provided in an embodiment of the present invention; Figure 5This is a schematic diagram of the upward-looking structure of the event-triggered AI visual fish measurement system provided in an embodiment of the present invention; Figure 6 This is a top view of the event-triggered AI visual fish measurement system provided in an embodiment of the present invention. Figure 7 This is a three-dimensional structural diagram of the event-triggered AI visual fish measurement system provided in an embodiment of the present invention.

[0019] In the diagram: 1. Protective shell; 2. First flap; 3. Pallet pod; 4. Weighing sensor; 5. Drive unit; 6. Second flap; 7. Camera; 8. Proximity sensor; 9. Protective cloth bottom ring; 10. Net bag rear retaining ring; 11. Net bag front retaining ring; 12. Mounting bracket; 13. Control box; 14. Antenna; 15. First guide plate. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0021] A specific embodiment of the present invention provides an event-triggered AI visual fish measurement method, characterized by comprising the following steps: Monitor the quality of the measurement area; When the quality change of the measurement area exceeds a preset threshold within a set time period, exit the sleep state and acquire continuous image data of the measurement area. The continuous image data is input into a pre-trained deep learning model to obtain fish target detection and tracking results; Based on the fish target detection and tracking results, fish are quantified according to preset measurement rules; After the preset stop conditions are met, the system resumes hibernation.

[0022] During the monitoring of the metering area's quality, a weighing sensor 4 can continuously or intermittently monitor the mass inside the tank. Once the detected weight exceeds a preset threshold (to ignore interference from water flow fluctuations, small debris, etc.), it is determined that a target has entered, and an interrupt signal is sent to the main controller to wake up the system from its dormant state. This normal dormancy mode, based on event-driven wake-up, effectively avoids resource waste and high power consumption caused by prolonged high-power operation. It achieves intelligent power management of "dormant when there are no fish, working when there are fish," significantly reducing average power consumption and allowing the equipment to operate long-term using batteries or solar energy.

[0023] When acquiring continuous image data of the measurement area, high-speed, high-resolution image acquisition equipment is used to ensure image clarity and integrity, providing a high-quality data foundation for subsequent deep learning model processing. After inputting the continuous image data into a pre-trained deep learning model, the model, leveraging its powerful feature extraction and classification capabilities, accurately outputs fish target detection and tracking results, including information such as fish location, quantity, and movement trajectory. Based on the fish target detection and tracking results, fish are measured according to preset measurement rules. These rules fully consider various possible scenarios, such as fish overlap and occlusion, to ensure measurement accuracy. After meeting preset stopping conditions, the system automatically returns to sleep mode to reduce energy consumption, awaiting the next trigger event.

[0024] Specifically, in this application, after acquiring the image data of the measurement area, the process further includes: A lightweight model is used to verify the target based on the acquired image data; If no valid fish target is detected, the process will be interrupted and enter a dormant state.

[0025] After acquiring image data of the technical area, the system first performs preprocessing. A lightweight model is used to quickly detect targets in the first frame or the first few frames to identify valid fish targets. If a valid fish target is found, the process proceeds to the next step; otherwise, it is considered a false trigger, and the system immediately returns to sleep mode. This visual verification mechanism, combined with the aforementioned weighing verification mechanism, greatly reduces false triggers caused by debris or shaking. This significantly improves the accuracy of system startup, reduces unnecessary energy consumption, enhances overall system energy efficiency, and avoids resource waste caused by non-fish-related factors.

[0026] In a specific embodiment of this application, the continuous image data is input into a pre-trained deep learning model to obtain fish target detection and tracking results. This step includes setting the initial velocity of the detected target in the first frame of image data and filtering out trajectories with a duration of less than 5 frames.

[0027] Specifically, the first frame of image data is based on the fish partially or completely entering the tank. The initial trajectory velocity of all detected fish targets is set to zero or a small random value. For trajectories that disappear after a very short duration (e.g., appearing only for 1-2 frames), the system classifies them as noise or false detections and does not perform measurement. This allows the tracking algorithm to quickly adapt to measurement scenarios in event-triggered modes, ensuring the accuracy of measurement from the first frame.

[0028] In some embodiments, wherein the continuous image data is input into a pre-trained deep learning model to obtain fish target detection and tracking results, this step further includes: Perform forward reasoning on each frame of image information to output the bounding box and confidence score of the fish. Output the coordinates of key points on the fish's body; When the bounding boxes overlap significantly or the confidence level is low, generate a pixel-level precise mask for each fish body.

[0029] Specifically, a pre-trained deep learning object detection model (such as YOLOv8, RT-DETR, or SSD) is used to perform forward inference on each frame of the image. The model's task is to find all objects in the image that could possibly be "fish" and mark their positions with a bounding box.

[0030] For each frame, the model outputs a list [x, y, w, h, confidence, class], where: (x, y) represents the center coordinates of the detected fish body bounding box; (w, h) represents the width and height of the bounding box; Confidence represents the level of confidence in the detection result (between 0 and 1), which is the probability that the model considers the object to be a fish. In practical applications, the higher the confidence value, the more confident the model is in judging that the detection result is a fish.

[0031] The class indicates the category of the detected object; in this scenario, it is fish.

[0032] With such an output list, we can clearly and accurately obtain relevant information about the fish in each frame of the image, providing basic data for subsequent measurement and other processing.

[0033] Based on target detection, key point detection is introduced; The model not only outputs bounding boxes, but also the coordinates of key points on the fish's body, most typically the "fish head" and "fish tail" or the "fish body center".

[0034] Attach a set of coordinates to each detection box, such as [head_x, head_y], [tail_x, tail_y].

[0035] Determining the direction of the fish's head provides strong prior knowledge for subsequent movement direction determination and trajectory association, greatly helping to solve the problem of unclear fish body orientation.

[0036] When the bounding boxes provided by the detection model have high overlap or low confidence, the system launches an instance segmentation model (such as Mask R-CNN) in parallel. This model generates a pixel-level precise mask for each individual fish in the image.

[0037] Each fish body corresponds to a binary mask image.

[0038] By analyzing these masks, it's possible to accurately determine whether fish within two boxes are closely adjacent or truly overlapping. If the mask shows the same connected component, it may be a false positive for the same fish; if the mask shows two independent connected components that overlap, they are counted as two separate fish, providing the tracker with more accurate location information for correction.

[0039] In some embodiments, wherein the continuous image data is input into a pre-trained deep learning model to obtain fish target detection and tracking results, this step further includes: For the trajectory of each fish that already exists in the previous frame, predict the most likely location of each fish in the current frame based on its historical speed and position; Match all bounding boxes detected in this frame with all predicted trajectory locations; Optimal matching is achieved using a fusion cost matrix, which includes motion cost and appearance cost. The motion cost and appearance cost are fused to form a total cost matrix. Each detection box is assigned an optimal existing trajectory ID, or a new trajectory is created, or an old trajectory is terminated. The system maintains at least 5 frames of valid trajectories.

[0040] The detection phase only tells us "what fish are in the current frame," but not "which fish in the previous frame is which fish in the current frame." The purpose of the tracking phase is to assign a unique ID to each fish and connect their positions in different frames to form a trajectory.

[0041] Trajectory prediction and correlation: Prediction: For the trajectory of each fish that already exists in the previous frame, use motion models such as Kalman filtering to predict its most likely location in the current frame based on its historical speed and position.

[0042] Association: Match all detected bounding boxes in this frame with all predicted trajectory locations. This is a data association problem.

[0043] Association cost calculation: Optimal matching is performed using a fusion cost matrix. Motion cost: Calculate the Euclidean distance between the center of the detection box and the predicted location. The smaller the distance, the lower the cost.

[0044] Appearance Cost: A lightweight Re-Identification network is used to extract a high-dimensional feature vector from each detection box. Similarly, the nearest appearance features are saved for each existing trajectory. The cosine distance between the appearance features of the new detection box and the existing trajectories is calculated. The more similar the appearances, the lower the cost.

[0045] Matching decision: Motion cost and appearance cost are fused with certain weights to form a total cost matrix. Then, using algorithms such as the Hungarian algorithm, an optimal existing trajectory ID is assigned to each detection box, or a new trajectory is created (a new fish enters the field of view), or an old trajectory is terminated (a fish leaves the field of view).

[0046] Track Management: The system maintains a list of tracks, each track containing: track ID, lifespan (how many frames it has existed), a list of recent locations, appearance feature cache, and a "metered" status flag.

[0047] Newly created tracks will have a "trial period". Only tracks that persist for more than a certain number of frames (e.g., 5 frames) will be considered valid tracks to avoid transient noise being mistaken for fish.

[0048] In some embodiments, the measurement rules in this application include: Pre-define one or more virtual measurement lines or polygonal measurement areas in the image coordinate system; When the fish's movement trajectory crosses the measuring line or polygonal area, a measuring action is performed; Double-counting treatment is applied to the already measured fish.

[0049] The anti-duplication measurement process includes: Assign a unique ID to the movement trajectory of each fish and set a "measured" status flag; Once the fish has been measured, the “measured” status flag is set to True; To prevent the fish from being weighed again before the weighing sensor 4 is reset.

[0050] Setting up a virtual metering line: In the image coordinate system, the user presets one or more virtual lines (measurement lines) or polygonal regions (measurement areas). For example, a straight line perpendicular to the channel can be drawn in the middle of the box channel.

[0051] Implementation of measurement rules: Marker determination: For a trajectory, the system examines the position sequence of its consecutive frames. By calculating the movement of the trajectory's center point or key points relative to the measuring line, its direction can be clearly determined.

[0052] Triggering conditions: Weighing sensor 4 sensed the data; Fish-like objects were detected in the video images; Weighing sensor 4 returns to its initial state; Measurement action: When all three conditions above are met simultaneously: The system performs metering.

[0053] Immediately set the "measured" status flag for this trajectory to True to prevent duplicate measurements.

[0054] Anti-duplication measurement and reset: Anti-duplication: Once a fish has been weighed, it will not be weighed again, even if its trajectory crosses the weighing line again, until it has completely left the tank (weighing sensor 4 is reset).

[0055] Reset: When the system determines that there are no fish in the tank based on the weighing signal and is about to enter hibernation, it will reset the "measured" status flag of all tracks to False, in preparation for the next measurement trigger.

[0056] In some embodiments, the stopping condition includes: the quality of the metering area recovers to below a threshold and remains below a preset time.

[0057] Specifically, after the fish has completely left the tank, the weighing sensor 4 detects that the mass data is below the threshold. After a configurable delay (e.g., 3 seconds), the camera 7 automatically shuts off, and the main controller re-enters sleep mode.

[0058] A specific embodiment of the present invention provides a system for applying the above-described event-triggered AI visual fish measurement method, including a protective shell 1, a first flap 2, a tray pod 3, a weighing sensor 4, a drive unit 5, a second flap 6, and a camera 7; The protective shell 1 is positioned above the water surface to protect the internal components and to house them. Specifically, the protective shell 1 can be made of high-strength, corrosion-resistant materials, effectively protecting the internal equipment from external environmental interference and damage, ensuring stable and reliable operation of the system in various complex aquaculture environments. The top of the protective shell 1 has a bottom ring 9 for installing the protective mesh, and the bottom of the protective shell 1 has an open opening with a rear net retainer ring 10 and a front net retainer ring 11 on either side of the open opening. A mounting bracket 12 with multiple hooks is installed on one side wall of the protective shell 1. A control box 13 is also installed on the side wall of the protective shell 1, integrating a control circuit board and related electronic components, and an antenna 14 is mounted on it for wireless communication with a data center.

[0059] A first flap 2 is provided at the upper inlet of the protective shell 1. The first flap 2 is connected to the upper inlet edge of the protective shell 1 via a hinge and can rotate around the hinge. Its function is to control the passage for fish to enter the protective shell 1. When fish are needed to enter, the first flap 2 opens under the pressure of the fish, allowing the fish to pass through smoothly. When fish are not needed or the system is not in operation, the first flap 2 can be closed to prevent external debris or non-target fish from entering the protective shell 1 and interfering with or damaging the internal equipment. It also helps to maintain a relatively stable environment inside the protective shell 1. For this purpose, the first flap 2 is connected to the protective shell 1 via a reset component.

[0060] A first guide plate 15 is also provided inside the protective shell 1. The first guide plate 15 is inclined at a certain angle, with its upper end connected to the top opening of the protective shell 1 and its lower end extending into the protective shell 1 and close to the bottom of the first flap 2. The function of the first guide plate 15 is to guide the fish entering the protective shell 1, so that the fish can fall along a preset path, thereby ensuring that the fish enter the tray pod below.

[0061] Below the first flap 2, a pallet pod 3 is installed. The pallet pod 3 is connected to the protective shell 1 via a weighing sensor 4, which can detect changes in the mass of the pallet pod 3 in real time. When a fish falls into the pallet pod 3, its mass increases, and the weighing sensor 4 converts this mass information into an electrical signal and transmits it to the subsequent control unit. The bottom wall of the pallet pod 3 is a second flap 6, which is connected to the drive end of the drive unit 5. The drive unit 5 can drive the second flap 6 to flip. When it is necessary to process the fish in the pallet pod 3 or to transfer the fish after measurement, the drive unit 5 drives the second flap 6 to open, allowing the fish in the pallet pod 3 to fall smoothly and complete the fish transfer operation. When it is not necessary for the fish to fall or no relevant instruction is received, the second flap 6 remains closed to prevent the fish from falling accidentally and to ensure the stability and accuracy of the entire measurement and processing process. The drive unit 5 can be a DC geared motor, which is connected to the second flap 6 through a motor torque rod and a motor transmission rod.

[0062] A camera 7 is installed on the top of the tray pod 3. The camera 7 is used to capture images of the fish inside the tray pod 3 in real time.

[0063] A proximity sensor 8 is also provided on the protective shell 1, and the proximity sensor 8 is located on the side where the first flap 2 is closed. Thus, the flipping of the first flap 2 will trigger the proximity sensor 8.

[0064] In this invention, the terms "one embodiment," "some embodiments," "example," "specific example," and "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0065] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An event-triggered AI visual fish measurement method, characterized in that, Includes the following steps: Monitor the quality of the measurement area; When the quality change of the measurement area exceeds the preset threshold within a set time period, the system exits the sleep state. Acquire continuous image data of the measurement area; The continuous image data is input into a pre-trained deep learning model to obtain fish target detection and tracking results; Based on the fish target detection and tracking results, fish are quantified according to preset measurement rules; After the preset stop conditions are met, the system resumes hibernation.

2. The event-triggered AI visual fish measurement method according to claim 1, characterized in that, After acquiring the image data of the measurement area, the process further includes: A lightweight model is used to verify the target based on the acquired image data; If no valid fish target is detected, the process is interrupted and the system enters the dormant state.

3. The event-triggered AI visual fish measurement method according to claim 1, characterized in that, The continuous image data is input into a pre-trained deep learning model to obtain fish target detection and tracking results, including: The initial velocity of the detected target in the first frame of image data is preset; Tracks with a duration of less than 5 frames are filtered out.

4. The event-triggered AI visual fish measurement method according to claim 1, characterized in that, The continuous image data is input into a pre-trained deep learning model to obtain fish target detection and tracking results, and the method further includes: Perform forward reasoning on each frame of image information to output the bounding box and confidence score of the fish. Output the coordinates of key points on the fish's body; When the bounding boxes overlap significantly or the confidence level is low, generate a pixel-level precise mask for each fish body.

5. The event-triggered AI visual fish measurement method according to claim 1, characterized in that, The continuous image data is input into a pre-trained deep learning model to obtain fish target detection and tracking results, and the method further includes: For the trajectory of each fish that already exists in the previous frame, predict the most likely location of each fish in the current frame based on its historical speed and position; Match all bounding boxes detected in this frame with all predicted trajectory locations; Optimal matching is achieved using a fusion cost matrix, which includes motion cost and appearance cost. The motion cost and appearance cost are fused to form a total cost matrix. Each detection box is assigned an optimal existing trajectory ID, or a new trajectory is created, or an old trajectory is terminated. The system maintains at least 5 frames of valid trajectories.

6. The event-triggered AI visual fish measurement method according to claim 1, characterized in that, The measurement rules include: Pre-define one or more virtual measurement lines or polygonal measurement areas in the image coordinate system; When the fish's movement trajectory crosses the measuring line or polygonal area, a measuring action is performed; Double-counting treatment is applied to the already measured fish.

7. The event-triggered AI visual fish measurement method according to claim 6, characterized in that, The anti-duplication measurement process includes: Assign a unique ID to the movement trajectory of each fish and set a "measured" status flag; Once the fish has been measured, set the "measured" status flag to True; To prevent the fish from being weighed again before the weighing sensor is reset.

8. The event-triggered AI visual fish measurement method according to claim 1, characterized in that, The stopping conditions include: the quality of the metering area recovers to below a threshold and remains below a preset time.

9. A system applying the event-triggered AI visual fish measurement method according to any one of claims 1-8, characterized in that, The device includes a protective shell, a first flap, a pallet pod, a weighing sensor, a drive unit, a second flap, and a camera. The first flap is located at the upper inlet of the protective shell, and the pallet pod is located below the first flap. The pallet pod is connected to the protective shell via a weighing sensor. The bottom wall of the pallet pod is the second flap, which is connected to the drive end of the drive unit. The camera is located at the top of the pallet pod.

10. The system according to claim 9, characterized in that, It also includes a proximity sensor disposed on the protective shell, the proximity sensor being disposed on one side of the first flap in the closed state.

Images