A method, apparatus, device and medium for counting fish

Abstract

The application discloses a fish population counting method, device, equipment and medium, comprising: recognizing an image frame sequence preprocessed according to motion and environment data to obtain object and state information, assigning an identifier to the object in the first frame and writing the identifier into a presequence frame set, obtaining a first object and a second object according to the state information of the object in the presequence frame set and a target frame, the first object inheriting the identifier, the second object being reassigned the identifier, writing the target frame into the presequence frame set, taking an image frame adjacent to the target frame as a new target frame, returning to obtain the state information of the object in the presequence frame set and the target frame until all the image frames are traversed, verifying the object corresponding to the identifier according to the motion direction of the object in the recording frame corresponding to the identifier, and obtaining the number of fish populations according to the verification result of the fish. The application can be applied to the field of fish population monitoring, the accuracy of fish population counting is improved by fusing environment and device data and introducing motion direction consistency verification.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and medium for counting fish schools. Background Technology

[0002] Fish counting and monitoring are crucial for understanding population dynamics, implementing ecological protection, and conducting environmental assessments. Traditional fish counting methods primarily rely on manual interpretation of sonar images or simple image thresholding algorithms. Manual methods are not only inefficient and time-consuming, but also prone to counting errors due to subjective fatigue. Furthermore, some contact surveys may disturb or harm the fish. To overcome these shortcomings, with advancements in computer vision technology, researchers have begun to introduce non-destructive testing methods based on visual analysis to achieve automated counting. However, this technology still faces significant challenges when processing real-world underwater sonar images: sonar images are often affected by seawater turbidity, bubbles, and suspended matter, leading to decreased image quality and blurred target outlines; in addition, the irregular swimming trajectories of fish and their significant mutual occlusion can easily cause target loss and duplicate counting during tracking; furthermore, static debris in the underwater environment such as corals, algae, and fishing nets have similar visual characteristics to fish, further increasing the false detection rate of the recognition algorithm. Under the combined influence of these factors, the accuracy and robustness of existing counting methods still fall short of practical application requirements.

[0003] Therefore, improving the accuracy of fish counting has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for counting fish schools to address the problem of improving the accuracy of fish school counting.

[0005] A method for counting fish schools, comprising: A sequence of image frames collected from a target fish group using a sonar device is obtained. Motion data collected by the sonar device and environmental data of the target fish group are used. Each image frame in the image frame sequence is preprocessed based on the motion data and the environmental data to obtain a preprocessed image frame sequence. For each image frame in the preprocessed image frame sequence, object recognition is performed to obtain the object in the corresponding image frame and the state information of the corresponding object. The state information includes detection box information and motion direction. Assign a unique identifier to each object in the first frame of the preprocessed image frame sequence, write the first frame into the preceding frame set, and take the image frame directly adjacent to the first frame as the target frame. Based on the state information of the objects in the preceding frame set and the target frame, perform trajectory tracking on each object in the target frame to obtain the first object whose trajectory can be tracked and the second object whose trajectory cannot be tracked in the target frame. For any first object, inherit the identifier of the corresponding object in the preceding frame set. For any second object, reassign a unique identifier to the second object. Write the target frame into the preceding frame set, take the image frame directly adjacent to the target frame as the new target frame, and return to execute the step of tracing the trajectory of each object in the target frame according to the state information of the objects in the preceding frame set and the target frame, until all image frames are traversed to obtain the record frame and the record individual in the preprocessed image frame sequence for each identifier corresponding to the object. For any object corresponding to an identifier, the fish validity of the object is verified based on the movement direction of the object in the record frame of the object corresponding to the identifier and the movement direction of the recorded individual of the object corresponding to the identifier in the corresponding record frame. The fish verification result is obtained, and the number of fish in the target fish group is obtained based on the fish verification results of all objects corresponding to identifiers.

[0006] A fish counting device, comprising: The data acquisition module is used to acquire the image frame sequence collected by the sonar device on the target fish group, the motion data collected by the sonar device and the environmental data of the environment in which the target fish group is located, and to preprocess each image frame in the image frame sequence according to the motion data and the environmental data to obtain the preprocessed image frame sequence. The recognition module is used to perform object recognition on each image frame in the preprocessed image frame sequence to obtain the object in the corresponding image frame and the state information of the corresponding object. The state information includes detection box information and motion direction. The trajectory tracking module is used to assign a unique identifier to each object in the first frame of the preprocessed image frame sequence, write the first frame into the preceding frame set, take the image frames directly adjacent to the first frame as target frames, perform trajectory tracking on each object in the target frame according to the state information of the objects in the preceding frame set and the target frame, and obtain the first object in the target frame whose trajectory can be tracked and the second object whose trajectory cannot be tracked. For any first object, the identifier of the corresponding object in the preceding frame set is inherited, and for any second object, a new unique identifier is assigned to the second object. The loop module is used to write the target frame into the preceding frame set, take the image frame directly adjacent to the target frame as the new target frame, and return to execute the step of tracing the trajectory of each object in the target frame according to the state information of the objects in the preceding frame set and the target frame, until all image frames are traversed to obtain the record frame and the record individual in the preprocessed image frame sequence for each identifier corresponding to the object. The counting module is used to perform fish validity verification on the object corresponding to any identifier based on the movement direction of the object in the record frame of the object corresponding to the identifier and the movement direction of the recorded individual of the object corresponding to the identifier in the corresponding record frame, to obtain the fish verification result, and to obtain the number of fish in the target fish group based on the fish verification results of all the objects corresponding to the identifiers.

[0007] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described fish counting method.

[0008] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described fish counting method.

[0009] The aforementioned fish counting method, apparatus, computer equipment, and storage medium preprocess the image frame sequence of the target fish swarm based on the motion data of the sonar equipment and the environmental data of the target fish swarm's environment during acquisition. This yields a preprocessed image frame sequence. Object recognition is performed on each image frame in the preprocessed sequence to obtain the object and its state information, including bounding box information and motion direction. A unique identifier is assigned to each object in the first frame of the preprocessed image frame sequence. The first frame is written into the preceding frame set. The image frames directly adjacent to the first frame are designated as target frames. Based on the state information of the objects in the preceding frame set and the target frames, trajectory tracking is performed on each object in the target frames to identify the first object whose trajectory can be tracked and the second object whose trajectory cannot be tracked. For any first object... Inherit the identifier of the corresponding object in the preceding frame set. For any second object, reassign a unique identifier to the second object. Write the target frame into the preceding frame set. Take the image frame directly adjacent to the target frame as the new target frame. Return to execute the step of tracing the trajectory of each object in the target frame based on the state information of the objects in the preceding frame set and the target frame, until all image frames are traversed. Obtain the record frame and the record individual in the corresponding record frame for each object corresponding to the identifier in the preprocessed image frame sequence. For any object corresponding to the identifier, perform fish validity verification on the object corresponding to the identifier based on the movement direction of the object in the record frame and the movement direction of the record individual in the corresponding record frame. Obtain the fish verification result. Based on the fish verification results of all objects corresponding to the identifier, obtain the number of fish in the target fish group.

[0010] By integrating environmental data and equipment motion information to preprocess image frames, problems such as underwater turbidity and bubble interference can be effectively overcome, significantly improving image quality and target recognition. On this basis, object motion trajectories are constructed through cross-frame state tracking, and a consistency verification mechanism based on motion direction is introduced to eliminate interference from static debris such as corals and algae, effectively solving the problem of repeated counting or undercounting caused by trajectory breaks. This improves the accuracy, robustness, and automation of fish counting in complex underwater environments, providing reliable data support for ecological monitoring and fisheries research. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1This is a schematic diagram of an application environment for the fish counting method in one embodiment of the present invention; Figure 2 This is a flowchart of a fish counting method according to an embodiment of the present invention; Figure 3 This is another flowchart of a fish counting method in one embodiment of the present invention; Figure 4 This is another flowchart of a fish counting method in one embodiment of the present invention; Figure 5 This is another flowchart of a fish counting method in one embodiment of the present invention; Figure 6 This is a schematic diagram of a fish counting device in one embodiment of the present invention; Figure 7 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] The fish counting method provided in this embodiment of the invention can be applied to, for example... Figure 1 The application environment is shown. Specifically, this fish counting method is applied in a counting system, which includes, for example, […]. Figure 1 The diagram illustrates a client and server that communicate over a network to address the problem of improving the accuracy of fish counting. The client, also known as the user terminal, is the program that provides local services to the client, corresponding to the server. The client can be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0015] In one embodiment, such as Figure 2 As shown, a method for counting fish schools is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included: Step S201: Obtain the image frame sequence collected by the sonar device on the target fish group, including the motion data collected by the sonar device and the environmental data of the environment in which the target fish group is located. Based on the motion data and environmental data, preprocess each image frame in the image frame sequence to obtain the preprocessed image frame sequence.

[0016] In this embodiment, the target fish group can refer to the fish group to be counted, the image frame sequence can refer to the sonar image sequence collected by the sonar device on the target fish group, the motion data can refer to the motion-related parameter data when the sonar device collects images, such as the acceleration and angular velocity of the sonar device, and the environmental data can refer to the data of the environment in which the target fish group is located, such as water temperature, salinity and turbidity.

[0017] Specifically, based on the motion data of the sonar equipment and the environmental data of the target fish group's environment, image enhancement is performed on each image frame in the image frame sequence to obtain a preprocessed image frame sequence.

[0018] Step S202: Perform object recognition on each image frame in the preprocessed image frame sequence to obtain the object in the corresponding image frame and the state information of the corresponding object.

[0019] In this embodiment, the object in the image frame can refer to the detected target object obtained by recognizing the image frame. The detected target object can include fish and static debris in the underwater environment such as corals, algae and fishing nets that are similar to the visual characteristics of fish. The state information can include detection box information and motion direction. The detection box information can refer to the bounding box parameters of the detected target object, which can include the center coordinates of the detection box. The motion direction can refer to the motion trend description of the detected target object, which can include the direction angle of the object's motion.

[0020] Specifically, based on a preset target detection algorithm, target recognition can be performed on each image frame in the preprocessed image frame sequence to obtain the detected target object (including real fish and static debris in the underwater environment such as corals, algae and fishing nets that are similar to the visual features of the object) and the detection box information and motion direction of the corresponding target object in the image frame.

[0021] Step S203: Assign a unique identifier to each object in the first frame of the preprocessed image frame sequence, write the first frame into the preceding frame set, and take the image frame directly adjacent to the first frame as the target frame. Based on the state information of the objects in the preceding frame set and the target frame, perform trajectory tracking on each object in the target frame to obtain the first object whose trajectory can be tracked and the second object whose trajectory cannot be tracked in the target frame. For any first object, inherit the identifier of the corresponding object in the preceding frame set. For any second object, reassign a unique identifier to the second object.

[0022] Step S204: Write the target frame into the preceding frame set, take the image frame directly adjacent to the target frame as the new target frame, return to execute the step of tracing the trajectory of each object in the target frame according to the state information of the objects in the preceding frame set and the target frame, until all image frames are traversed, and obtain the record frame and the record individual in the preprocessed image frame sequence for each identifier corresponding to the object.

[0023] In this embodiment, the first frame can refer to the first image frame in the preprocessed image frame sequence, the preceding frame set can refer to a pre-defined image frame set, the target frame can refer to the image frame to be tracked, the first object can refer to the object in the target frame whose trajectory can be tracked in the preceding frame set, that is, the object has appeared in both the target frame and the preceding frame set, the second object can refer to the object in the target frame whose trajectory cannot be tracked in the preceding frame set, that is, the object has not appeared in the preceding frame set, the record frame can refer to the image frame in the preprocessed image frame sequence in which the object corresponding to the identifier has appeared, and the record individual can refer to the detection target instance corresponding to the object corresponding to the identifier in the image frame in which it has appeared.

[0024] Specifically, a unique identifier is assigned to each object in the first frame (first image frame) of the preprocessed image frame sequence. The first frame is added to the preceding frame set. The target frames that are directly adjacent to the first frame are determined. Based on the state information of each object in the preceding frame set and the state information of each object in the target frame, trajectory tracking is performed on each object in the target frame. The first object in the target frame whose trajectory can be tracked from the preceding frame set and the second object whose trajectory cannot be tracked are determined. For any first object, the identifier of the target detection instance that appeared in the preceding frame set is determined and inherited. For any second object, a new identifier is reassigned to the second object. The tracked target frame is written into the preceding frame set. The image frames that are directly adjacent to the tracked target frame are determined and the image frame is taken as the new target frame. The process of performing trajectory tracking on each object in the target frame based on the state information of the objects in the preceding frame set and the target frame is returned to continue until all image frames are traversed, and the record frame and the record individual in the corresponding record frame for each object in the preprocessed image frame sequence are obtained.

[0025] Step S205: For any object corresponding to an identifier, perform fish validity verification on the object corresponding to the identifier based on the movement direction of the object in the record frame of the object corresponding to the identifier and the movement direction of the recorded individual of the object corresponding to the identifier in the corresponding record frame, and obtain the fish verification result. Based on the fish verification results of all objects corresponding to identifiers, obtain the number of fish in the target fish group.

[0026] In this embodiment, the fish verification result can refer to the result of fish verification of the object, the purpose of which is to exclude static debris such as corals, algae and fishing nets that are similar to the visual characteristics of fish.

[0027] Specifically, for any object corresponding to an identifier, the fish validity of the object corresponding to the identifier is verified based on the movement direction of all objects in the recording frame of the object corresponding to the identifier and the movement direction of the individual recorded object of the object corresponding to the identifier, and the fish verification result is obtained (i.e., whether the object corresponding to the identifier is a real fish or a static object with similar visual characteristics to fish). Based on the fish verification results of all objects corresponding to the identifier, the number of fish in the target fish is obtained.

[0028] In this embodiment, by fusing environmental data and device motion information to preprocess image frames, problems such as underwater turbidity and bubble interference can be effectively overcome, significantly improving image quality and target recognition. On this basis, object motion trajectories are constructed through cross-frame state tracking, and a consistency verification mechanism based on motion direction is introduced to eliminate interference from static debris such as corals and algae, effectively solving the problem of repeated counting or undercounting caused by trajectory breaks. This improves the accuracy, robustness, and automation of fish counting in complex underwater environments, providing reliable data support for ecological monitoring and fisheries research.

[0029] In one embodiment, such as Figure 3 As shown, a method for counting fish is provided. In step S201 above, each image frame in the image frame sequence is preprocessed based on motion data and environmental data to obtain a preprocessed image frame sequence, including the following steps: Step S301: Based on environmental data, perform contrast enhancement on each image frame in the image frame sequence, and based on motion data, perform blur removal on each image frame in the image frame sequence to obtain the enhanced image frame sequence.

[0030] Step S302: Perform wavelet denoising on the enhanced image frame sequence to obtain the preprocessed image frame sequence.

[0031] Specifically, the grayscale histogram distribution of each image frame is obtained. Based on environmental data, the contrast of each image frame is dynamically adjusted using contrast-limited adaptive histogram equalization to enhance the boundary difference between the fish and suspended objects. Based on motion data, motion blur in each image frame is corrected to obtain an enhanced image frame sequence. Based on wavelet transform, the enhanced image frame sequence is denoised to suppress the interference of bubbles and suspended objects, resulting in a preprocessed image frame sequence.

[0032] In this embodiment, the acquired image frames are enhanced and denoised by using motion data from the sonar device and environmental data of the underwater environment in which the fish swarm is located. This reduces the inherent blurring, low contrast, and high noise problems of deep-sea sonar images, providing clear and reliable input for subsequent target detection and tracking algorithms, thereby further improving the accuracy of fish swarm counting.

[0033] In one embodiment, such as Figure 4 As shown, a method for counting fish is provided. In step S203 above, based on the state information of objects in the preceding frame set and the target frame, trajectory tracking is performed on each object in the target frame to obtain the first object in the target frame whose trajectory can be tracked. This includes the following steps: Step S401: For any object in the target frame, based on the object's state information and the state information of objects in the preceding frame set, detect whether there is an object in the preceding frame set that matches the object's state information.

[0034] Step S402: If an object matching the state information of the object is detected in the direct preceding frame adjacent to the target frame in the preceding frame set, then the object is directly determined as the first object.

[0035] In this embodiment, the direct preceding frame can refer to the image frame in the preceding frame set that is directly adjacent to the preceding frame of the target frame.

[0036] Specifically, for any object in the target frame, based on the object's state information and the state information of objects in the preceding frame set, it is detected whether there is an object in the preceding frame set whose state information matches that object's state information. In the process of detecting whether there is an object in the preceding frame set whose state information matches that object's state information, a cost function can be used: The system uses a metric to measure the degree of matching between the state information of the current object and the state information of any object in the preceding frame set. Here, `cost` represents the cost; a higher value indicates a lower degree of matching between the current object and its corresponding object in the preceding frame. `IoU` is the intersection-over-union ratio, calculated based on the bounding box information of the current object and its corresponding object in the preceding frame, measuring the degree of spatial overlap between the two bounding boxes. A higher value indicates greater overlap between the two objects. This is the absolute difference between the motion direction of the object and the motion direction of the object in the previous frame. To ensure that the proportions are consistent, this difference is normalized to the [0,1] interval (because the maximum angle difference is 180°, dividing by 180° will normalize it). These are preset weighting coefficients used to adjust the importance of directional differences in the total cost.

[0037] If an object whose state information matches that of the target object is detected in the direct preceding frame adjacent to the target frame in the preceding frame set, that is, if an object whose cost meets a preset threshold is detected in the direct preceding frame, then the object is directly determined as the first object.

[0038] Step S403: If no object matching the object's state information is detected in the direct preceding frame, and an object matching the object's state information is detected in the image frames other than the direct preceding frame in the preceding frame set, then the state information of all objects matching the object's state information in the preceding frame set is input into the target prediction model, and the predicted state information of the object is obtained through the target prediction model.

[0039] Step S404: Compare the predicted state information of the object with the state information of the object. If the comparison is successful, the object is determined to be the first object.

[0040] In this embodiment, the target prediction model can refer to a deep learning model that has been trained to predict state information. After training, the model can predict the state information of the corresponding object in the target frame based on the state information of the object in the preceding frame. The predicted state information can refer to the state information of the corresponding object predicted by the target prediction model.

[0041] Specifically, if no object matching the state information of the object is detected in the direct preceding frame, and an object matching the state information of the object is detected in the image frames other than the direct preceding frame in the preceding frame set, then the state information of all objects matching the state information of the object in the preceding frame set is input into the target prediction model, and the predicted state information of the object is obtained through the target prediction model. The predicted state information of the object is then compared with the state information of the object. The comparison can be similarity comparison, etc. If the comparison is successful, the object is determined to be the first object.

[0042] Optionally, based on the state information of objects in the preceding frame set and the target frame, trajectory tracking is performed on each object in the target frame to obtain a second object in the target frame whose trajectory could not be tracked. This includes: for any object in the target frame, based on the object's state information and the state information of objects in the preceding frame set, detecting whether there is an object in the preceding frame set that matches the object's state information; if no object in the preceding frame set that matches the object's state information is detected, then the object is determined to be the second object.

[0043] Accordingly, after comparing the predicted state information of the object with the state information of the object, the method further includes: if the comparison fails, then the object is determined to be the second object.

[0044] In this embodiment, inter-frame object matching is performed by constructing a cost function that fuses position and motion direction, and a target prediction model is introduced to intelligently compensate for the association of discontinuous frames. This effectively solves the problems of tracking breakage and identity switching caused by high-speed swimming of fish, brief target occlusion, or image quality fluctuations. It not only relies on the matching of directly adjacent frames, but can also achieve trajectory re-association and recovery across several intermediate frames, thereby significantly improving the continuity and stability of tracking and ensuring that each individual fish in the group always maintains a unique identity throughout the sequence, thereby further improving the accuracy of fish counting.

[0045] In one embodiment, such as Figure 5 As shown, a fish counting method is provided. In step S205 above, for any object corresponding to an identifier, the fish validity of the object corresponding to the identifier is verified based on the movement direction of the object in the record frame of the object corresponding to the identifier and the movement direction of the recorded individual of the object corresponding to the identifier in the corresponding record frame, and the fish verification result is obtained. The method includes the following steps: Step S501: For any record frame of the object corresponding to the identifier, determine the overall motion direction of the object in the record frame based on the state information of the object in the record frame.

[0046] Step S502: Compare the motion direction of the individual record of the object corresponding to the identifier in the record frame with the overall motion direction of the object in the record frame to obtain the comparison result.

[0047] Step S503: Determine the number of record frames that pass the comparison. If the number meets the preset threshold, the fish verification result of the object corresponding to the identifier is determined to be passed. If the number does not meet the preset threshold, the fish verification result of the object corresponding to the identifier is determined to be failed.

[0048] In this embodiment, the overall motion direction can refer to the overall motion trend of all objects in the recording frame, and the preset threshold can refer to the number of times that a pre-set object must pass through to be determined as a fish.

[0049] Specifically, for any object corresponding to an identifier, all recording frames corresponding to that object are determined. For any recording frame, based on the state information of all objects in that recording frame, the overall motion direction of all objects in that recording frame is extracted using a density-based clustering algorithm. The motion direction of the individual recordings of the object corresponding to the identifier in that recording frame is compared with the overall motion direction of all objects in that recording frame to obtain a comparison result. For example, if the difference between the motion direction of the individual recordings and the overall motion direction is less than a preset value, the comparison result is determined to be passed. The number of passing records corresponding to that identifier is determined. If the number meets a preset threshold, the fish verification result of the object corresponding to the identifier is determined to be passed, that is, the object corresponding to the identifier is a real fish. If the number does not meet the preset threshold, the fish verification result of the object corresponding to the identifier is determined to be failed, that is, the object corresponding to the identifier is not a real fish, but may be static debris in the underwater environment such as corals, algae, and fishing nets that are similar to the visual characteristics of fish.

[0050] Optionally, based on the fish validation results of all target objects, the number of objects in the target fish group is obtained, including: Determine the number of objects corresponding to the identifiers that passed the fish verification, and set the number to the number of fish in the target fish group.

[0051] That is, count the number of real fish corresponding to the identifier. Since the identifier uniquely identifies the fish, the number is the number of fish in the target fish group.

[0052] In this embodiment, by comparing the direction of individual movement with the overall movement trend of the fish school, static debris such as corals and fishing nets, as well as interference objects with abnormal movement trajectories, are effectively eliminated, greatly reducing the false detection rate and thus improving the accuracy and reliability of the final fish school counting results.

[0053] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0054] In one embodiment, a fish counting device is provided, which corresponds one-to-one with the fish counting methods described in the above embodiments. For example... Figure 6 As shown, the fish counting device includes a data acquisition module 61, an identification module 62, a trajectory tracking module 63, a loop module 64, and a counting module 65. Detailed descriptions of each functional module are as follows: Data acquisition module 61 is used to acquire image frame sequences collected by sonar equipment from target fish groups, motion data collected by sonar equipment and environmental data of the environment in which the target fish groups are located, and preprocess each image frame in the image frame sequence according to the motion data and the environmental data to obtain a preprocessed image frame sequence. The recognition module 62 is used to perform object recognition on each image frame in the preprocessed image frame sequence to obtain the object in the corresponding image frame and the state information of the corresponding object. The state information includes detection box information and motion direction. The trajectory tracking module 63 is used to assign a unique identifier to each object in the first frame of the preprocessed image frame sequence, write the first frame into the preceding frame set, take the image frame directly adjacent to the first frame as the target frame, perform trajectory tracking on each object in the target frame according to the state information of the objects in the preceding frame set and the target frame, and obtain the first object in the target frame whose trajectory can be tracked and the second object whose trajectory cannot be tracked. For any first object, the identifier of the corresponding object in the preceding frame set is inherited, and for any second object, a new unique identifier is assigned to the second object. The loop module 64 is used to write the target frame into the preceding frame set, take the image frame directly adjacent to the target frame as the new target frame, and return to execute the step of tracing the trajectory of each object in the target frame according to the state information of the objects in the preceding frame set and the target frame, until all image frames are traversed to obtain the record frame and the record individual in the preprocessed image frame sequence for each identifier corresponding to the object. The counting module 65 is used to perform fish validity verification on the object corresponding to any identifier based on the movement direction of the object in the record frame of the object corresponding to the identifier and the movement direction of the recorded individual of the object corresponding to the identifier in the corresponding record frame, and obtain the fish verification result. Based on the fish verification results of all objects corresponding to the identifier, the number of fish in the target fish group is obtained.

[0055] Optionally, the data acquisition module 61 mentioned above includes: An image enhancement unit is configured to perform contrast enhancement on each image frame in the image frame sequence based on the environmental data, and to remove blur from each image in the image frame sequence based on the motion data, thereby obtaining an enhanced image frame sequence. An image denoising unit is used to perform wavelet denoising on the enhanced image frame sequence to obtain the preprocessed image frame sequence.

[0056] Optionally, the trajectory tracking module 63 mentioned above includes: The first detection unit is used to detect, for any object in the target frame, whether there is an object in the preceding frame set whose state information matches that of the object, based on the state information of the object and the state information of the objects in the preceding frame set. The first determining unit is configured to directly determine the object as the first object if it detects that there is an object in the direct preceding frame adjacent to the target frame in the preceding frame set that matches the state information of the object; The prediction unit is configured to, if it detects that there is no object matching the state information of the object in the direct preceding frame, and detects that there is an object matching the state information of the object in the image frames in the preceding frame set other than the direct preceding frame, input the state information of all objects matching the state information of the object in the preceding frame set into the target prediction model, and obtain the predicted state information of the object through the target prediction model; The second determining unit is used to compare the predicted state information of the object with the state information of the object. If the comparison is successful, the object is determined to be the first object.

[0057] Optionally, the trajectory tracking module 63 described above includes: The second detection unit is used to detect, for any object in the target frame, whether there is an object in the preceding frame set whose state information matches that of the object, based on the state information of the object and the state information of the objects in the preceding frame set. The third determining unit is configured to determine the object as the second object if it is detected that there is no object in the preceding frame set that matches the state information of the object; The fourth determining unit is used to determine that the object is the second object if the comparison fails.

[0058] Optionally, the counting module 65 includes: The fifth determining unit is used to determine the overall motion direction of the object in the recording frame based on the state information of the object in the recording frame for any recording frame of the object corresponding to the identifier; The comparison unit is used to compare the motion direction of the recorded individual object corresponding to the identifier in the recording frame with the overall motion direction of the object in the recording frame to obtain the comparison result; The sixth determining unit is used to determine the number of record frames whose comparison result is passed. If the number meets a preset threshold, the fish verification result of the object corresponding to the identifier is determined to be passed. If the number does not meet the preset threshold, the fish verification result of the object corresponding to the identifier is determined to be failed.

[0059] Optionally, the counting module 65 includes: The seventh determining unit is used to determine the number of objects corresponding to the identifiers whose fish verification results are passed; The eighth determining unit is used to determine the quantity as the number of fish in the target fish group.

[0060] Specific limitations regarding the fish counting device can be found in the limitations of the fish counting method described above, and will not be repeated here. Each module in the aforementioned fish counting device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0061] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and the database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores image frame sequences, motion data, and environmental data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a fish counting method.

[0062] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the fish counting method described in the above embodiment, for example... Figure 2 As shown in S201-S205, or Figures 3 to 5 As shown, to avoid repetition, it will not be described again here. Alternatively, the processor executes the computer program to implement the functions of each module / unit in this embodiment of the fish counting device, for example, Figure 6 The functions of the data acquisition module 61, recognition module 62, trajectory tracking module 63, loop module 64, and counting module 65 shown are not described again here to avoid repetition.

[0063] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the fish counting method described in the above embodiment, for example... Figure 2 As shown in S201-S205, or Figures 3 to 5As shown, to avoid repetition, it will not be described again here. Alternatively, the processor executes the computer program to implement the functions of each module / unit in this embodiment of the fish counting device, for example, Figure 6 The functions of the data acquisition module 61, recognition module 62, trajectory tracking module 63, loop module 64, and counting module 65 shown are not described again here to avoid repetition.

[0064] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0065] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0066] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for counting fish schools, characterized in that, include: A sequence of image frames collected from a target fish group using a sonar device is obtained. Motion data collected by the sonar device and environmental data of the target fish group are used. Each image frame in the image frame sequence is preprocessed based on the motion data and the environmental data to obtain a preprocessed image frame sequence. For each image frame in the preprocessed image frame sequence, object recognition is performed to obtain the object in the corresponding image frame and the state information of the corresponding object. The state information includes detection box information and motion direction. Assign a unique identifier to each object in the first frame of the preprocessed image frame sequence, write the first frame into the preceding frame set, and take the image frame directly adjacent to the first frame as the target frame. Based on the state information of the objects in the preceding frame set and the target frame, perform trajectory tracking on each object in the target frame to obtain the first object whose trajectory can be tracked and the second object whose trajectory cannot be tracked in the target frame. For any first object, inherit the identifier of the corresponding object in the preceding frame set. For any second object, reassign a unique identifier to the second object. Write the target frame into the preceding frame set, take the image frame directly adjacent to the target frame as the new target frame, and return to execute the step of tracing the trajectory of each object in the target frame according to the state information of the objects in the preceding frame set and the target frame, until all image frames are traversed to obtain the record frame and the record individual in the preprocessed image frame sequence for each identifier corresponding to the object. For any object corresponding to an identifier, the fish validity of the object is verified based on the movement direction of the object in the record frame of the object corresponding to the identifier and the movement direction of the recorded individual of the object corresponding to the identifier in the corresponding record frame. The fish verification result is obtained, and the number of fish in the target fish group is obtained based on the fish verification results of all objects corresponding to identifiers.

2. The fish counting method according to claim 1, characterized in that, The step of preprocessing each image frame in the image frame sequence based on the motion data and the environmental data to obtain a preprocessed image frame sequence includes: Based on the environmental data, each image frame in the image frame sequence is contrast-enhanced, and based on the motion data, each image frame in the image frame sequence is blurred to obtain an enhanced image frame sequence. Wavelet denoising is performed on the enhanced image frame sequence to obtain the preprocessed image frame sequence.

3. The fish counting method according to claim 1, characterized in that, The step of tracking the trajectory of each object in the target frame based on the state information of the objects in the preceding frame set and the target frame to obtain the first object in the target frame whose trajectory can be tracked includes: For any object in the target frame, based on the object's state information and the state information of objects in the preceding frame set, it is detected whether there is an object in the preceding frame set that matches the object's state information; If an object matching the state information of the object is detected in the direct preceding frame adjacent to the target frame in the preceding frame set, then the object is directly determined to be the first object; If it is detected that there is no object matching the state information of the object in the direct preceding frame, and it is detected that there is an object matching the state information of the object in the image frames in the preceding frame set other than the direct preceding frame, then the state information of all objects matching the state information of the object in the preceding frame set is input into the target prediction model, and the predicted state information of the object is obtained through the target prediction model. The predicted state information of the object is compared with the state information of the object. If the comparison is successful, the object is determined to be the first object.

4. The fish counting method according to claim 3, characterized in that, The step of tracking the trajectory of each object in the target frame based on the state information of the objects in the preceding frame set and the target frame to obtain a second object in the target frame whose trajectory could not be tracked includes: For any object in the target frame, based on the object's state information and the state information of objects in the preceding frame set, it is detected whether there is an object in the preceding frame set that matches the object's state information; If it is detected that there is no object in the preceding frame set that matches the state information of the object, then the object is determined to be the second object; Accordingly, after comparing the predicted state information of the object with the state information of the object, the method further includes: If the comparison fails, the object is determined to be the second object.

5. The fish counting method according to claim 1, characterized in that, For any object corresponding to an identifier, the fish validity verification is performed on the object corresponding to the identifier based on the motion direction of the object in the record frame and the motion direction of the recorded individual of the object corresponding to the identifier in the corresponding record frame, to obtain the fish verification result, including: For any record frame of the object corresponding to the identifier, the overall motion direction of the object in the record frame is determined based on the state information of the object in the record frame; The motion direction of the individual record of the object corresponding to the identifier in the record frame is compared with the overall motion direction of the object in the record frame to obtain the comparison result; The number of record frames that pass the comparison is determined. If the number meets a preset threshold, the fish verification result of the object corresponding to the identifier is determined to be passed. If the number does not meet the preset threshold, the fish verification result of the object corresponding to the identifier is determined to be failed.

6. The fish counting method according to claim 5, characterized in that, The step of obtaining the number of objects in the target fish group based on the fish verification results of all target objects includes: Determine the number of objects corresponding to the identifiers for which the fish verification result is passed; The quantity refers to the number of fish in the target fish group.

7. A fish counting device, characterized in that, include: The data acquisition module is used to acquire the image frame sequence collected by the sonar device on the target fish group, the motion data collected by the sonar device and the environmental data of the environment in which the target fish group is located, and to preprocess each image frame in the image frame sequence according to the motion data and the environmental data to obtain the preprocessed image frame sequence. The recognition module is used to perform object recognition on each image frame in the preprocessed image frame sequence to obtain the object in the corresponding image frame and the state information of the corresponding object. The state information includes detection box information and motion direction. The trajectory tracking module is used to assign a unique identifier to each object in the first frame of the preprocessed image frame sequence, write the first frame into the preceding frame set, take the image frame directly adjacent to the first frame as the target frame, perform trajectory tracking on each object in the target frame according to the state information of the objects in the preceding frame set and the target frame, and obtain the first object in the target frame whose trajectory can be tracked and the second object whose trajectory cannot be tracked. For any first object, the identifier of the corresponding object in the preceding frame set is inherited, and for any second object, a new unique identifier is assigned to the second object. The loop module is used to write the target frame into the preceding frame set, take the image frame directly adjacent to the target frame as the new target frame, and return to execute the step of tracing the trajectory of each object in the target frame according to the state information of the objects in the preceding frame set and the target frame, until all image frames are traversed to obtain the record frame and the record individual in the preprocessed image frame sequence for each identifier corresponding to the object. The counting module is used to perform fish validity verification on the object corresponding to any identifier based on the movement direction of the object in the record frame of the object corresponding to the identifier and the movement direction of the recorded individual of the object corresponding to the identifier in the corresponding record frame, to obtain the fish verification result, and to obtain the number of fish in the target fish group based on the fish verification results of all the objects corresponding to the identifiers.

8. The fish counting device according to claim 7, characterized in that, The data acquisition module includes: An enhancement unit is configured to perform contrast enhancement on each image frame in the image frame sequence based on the environmental data, and to remove blur from each image in the image frame sequence based on the motion data, thereby obtaining an enhanced image frame sequence. A denoising unit is used to perform wavelet denoising on the enhanced image frame sequence to obtain the preprocessed image frame sequence.

9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the fish counting method as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the fish counting method as described in any one of claims 1 to 6.

Images