A fish track monitoring system based on edge computing

Abstract

The present invention discloses a fish school trajectory monitoring system based on edge computing, including a user terminal device and at least one edge device, the edge device including a camera and an edge computing mainboard, the camera is used to collect fish school movement video, the edge computing mainboard includes: a data receiving unit, used to receive fish school movement video and perform image extraction; a detection unit, used to deploy a pre-trained YOLOv5 model and detect the image; a data analysis unit, used to obtain the centroid coordinates of the fish school for each frame image, and perform the following operations: calculate the total displacement of the fish school in the Kth cycle; determine whether the difference between the total displacement of the fish school in two adjacent cycles is greater than a preset threshold; the user terminal device is used to receive the warning information issued by each edge computing mainboard. The system realizes abnormal behavior monitoring of fish schools based on edge computing, saves server computing resources, improves the real-time and accuracy of monitoring, and has low labor costs, convenient deployment, and a wide range of applications.

Description

Technical Field

[0001] The present invention belongs to the technical field of aquaculture, and in particular relates to a fish school trajectory monitoring system based on edge computing. Background Art

[0002] In the process of fish aquaculture, water quality, temperature, and the health of fish are all very important parts. Under normal conditions, the movement trajectory of fish is generally stable. When the environment changes or the health of the fish is poor, the trajectory of the fish will be more chaotic, which means that an abnormal situation has occurred and needs to be discovered and dealt with in time to avoid major losses.

[0003] In recent years, the technology of fish aquaculture has been developing in the direction of intelligence and automation. However, existing enterprises often use traditional machine learning combined with digital image processing algorithms to monitor fish trajectories, which has a low accuracy rate for fish trajectory monitoring. In addition, since a large amount of calculated data needs to be uploaded to the server for calculation, it not only increases the time for fish trajectory monitoring and makes it difficult to meet real-time requirements, but also occupies precious server resources. Or they still rely on a series of traditional breeding modes such as pure manual or manual plus sensors (such as infrared sensors, ultrasonic sensors, etc.) to operate. Due to personal subjective factors and high sensor failure rate and easy aging, the accuracy of fish trajectory monitoring in fish aquaculture is low. At the same time, due to the increasing cost of manpower and sensor configuration, the breeding cost is too high.

[0004] With the continuous progress in the field of edge computing, the cost of edge computing equipment is getting lower and lower. Combined with the rapid development of deep learning in the fields of computer vision and image processing, more lightweight and fast models have been proposed, and the requirements for computer performance have been reduced. This makes it possible to monitor the trajectory of fish schools by combining edge computing and computer vision methods instead of traditional manual or sensor methods. Therefore, this application proposes a low-cost, high-accuracy and real-time fish track monitoring system based on edge computing. Summary of the invention

[0005] The purpose of the present invention is to propose a fish trajectory monitoring system based on edge computing to address the above-mentioned problems. The system can accurately monitor abnormal behavior of fish schools based on edge computing, reduce labor costs in the breeding process, save server computing resources, improve the real-time performance of fish trajectory prediction, and is easy to deploy and has a wide range of applications.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] The present invention proposes a fish track monitoring system based on edge computing, comprising a user terminal device and at least one edge device, wherein:

[0008] The edge device includes a camera and an edge computing mainboard. The camera is used to collect fish movement videos. The edge computing mainboard includes a data receiving unit, a detection unit, and a data analysis unit connected in sequence, wherein:

[0009] A data receiving unit, used for receiving the fish school movement video and performing image extraction;

[0010] The detection unit is used to deploy the pre-trained YOLOv5 network model and detect the image through the YOLOv5 network model to obtain the detection frame of each fish and the corresponding position information [x, y, w, h], where (x, y) is the center coordinate of the detection frame, that is, the center of mass coordinate of the fish, w is the width of the detection frame, and h is the height of the detection frame;

[0011] The data analysis unit is used to obtain the coordinates of the fish school's centroid in each frame of the image and perform the following operations:

[0012] Calculate the total displacement S of the school of fish in the Kth cycle K , satisfying the following formula: Among them, (x i ,y i ) is the coordinate of the fish school centroid of the i-th frame image in the K-th period, n is the total number of images in the K-th period, K = 1, 2, 3, ...;

[0013] Determine whether the difference between the total displacement of the fish school in two adjacent cycles is greater than a preset threshold. If so, the fish school movement trajectory is considered abnormal and a warning message is sent to the user terminal device. Otherwise, the fish school movement trajectory is considered normal and no warning message is sent to the user terminal device.

[0014] User terminal equipment is used to receive warning information issued by the data analysis unit of each edge computing mainboard.

[0015] Preferably, the user terminal device is a server.

[0016] Preferably, the coordinates of the centroid of the fish school are obtained by using a weighted average method based on the position information of each fish in each frame of the image.

[0017] Preferably, the data receiving unit is further used to preprocess the extracted image, wherein the preprocessing includes sequentially compressing the image and removing the background based on a mixed Gaussian model.

[0018] Preferably, the preset threshold is 25%.

[0019] Preferably, the edge computing motherboard is a jetson nano motherboard.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: the system uses a camera and an edge computing motherboard to complete the deployment of edge devices. The edge computing motherboard detects fish schools by deploying a lightweight YOLOv5 model, and simulates the fish school trajectory according to the change of the fish school's center of mass to determine whether there is abnormal behavior. Before detection, the image is preprocessed to remove the influence of image background change factors on the model prediction accuracy, thereby solving the problem that traditional algorithms are difficult to accurately monitor the fish school trajectory, achieving a more accurate trajectory monitoring effect, and reducing the labor cost in the breeding process. At the same time, it also avoids uploading a large amount of data to the server for model inference calculations, saving precious server resources, and improving the real-time prediction of fish school trajectory in the actual breeding process. It is easy to deploy and has a wide range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a structural schematic diagram of the fish school trajectory monitoring system based on edge computing of the present invention;

[0022] Figure 2 It is a flow chart of the fish track monitoring system based on edge computing of the present invention;

[0023] Figure 3 The original image extracted from the fish school motion video by the data receiving unit of the present invention;

[0024] Figure 4 Outputting result graphs for the detection unit of the present invention;

[0025] Figure 5 It is a fish school movement trajectory diagram under normal conditions of the present invention;

[0026] Figure 6 This is a diagram of the movement trajectory of a school of fish under abnormal conditions of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0029] like Figure 1-6As shown, a fish track monitoring system based on edge computing includes a user terminal device and at least one edge device, wherein:

[0030] The edge device includes a camera and an edge computing mainboard. The camera is used to collect fish movement videos. The edge computing mainboard includes a data receiving unit, a detection unit, and a data analysis unit connected in sequence, wherein:

[0031] A data receiving unit, used for receiving the fish school movement video and performing image extraction;

[0032] The detection unit is used to deploy the pre-trained YOLOv5 network model and detect the image through the YOLOv5 network model to obtain the detection frame of each fish and the corresponding position information [x, y, w, h], where (x, y) is the center coordinate of the detection frame, that is, the center of mass coordinate of the fish, w is the width of the detection frame, and h is the height of the detection frame;

[0033] The data analysis unit is used to obtain the coordinates of the fish school's centroid in each frame of the image and perform the following operations:

[0034] Calculate the total displacement S of the school of fish in the Kth cycle K , satisfying the following formula: Among them, (x i ,y i ) is the coordinate of the fish school centroid of the i-th frame image in the K-th period, n is the total number of images in the K-th period, K = 1, 2, 3, ...;

[0035] Determine whether the difference between the total displacement of the fish school in two adjacent cycles is greater than a preset threshold. If so, the fish school movement trajectory is considered abnormal and a warning message is sent to the user terminal device. Otherwise, the fish school movement trajectory is considered normal and no warning message is sent to the user terminal device.

[0036] User terminal equipment is used to receive warning information issued by the data analysis unit of each edge computing mainboard.

[0037] Among them, the detection unit is used to deploy the pre-trained YOLOv5 network model. For the training of the YOLOv5 network model, the image set extracted from the fish activity video taken by the camera can be used as the data set, such as dividing it into a training set and a test set in a ratio of 8:2. Label the training set and then train it. The trained model is tested with the test set. If the recognition accuracy reaches the preset accuracy (such as 95%), the model training is considered complete, the model parameter file is saved, and the training is stopped to obtain the pre-trained YOLOv5 network model. Otherwise, continue training until the recognition accuracy reaches more than 95%. The pre-trained YOLOv5 network model is directly deployed on the edge computing motherboard to monitor the activity trajectory of the fish school in real time, which is convenient to deploy.

[0038] The fish movement trajectory is then judged through the data analysis unit. When the difference between the total displacement of the fish in two adjacent cycles is less than or equal to the preset threshold, there is no need for the user terminal device to send out warning information, and the fish movement trajectory continues to be monitored in real time. When the difference between the total displacement of the fish in two adjacent cycles is greater than the preset threshold, the fish movement trajectory is considered abnormal, and a warning message is sent to the user terminal device. The preset threshold can be designed according to actual needs. The user terminal device can be connected to the data analysis units of multiple edge computing motherboards at the same time. If the edge device includes m, the user terminal device is used to receive warning information issued by m edge devices.

[0039] Specifically, the total displacement of the school of fish in each cycle is calculated as follows: In this embodiment, each cycle (such as 100s) includes 100 frames of images. According to the centroid position of the school of fish in each frame of the image, the position distance between two adjacent frames is calculated, and the total displacement of the school of fish in 100 frames S is obtained after adding them. K Repeat this operation in the next cycle to obtain the second fish school displacement sum S K+1 , when S K and S K+1 When the difference does not exceed the preset threshold, the fish school is considered to be in a normal state, otherwise it is considered to be in an abnormal state. K Calculation formula: The fish school motion trajectory diagram is obtained by connecting the fish school mass center of each frame image in each cycle. The coordinates of the fish school mass center of each frame image (x i ,y i ) are multiplied by the width and height of the image to convert into pixel coordinates, such as Figure 5 , 6 As shown, Figure 5 This is the trajectory diagram of the fish school under normal behavior. Figure 6 This is the trajectory diagram of the fish school under abnormal behavior.

[0040] The system uses cameras and edge computing motherboards to complete the deployment of edge devices. The edge computing motherboard detects fish schools by deploying a lightweight YOLOv5 model, and simulates the fish school trajectory according to the change of the fish school's center of mass to determine whether there is abnormal behavior. It solves the problem that traditional algorithms are difficult to accurately monitor the fish school's trajectory, achieves more accurate trajectory monitoring effects, and reduces labor costs in the breeding process. At the same time, it directly calculates and processes local data before transmitting it to the user terminal device, avoiding uploading large amounts of data to the server for model inference operations, saving valuable server resources, avoiding many unnecessary expenses, and improving the real-time prediction of fish school trajectories in the actual breeding process. It is easy to deploy, helps to reduce breeding costs, and has a wide range of applications.

[0041] In one embodiment, the user terminal device is a server. Or it can also be a mobile phone, computer, tablet or other device for receiving and viewing warning information. The edge computing motherboard and the user terminal device can be wirelessly connected (such as WIFI) or wired, which is a prior art and will not be described here. After the user views the warning information, the abnormal situation of the fish school can be further processed in time (such as feeding, oxygenation, water change, etc.), which can largely avoid the losses caused by the untimely discovery of abnormalities, improve the breeding quality and survival rate, and is more efficient and less costly than traditional methods, especially suitable for large-scale deployment of large fish aquaculture enterprises.

[0042] In one embodiment, the coordinates of the centroid of the school of fish are obtained by weighted averaging based on the position information of each fish in each frame of image, or can be calculated by mathematical averaging.

[0043] In one embodiment, the data receiving unit is also used to preprocess the extracted image, and the preprocessing is to compress the image in turn and remove the background based on the mixed Gaussian model. The image is compressed according to a preset ratio, such as compressing the image width and height to 0-1 to obtain a corresponding compressed image. The image is then detected by the YOLOv5 network model to obtain the detection frame of each fish and the corresponding position information [x, y, w, h], where (x, y) is the center coordinate of the detection frame after image compression, w is the width of the detection frame divided by the width of the image after compression, and h is the height of the detection frame divided by the height of the image after compression. The prior art is difficult to meet the real underwater fish target detection requirements when the background light of the picture changes. The present application uses a background removal method based on a mixed Gaussian model to distinguish the foreground and background according to the pixel change rate, and preprocesses the image before inputting the YOLOv5 model, thereby removing the influence of the image background change factor on the model prediction accuracy, improving the accuracy of fish target detection, and obtaining a more accurate trajectory monitoring result. The background removal method based on the mixed Gaussian model is a common method in the prior art and will not be repeated here.

[0044] In one embodiment, the preset threshold is 25%.

[0045] In one embodiment, the edge computing motherboard is a jetson nano motherboard. The edge computing motherboard adopts a jetson nano motherboard, which has a higher computing speed, or may also adopt a motherboard in the prior art.

[0046] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0047] The above-described embodiments only express the more specific and detailed embodiments described in this application, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of this application, which all belong to the protection scope of this application. Therefore, the protection scope of the patent application shall be based on the attached claims.

Claims

1. A fish track monitoring system based on edge computing, characterized by: The fish track monitoring system based on edge computing includes a user terminal device and at least one edge device, wherein: The edge device includes a camera and an edge computing mainboard, the camera is used to collect fish movement video, and the edge computing mainboard includes a data receiving unit, a detection unit and a data analysis unit connected in sequence, wherein: The data receiving unit is used to receive the fish school motion video and perform image extraction; The detection unit is used to deploy a pre-trained YOLOv5 network model, and detect the image through the YOLOv5 network model to obtain a detection frame of each fish and corresponding position information [x, y, w, h], where (x, y) is the center coordinate of the detection frame, that is, the center of mass coordinate of the fish, w is the width of the detection frame, and h is the height of the detection frame; The data analysis unit is used to obtain the coordinates of the fish school's centroid in each frame of image and perform the following operations: Calculate the total displacement of the school of fish in the Kth period , satisfying the following formula: ,in,( is the coordinate of the fish school centroid of the i-th frame image in the K-th period, n is the total number of images in the K-th period, K=1,2,3,…; Determine whether the difference between the total displacements of the fish school in two adjacent cycles is greater than a preset threshold value. If so, the fish school movement trajectory is considered abnormal, and a warning message is sent to the user terminal device. Otherwise, the fish school movement trajectory is considered normal, and no warning message is sent to the user terminal device. The centroid coordinates of the fish school are obtained by using a weighted average method based on the position information of each fish in each frame of the image; The user terminal device is used to receive the warning information issued by the data analysis unit of each edge computing mainboard.

2. The fish school trajectory monitoring system based on edge computing as claimed in claim 1, characterized in that: The user terminal device is a server.

3. The fish school trajectory monitoring system based on edge computing as claimed in claim 1, characterized in that: The data receiving unit is also used to preprocess the extracted image, and the preprocessing is to compress the image in sequence and remove the background based on a mixed Gaussian model.

4. The fish school trajectory monitoring system based on edge computing as claimed in claim 1, characterized in that: The preset threshold is 25%.

5. The fish school trajectory monitoring system based on edge computing as claimed in claim 1, characterized in that: The edge computing mainboard is a jetson nano mainboard.

Images