Method and system for monitoring stress status of farmed fish, device and medium

Abstract

The present application relates to a kind of fish stress state monitoring method and system for factory farming, equipment and medium.The method comprises: obtaining underwater fish video data of factory farming by camera device;Fish in video data is detected and identified using full convolution neural network algorithm;According to the relative pixel position of fish in video data, fish is classified;And the water layer distribution of fish is quantified, and stress line is calculated according to the water layer distribution of fish.The present application uses the first thirteen layers of VGG-16 as feature extraction backbone network, establishes detection module based on multi-scale convolution kernel for target positioning and identification, fuses deep information and shallow information, solves the problem of multi-scale and occlusion in breeding environment, at the same time, the number of each type of fish in input video is visualized, which is convenient for aquaculture personnel to analyze the dynamic change of fish behavior, and provides a non-invasive, efficient and intelligent tool for real-time monitoring of fish.

Description

Technical Field

[0001] The present invention relates to the technical field of aquatic animal status monitoring methods, and in particular to a method, system, equipment and medium for monitoring the stress status of fish used in factory farming. Background Art

[0002] With the continuous development of the aquaculture industry, automated monitoring of all aspects of factory farming is particularly important to ensure a safe farming environment, healthy fish living conditions and fish quality, especially monitoring and evaluating fish behavior.

[0003] In aquaculture, stressors to fish include improper handling during tank separation, deteriorating water quality during transportation, and excessive stocking density. These can all cause significant stress to fish, leading to bottom-dwelling behavior. Changes in fish behavior can be combined with changes in their distribution across the water layer to assess their status.

[0004] Traditional methods for observing fish stress, such as calculating the time it takes for fish to move from the bottom to the top or measuring cortisol concentrations, are labor-intensive and can cause damage to the fish. Therefore, a low-cost, non-destructive method for monitoring fish stress is needed. Summary of the Invention

[0005] In response to the problems in the prior art, the present invention provides a method and system, equipment and medium for monitoring the stress status of fish used in factory farming.

[0006] In a first aspect, the present invention provides a method for monitoring stress status of fish used in factory farming, comprising:

[0007] Obtain video data of factory-farmed underwater fish through a camera device;

[0008] Detecting and identifying fish in the video data using a fully convolutional neural network algorithm;

[0009] classifying the fish species according to their relative pixel positions in the video data; and

[0010] The water layer distribution of the fish is quantified, and a stress line is calculated based on the water layer distribution of the fish.

[0011] Furthermore, the detecting and identifying fish in the video data using a fully convolutional neural network algorithm includes:

[0012] Use the first 13 layers of VGG-16 as the backbone network to extract image features; and

[0013] A detection module based on multi-scale convolution kernel is established for target positioning and recognition.

[0014] Furthermore, the establishment of a detection module based on a multi-scale convolution kernel for target positioning and recognition includes:

[0015] Extract feature maps from the downsampling points and concatenate and fuse them to obtain feature maps of three scales; and

[0016] The feature maps of the three scales are input into the corresponding three columns of multi-scale convolution kernels for fusion, and finally the detection results of large, medium and small scale targets are obtained.

[0017] Furthermore, classifying the fish according to relative pixel positions of the fish in the video data includes:

[0018] For detection results located at 20%-50% pixel positions of the video data, classifying the fish as pelagic fish; and

[0019] For the detection results located at 80%-50% pixel positions of the video data, the fish are classified as bottom-layer fish.

[0020] Furthermore, the quantification of the water layer distribution of the fish and calculation of the stress line according to the water layer distribution of the fish include:

[0021] Obtain the number of each type of fish per unit time in the video data;

[0022] The optimal ratio of bottom fish under stress and the optimal ratio of bottom fish under no stress are obtained by K-means clustering method; and

[0023] The coercion line is obtained by calculation.

[0024] Furthermore, obtaining the number of each type of fish per unit time in the video data includes:

[0025] Determine the total number of frames per unit time;

[0026] Sum the number of each type of fish in each frame of image per unit time; and

[0027] Take the average of the summed results as the number of each type of fish in the current unit time.

[0028] Furthermore, the calculation to obtain the coercion line includes:

[0029] The coercion line is obtained by the following formula:

[0030]

[0031] Wherein, x is the coercion line, x 1 is the optimal ratio of bottom fish in the no-stress state, x 2 is the optimal ratio of bottom fish under the stress state.

[0032] In a second aspect, the present invention provides a fish stress status monitoring system for factory farming, comprising:

[0033] A video data acquisition module is used to acquire video data of factory-farmed underwater fish through a camera device;

[0034] a detection and recognition module for detecting and identifying fish in the video data using a fully convolutional neural network algorithm;

[0035] a classification module, configured to classify the fish according to relative pixel positions of the fish in the video data; and

[0036] The stress line calculation module is used to quantify the water layer distribution of the fish and calculate the stress line according to the water layer distribution of the fish.

[0037] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the method for monitoring the stress status of fish in factory farming as described in the first aspect are implemented.

[0038] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method for monitoring the stress status of fish in factory farming as described in the first aspect are implemented.

[0039] The present invention provides a method, system, device, and medium for monitoring the stress status of fish in factory farming. By utilizing the first thirteen layers of VGG-16 as the feature extraction backbone network, a detection module based on multi-scale convolution kernels is established for target positioning and identification, integrating deep information with shallow information to solve the multi-scale and occlusion problems that arise in the farming environment. At the same time, the present invention visualizes the number of each type of fish in the input video, facilitating farmers' analysis of dynamic changes in fish behavior, and providing a non-invasive, efficient, and intelligent tool for real-time monitoring of fish. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] A more complete understanding of exemplary embodiments of the present invention may be obtained by referring to the following drawings:

[0041] Figure 1 A flow chart of a method for monitoring stress status of fish in factory farming provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of the structure of a fully convolutional neural network unit provided in an embodiment of the present invention;

[0043] Figure 3 and Figure 4 A graph showing the number of each type of fish and the proportion of fish in the bottom layer changing over time, provided by an embodiment of the present invention;

[0044] Figure 5 A structural block diagram of a fish stress status monitoring system for factory farming provided by an embodiment of the present invention; and

[0045] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0047] Figure 1 This is a flow chart of a method for monitoring stress status of fish in factory farming provided by an embodiment of the present invention.

[0048] Reference Figure 1 The method for monitoring stress status of fish used in factory farming includes the following steps:

[0049] S101: Obtain video data of factory-farmed underwater fish through a camera device;

[0050] S103: Detect and identify fish in video data using a fully convolutional neural network algorithm;

[0051] S105: Classifying the fish according to their relative pixel positions in the video data; and

[0052] S107: Quantify the water layer distribution of the fish, and calculate the stress line based on the water layer distribution of the fish.

[0053] In the embodiment, the stress line is based on the water layer distribution of fish in the non-stressed and stressed states, and the proportion of fish in the lower layer under the stressed state is obtained by a clustering method.

[0054] In the embodiment, specifically, step S101 includes using a camera device to obtain video data of factory-farmed underwater fish, where the video data includes video data of different fish pond environments, different light conditions, and the like.

[0055] In the embodiment, specifically, step S103 includes using the first 13 layers of VGG-16 as the backbone network to extract image features; and establishing a detection module based on multi-scale convolution kernels for target positioning and recognition: extracting feature maps from the downsampling respectively and splicing and fusing them to obtain feature maps of three scales; inputting the feature maps of the three scales into the corresponding three columns of multi-scale convolution kernels for fusion, and finally obtaining detection results of large, medium and small scale targets.

[0056] Reference Figure 2 , where k=3, c=64, d=1 for the convolutional layers of the backbone network (layers 1-2), and k=2 for max pooling; k=3, c=128, d=1 for the convolutional layers of the layers 3-4, and k=2 for max pooling; k=3, c=256, d=1 for the convolutional layers of the layers 5-7, and k=2 for max pooling; k=3, c=512, d=1 for the convolutional layers of the layers 8-10; k=3, c=512, d=1 for the convolutional layers of the layers 11-13, and k=2 for max pooling; where k is the convolution kernel size, c is the number of channels, and d is the dilation rate.

[0057] The detection module based on multi-scale convolution kernels extracts feature maps from the backbone network at 8x, 16x, and 32x downsampling, and then fuses them into three columns of multi-scale convolution kernels. The feature maps of the 8x and 16x downsampling are then concatenated and fused with the feature maps of the upsampling results to generate new feature maps.

[0058] Among them, the three columns of multi-scale convolution kernels include three parallel convolution networks, namely, the L column, using large-scale convolution kernels: 7×7, 5×5, 5×5, 5×5; the M column, using medium-scale convolution kernels: 5×5, 3×3, 3×3, 3×3; the S column, using small-scale convolution kernels: 3×3, 1×1, 1×1, 1×1; the spliced ​​feature map is input into the three-column convolutional neural network, and finally the detection results of large, medium and small scale targets are obtained.

[0059] In an embodiment, the fully convolutional neural network is trained as follows:

[0060] (1) The input video data is scaled to a video image of size 416×416, and each convolutional layer of the constructed backbone network is connected to a ReLU layer;

[0061] (2) The fully convolutional neural network learns the texture features of the image, and finally outputs three scale feature maps with sizes of 13×13, 26×26, and 52×52 respectively through three columns of convolutional neural networks;

[0062] (3) After training and testing, the fully convolutional neural network algorithm is obtained.

[0063] In an embodiment, the sample video data is one hour of video data obtained after fish are moved from a fish breeding pond to two new fish breeding ponds, one of which is treated with nicotine and the other is left untreated.

[0064] In the embodiment, specifically, step S105 includes determining the relative pixel position of the detection result in the input video and classifying the fish. If the detection result is between 20% and 50% of the pixel position in the input video, it is determined to be an upper layer fish and marked as Class A fish; if the detection result is between 80% and 50% of the pixel position in the input video, it is determined to be a lower layer fish and marked as Class B fish.

[0065] In the embodiment, specifically, step S107 includes calculating the number of each type of fish and the proportion of bottom-layer fish per unit time in the video data, and calculating the optimal bottom-layer fish ratio under stress and no stress using the K-means algorithm, thereby obtaining the stress line size and the time it takes for fish to move from the bottom layer to the upper layer. Specifically,

[0066] First, based on the fish classification results obtained in step S105, the number of each type of fish per unit time in the video data is calculated as follows:

[0067] (1) Determine the total number of frames per unit time as follows:

[0068] frames = fps * 60

[0069] Where frams is the total number of frames per minute (unit time); fps is the frame rate of the video data.

[0070] (2) Sum the number of each type of fish in each frame of image per minute, as follows:

[0071]

[0072] in, is the sum of the number of type A fish in each frame image in the jth minute; is the number of fish of type A in the image frame at the jth minute and the i-th frame.

[0073]

[0074] in, is the sum of the number of B-type fish in each frame of image in the jth minute; is the number of fish of category B in the image of frame i at the jth minute.

[0075] (3) Take the average value as the number of each type of fish in the current minute, as shown below:

[0076]

[0077] in, is the average number of fish of type A in the jth minute; is the sum of the number of Class A fish in each frame image in the jth minute; frams is the total number of frames per minute.

[0078]

[0079] in, is the average number of Class B fish in the hth minute; is the sum of the number of Class B fish in each frame image in the jth minute; frams is the total number of frames per minute.

[0080] Then, based on the number of each type of fish per unit time in the video data (i.e., the number of type A fish and the number of type B fish per unit time), the proportion of bottom fish (i.e., type B fish) per unit time is obtained as follows:

[0081]

[0082] Where P is the proportion of Class B fish to all fish in the jth minute; is the average number of Class A fish in the jth minute; is the average number of Class B fish in the jth minute.

[0083] Finally, based on the proportion of Class B fish per unit time, the K-means algorithm is used to calculate the optimal proportion of bottom-layer fish under stress and the optimal proportion of bottom-layer fish under no stress, thereby obtaining the size of the stress line and the time it takes for fish to move from the bottom layer to the upper layer, as follows:

[0084] (1) The current unit time and the proportion of fish in the lower layer form a key-value pair<k,v> , where k represents the kth minute, and v represents the proportion of bottom fish corresponding to the kth minute;

[0085] (2) The optimal ratio of bottom fish in the non-stress state is obtained by K-means clustering method, which is recorded as the seed point x 1 , and take a value greater than or equal to the seed point x 1 The maximum k value is marked as the time t taken by fish to move from the lower layer to the upper layer under no stress 1Similarly, the optimal ratio of bottom fish under stress can be obtained, which is recorded as seed point x 2 , and take a value less than or equal to x 2 The minimum k value of the seed point is recorded as the time t taken by fish to move from the lower layer to the upper layer under stress 2 ;

[0086] (3) Calculate the size of the coercion line as follows:

[0087]

[0088] By quantifying the underwater distribution of fish, when the proportion of fish in the lower layer exceeds the stress line and the time exceeds t 1 , at t 1 to t 2 During this period, breeders can perform relevant operations to relieve the stress on the young fish, so that the young fish can maintain homeostasis and good health.

[0089] In order to enable the aquaculture personnel to observe and analyze the stress state of the fish more clearly, a visual display can be used. Figure 3 and Figure 4 A graph showing the change in the number of each type of fish and the proportion of bottom-layer fish over time provided by an embodiment of the present invention is shown.

[0090] exist Figure 3 and Figure 4 In , the horizontal axis represents time, that is, the total duration of the video data, which can be obtained by the following formula:

[0091]

[0092] The unit of the total duration of the video data is minutes, nframs is the total number of frames of the video data, and fps is the frame rate of the video data. In addition, video data with frame images less than one minute will be discarded.

[0093] in addition, Figure 3 and Figure 4 The vertical axis represents the number of fish of type A and type B per unit time in the video data, and the proportion of fish of type B per unit time in the video data. The number of fish of type A and type B per unit time in the video data corresponds to the number of fish of type A and type B obtained in step S107 described in the above reference embodiment. and And the proportion of Class B fish per unit time in the video data corresponds to P obtained in step S107 described in the above reference embodiment.

[0094] Figure 5 This is a structural diagram of a fish stress status monitoring system for factory farming provided by an embodiment of the present invention. Figure 5, the system 500 includes:

[0095] The video data acquisition module 501 is used to acquire video data of factory-farmed underwater fish through a camera device;

[0096] a detection and identification module 503 for detecting and identifying fish in the video data using a fully convolutional neural network algorithm;

[0097] a classification module 505 for classifying the fish according to relative pixel positions of the fish in the video data; and

[0098] The stress line calculation module 507 is used to quantify the water layer distribution of the fish and calculate the stress line according to the water layer distribution of the fish.

[0099] As can be seen from the above, each module 501 to 507 of the system 500 can respectively execute each step of the monitoring method described with reference to the above embodiment, and the details thereof will not be described here again.

[0100] From the above, it can be seen that the present invention provides a method and system for monitoring the stress status of fish for factory farming. By using the first thirteen layers of VGG-16 as the feature extraction backbone network, a detection module based on multi-scale convolution kernels is established for target positioning and recognition, and deep information is integrated with shallow information to solve the multi-scale and occlusion problems that arise in the farming environment. At the same time, the present invention visualizes the number of each type of fish in the input video, which facilitates farmers to analyze the dynamic changes in fish behavior, providing a non-invasive, efficient and intelligent tool for real-time monitoring of fish.

[0101] In another aspect, the present invention provides an electronic device. Figure 6 As shown, the electronic device 600 includes a processor 601, a memory 602, a communication interface 603 and a communication bus 604;

[0102] The processor 601, memory 602, and communication interface 603 communicate with each other via the communication bus 304;

[0103] The processor 601 is used to call the computer program in the memory 602. When the processor 601 executes the computer program, the steps of the method for monitoring the stress state of factory-farmed fish provided by the embodiment of the present invention are implemented.

[0104] In addition, the computer program in the above-mentioned memory can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several computer programs to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0105] On the other hand, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for monitoring the stress status of fish in factory farming provided by the embodiment of the present invention as described above.

[0106] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for monitoring stress status of fish in factory farming, characterized in that: include: Obtain video data of factory-farmed underwater fish through a camera device; Detecting and identifying fish in the video data using a fully convolutional neural network algorithm; Classifying the fish species according to their relative pixel positions in the video data; as well as The water layer distribution of the fish is quantified, and a stress line is calculated based on the water layer distribution of the fish. The quantification of the water layer distribution of the fish and the calculation of the stress line according to the water layer distribution of the fish include: Obtain the number of each type of fish per unit time in the video data; The optimal ratio of bottom fish under stress and the optimal ratio of bottom fish under no stress are obtained by K-means clustering method; and The coercion line is calculated, The step of calculating the coercion line includes: The coercion line is obtained by the following formula: Wherein, x is the coercion line, x 1 is the optimal ratio of bottom fish in the no-stress state, x 2 is the optimal ratio of bottom fish under the stress state.

2. The method for monitoring stress status of fish in factory farming according to claim 1, characterized in that: The detecting and identifying fish in the video data using a fully convolutional neural network algorithm includes: Use the first 13 layers of VGG-16 as the backbone network to extract image features; and A detection module based on multi-scale convolution kernel is established for target positioning and recognition.

3. The method for monitoring stress status of fish in factory farming according to claim 2, characterized in that: The establishment of a detection module based on a multi-scale convolution kernel for target positioning and recognition includes: Extract feature maps from the downsampling points and concatenate and fuse them to obtain feature maps of three scales; and The feature maps of the three scales are input into the corresponding three columns of multi-scale convolution kernels for fusion, and finally the detection results of large, medium and small scale targets are obtained.

4. The method for monitoring stress status of fish used in factory farming according to claim 1, characterized in that: The classifying the fish according to relative pixel positions of the fish in the video data comprises: For detection results located at 20%-50% pixel positions of the video data, classifying the fish as pelagic fish; and For the detection results located at 80%-50% pixel positions of the video data, the fish are classified as bottom-layer fish.

5. The method for monitoring stress status of fish used in factory farming according to claim 1, characterized in that: The obtaining of the number of each type of fish per unit time in the video data comprises: Determine the total number of frames within the unit time; Summing up the number of each type of fish in each frame of image within the unit time; and Take the average of the summed results as the number of each type of fish in the current unit time.

6. A fish stress status monitoring system for factory farming, characterized in that: include: A video data acquisition module is used to acquire video data of factory-farmed underwater fish through a camera device; a detection and recognition module for detecting and identifying fish in the video data using a fully convolutional neural network algorithm; a classification module, configured to classify the fish species according to relative pixel positions of the fish species in the video data; as well as A stress line calculation module is used to quantify the water layer distribution of the fish and calculate the stress line based on the water layer distribution of the fish. The quantification of the water layer distribution of the fish and the calculation of the stress line according to the water layer distribution of the fish include: Obtain the number of each type of fish per unit time in the video data; The optimal ratio of bottom fish under stress and the optimal ratio of bottom fish under no stress are obtained by K-means clustering method; and The coercion line is calculated, The step of calculating the coercion line includes: The coercion line is obtained by the following formula: Wherein, x is the coercion line, x 1 is the optimal ratio of bottom fish in the no-stress state, x 2 is the optimal ratio of bottom fish under the stress state.

7. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for monitoring stress status of fish used in factory farming as claimed in any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for monitoring stress status of fish used in factory farming as claimed in any one of claims 1 to 5 are implemented.

Images