A method and terminal for detecting hypoxia in farmed fish based on machine vision technology

Abstract

The invention discloses a method and a terminal for detecting hypoxia in farmed fish based on machine vision technology. The method comprises: collecting underwater farmed fish hypoxia video data, and optimizing image data in the farmed fish hypoxia video data to obtain a farmed fish hypoxia data set; inputting the farmed fish hypoxia data set into a deep learning neural network model for training to obtain a fish recognition neural network model; loading the fish recognition neural network model into a fish hypoxia judgment model to obtain a fish hypoxia detection model; performing hypoxia detection on underwater farmed fish through the fish hypoxia detection model, outputting detection results, and issuing an early warning. The method for detecting hypoxia in farmed fish based on machine vision technology proposed by the invention solves the technical problem that the existing fish hypoxia recognition detection method still has a low fish hypoxia recognition detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of farmed fish hypoxia detection, and in particular to a farmed fish hypoxia detection method and terminal based on machine vision technology. Background Art

[0002] my country is a major fishing country, with aquaculture exceeding 1.3 trillion yuan in scale, and is an important part of my country's agricultural economy. However, in this field, the flooding of farmed fish caused by hypoxia is one of the most worrying issues for farmers. Every year, due to the fear of fish death due to hypoxia, dispatching personnel to patrol the huge fish ponds not only consumes a lot of manpower, but also has problems such as the inability to monitor 24 hours a day, the delay in finding hypoxia, or the inaccurate judgment of artificial hypoxia, which causes fish flooding, resulting in serious aquatic product losses.

[0003] The traditional method for detecting hypoxia in farmed fish is mainly based on the dissolved oxygen measurement method, but the sensor of the dissolved oxygen meter cannot be placed in the water for a long time, and each parameter must be zeroed and calibrated before each use. In addition, due to the complex underwater environment of the fish pond, there are large differences in the oxygen content of the water, which seriously affects the detection accuracy of the dissolved oxygen meter; in the aquaculture fish pond hypoxia early warning device with the publication number CN108575830A, the effective detection water area of ​​the dissolved oxygen sensor is very small, and the detection result cannot represent the oxygen content of all water areas, and the accuracy of fish hypoxia judgment is low; in similar technologies, the background modeling fish pond hypoxia detection method with the publication number CN107730495A adopts the method of shooting images from above the water surface. On the one hand, the water body of the fish farm is generally turbid. When the fish is slightly hypoxic, the fish will swim to the upper water area, that is, the dark floating head phenomenon will occur. The ordinary camera will be difficult to shoot the underwater fish through the turbid water surface due to the reduced transparency of the water body, which will result in poor shooting effect of the fish floating head phenomenon; when the fish is hypoxic and floating head, Only the fish mouth appears on the water surface, and the target to be detected is small, which makes identification difficult. On the other hand, environmental factors such as water surface reflection, insufficient light, gusts of wind causing waves on the water surface, etc. will not only affect the video shooting effect, but also cause misidentification of the model. At the same time, the background modeling technology used has the disadvantages of being easily disturbed by environmental and climatic factors and being unable to detect stationary targets. Therefore, the detection accuracy of this method is low. In the method for identifying fish hypoxia through video data with the publication number CN115802009A, there is also the disadvantage of shooting images above the water surface. Although water color image recognition technology is introduced in this method to make a secondary judgment on hypoxia, when water quality detection is performed by this technology, on the one hand, because the color histogram needs color quantization, the image feature dimension generated is high, which is not conducive to retrieval, and the retrieval efficiency of the color matrix is ​​relatively low, which is easy to cause misdetection under quantization processing. On the other hand, the construction of its data set relies heavily on expert judgment, and there are errors caused by subjective judgment, and the comparability and repeatability of water quality detection results are low. Therefore, this method cannot solve the problem of low detection accuracy of fish hypoxia identification method, and it also needs to use multiple models, which is complex and difficult to implement. In addition, the above two methods do not distinguish the degree of fish hypoxia, and the hypoxia detection effect is not perfect.

[0004] Therefore, the existing fish hypoxia identification and detection methods still have the technical problem of low accuracy in fish hypoxia identification and detection. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the defects of the prior art, the present invention provides a farmed fish hypoxia detection method and terminal based on machine vision technology to solve the technical problem that the existing fish hypoxia identification and detection methods still have low accuracy in fish hypoxia identification and detection.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] In a first aspect, the present invention provides a method for detecting hypoxia in farmed fish based on machine vision technology, comprising:

[0008] Collecting underwater farmed fish hypoxia video data, and optimizing the image data in the farmed fish hypoxia video data to obtain a farmed fish hypoxia data set;

[0009] Inputting the farmed fish hypoxia data set into a deep learning neural network model for training to obtain a fish recognition neural network model;

[0010] Loading the fish recognition neural network model into a fish hypoxia judgment model to obtain a fish hypoxia detection model;

[0011] The fish hypoxia detection model is used to detect hypoxia in underwater farmed fish, output the detection results, and issue an early warning.

[0012] In one implementation, the collecting of underwater farmed fish hypoxia video data and the optimization of image data in the farmed fish hypoxia video data to obtain a farmed fish hypoxia data set include:

[0013] An underwater camera is pre-installed under the water surface, and the shooting angle of the underwater camera is parallel to the water surface to collect video data of the oxygen-deficient floating heads of farmed fish;

[0014] Extract the video data into multiple pictures, and select the pictures of fish floating due to lack of oxygen;

[0015] Dehazing the image of the fish's hypoxic floating head and optimizing it through data enhancement;

[0016] The optimized fish hypoxia floating head picture is annotated with a rectangular frame to obtain the farmed fish hypoxia data set.

[0017] In one implementation, the inputting the farmed fish hypoxia dataset into a deep learning neural network model for training to obtain a fish recognition neural network model includes:

[0018] Inputting the farmed fish hypoxia data set into a deep learning neural network model for training to obtain a fish recognition neural network model;

[0019] Adjusting the hyperparameters of the fish identification neural network model, and comparing the detection accuracy of the fish identification network models with different hyperparameters;

[0020] According to the detection accuracy, the fish recognition neural network model with the detection accuracy meeting the requirements is selected.

[0021] In one implementation, the fish identification neural network model is loaded into a fish hypoxia judgment model to obtain a fish hypoxia detection model, including:

[0022] Establishing a fish hypoxia judgment model corresponding to the fish identification neural network model;

[0023] The fish identification neural network model is loaded into the fish hypoxia judgment model to obtain the fish hypoxia detection model.

[0024] In one implementation, the step of loading the fish identification neural network model into a fish hypoxia judgment model to obtain a fish hypoxia detection model further includes:

[0025] Adjusting the images in the farmed fish hypoxia dataset to a size that conforms to the fish hypoxia judgment model, and performing normalization processing on the images;

[0026] Extracting image features of the processed image and mapping the image features to anchor box coordinates and category probabilities;

[0027] The anchor frame and category of the target are predicted on the fish hypoxia judgment model, and the target detection is completed by retaining the anchor frame that meets the requirements through non-maximum suppression, so as to obtain the fish hypoxia detection model.

[0028] In one implementation, the step of establishing a fish hypoxia judgment model corresponding to the fish identification neural network model includes:

[0029] When the number of hypoxic fish is greater than the hypoxic fish number threshold, the duration of fish hypoxia begins to be recorded;

[0030] When the duration of fish hypoxia is greater than the fish hypoxia time threshold, the fish is judged to be hypoxic and an hypoxia alarm is issued;

[0031] When the number of the hypoxic fish becomes 0, the hypoxia release time starts to be recorded;

[0032] When the hypoxia release time is greater than the preset hypoxia release time threshold, it is determined that the fish is not hypoxic, and the hypoxia alarm is released;

[0033] Output fish hypoxia video with the number of hypoxic fish and hypoxia alarm.

[0034] In one implementation, the method of performing hypoxia detection on underwater farmed fish by using the fish hypoxia detection model, outputting the detection result, and issuing an early warning includes:

[0035] Determining the degree of hypoxia of the anoxic fish according to the position of the underwater camera that detects the anoxic fish;

[0036] The detection result is output according to the degree of hypoxia, and an early warning is issued.

[0037] In one implementation, judging the degree of hypoxia of the anoxic fish according to the position of the underwater camera that detects the anoxic fish includes:

[0038] When the hypoxia alarm pops up when fish hypoxia is detected only in the center of the pond, it is judged as mild hypoxia;

[0039] When the hypoxia alarm pops up when fish hypoxia is detected only near the fish pond, it is judged as moderate hypoxia;

[0040] When hypoxia is detected in the center and at the edge of the fish pond at the same time and the hypoxia alarm pops up, it is judged as severe hypoxia.

[0041] In a second aspect, the present invention further provides a terminal, comprising: a processor and a memory, wherein the memory stores a farmed fish hypoxia detection program based on machine vision technology, and when the farmed fish hypoxia detection program based on machine vision technology is executed by the processor, it is used to implement the operation of the farmed fish hypoxia detection method based on machine vision technology as described in the first aspect.

[0042] In a third aspect, the present invention further provides a computer-readable storage medium, which stores a farmed fish hypoxia detection program based on machine vision technology, and when the farmed fish hypoxia detection program based on machine vision technology is executed by a processor, it is used to implement the operation of the farmed fish hypoxia detection method based on machine vision technology as described in the first aspect.

[0043] The present invention adopts the above technical solution to achieve the following effects:

[0044] The present invention sets an underwater camera underwater in advance to more directly collect underwater farmed fish hypoxia data, thereby solving the problem that the dark floating head phenomenon is difficult to capture and the target is too small to detect when the fish floats; and by optimizing the collected farmed fish hypoxia data, the interference of environmental factors such as water quality is reduced, and the quality of fish hypoxia data is improved; the deep learning neural network model is trained by high-quality fish hypoxia data, and the trained deep learning neural network model is used to perform hypoxia recognition and detection on fish, which greatly improves the accuracy of fish hypoxia detection and recognition; at the same time, the present invention judges the fish hypoxia situation by the fish floating head behavior and the fish floating head position, thereby avoiding the problem of large difference in water oxygen content in the water oxygen detection method, making the hypoxia judgment more accurate, and further improving the hypoxia detection effect, so that farmers can make different decisions according to the degree of fish hypoxia; the farmed fish hypoxia detection method based on machine vision technology proposed by the present invention solves the technical problem that the existing fish hypoxia recognition and detection method still has a low accuracy rate of fish hypoxia recognition and detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0046] Figure 1 It is a flow chart of a method for detecting hypoxia in farmed fish based on machine vision technology in one implementation of the present invention.

[0047] Figure 2 It is a schematic diagram of the model quantization process of the fish hypoxia detection method in one implementation of the present invention.

[0048] Figure 3 It is a schematic diagram of a deep learning neural network training process in one implementation of the present invention.

[0049] Figure 4 It is a schematic diagram of a fish hypoxia judgment mechanism in one implementation of the present invention.

[0050] Figure 5 It is a schematic diagram of a technical layout flow chart in one implementation of the present invention.

[0051] Figure 6 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0052] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] The present invention provides a method and terminal for detecting hypoxia in farmed fish based on machine vision technology. In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0054] Exemplary Methods

[0055] In the field of aquaculture, the flooding of farmed fish caused by hypoxia is one of the most worrying issues for fish farmers. Every year, fish die from hypoxia. Dispatching personnel to patrol huge fish ponds not only consumes a lot of manpower, but also causes problems such as the inability to monitor 24 hours a day, delayed detection of hypoxia, or inaccurate judgment of hypoxia, which in turn causes fish flooding, resulting in serious aquatic losses. Existing fish hypoxia identification and detection technology also has technical problems such as being easily affected by environmental factors and having low detection accuracy.

[0056] In view of the above technical problems, the embodiment of the present invention provides a method for detecting hypoxia in farmed fish based on machine vision technology. By pre-setting an underwater camera underwater, underwater farmed fish hypoxia data is collected more directly, solving the problem that the dark floating head phenomenon is difficult to capture and the target is too small to detect when the fish floats; and by optimizing the collected farmed fish hypoxia data, the interference of environmental factors such as water quality is reduced, and the quality of fish hypoxia data is improved; the deep learning neural network model is trained by high-quality fish hypoxia data, and the trained deep learning neural network model is used to perform hypoxia recognition and detection on fish, which greatly improves the accuracy of fish hypoxia detection and recognition; at the same time, the present invention judges the fish hypoxia situation by the fish floating head behavior and the fish floating head position, avoids the problem of large difference in water oxygen content in the water oxygen detection method, makes the hypoxia judgment more accurate, and further improves the hypoxia detection effect, so that farmers can make different decisions according to the degree of fish hypoxia; the method for detecting hypoxia in farmed fish based on machine vision technology proposed by the present invention solves the technical problem that the existing fish hypoxia recognition and detection method still has a low accuracy rate of fish hypoxia recognition and detection.

[0057] like Figure 1 As shown, the embodiment of the present invention provides a method for detecting hypoxia in farmed fish based on machine vision technology, comprising the following steps:

[0058] Step S100, collecting underwater farmed fish hypoxia video data, and optimizing the image data in the farmed fish hypoxia video data to obtain a farmed fish hypoxia data set.

[0059] In this embodiment, the farmed fish hypoxia detection method based on machine vision technology is applied to a terminal, and the terminal includes but is not limited to: computers and mobile terminals and other devices; the terminal is provided with a training platform for the farmed fish hypoxia detection method based on machine vision technology.

[0060] In this embodiment, when fish feel that they are short of oxygen, they usually swim in groups in the upper layer of the water, which is the "dark floating head" phenomenon. As the oxygen content further decreases, the fish will swim in groups to the surface of the water to breathe for a long time, which is the "floating head" phenomenon. They may even float on the water surface with their bodies tilted. This embodiment uses machine vision technology to monitor such floating head phenomena of fish hypoxia to achieve an all-weather, real-time, and high-precision fish hypoxia detection method, which will play an important role in reducing fish hypoxia deaths, improving the quality of aquatic products, and promoting aquaculture.

[0061] In this embodiment, if Figure 2As shown, by setting up an underwater camera under the water surface in advance, video data containing hypoxia behavior of underwater farmed fish can be collected more directly, and the video data can be processed to reduce the interference of environmental factors and improve the quality of fish hypoxia data. The processed data is input into a deep learning neural network model for training to obtain a fish recognition network model; the fish recognition network model is loaded into a fish hypoxia judgment model to obtain a fish hypoxia detection model, and hypoxia detection is performed on farmed fish through the fish hypoxia detection model, and the detection results are output.

[0062] Specifically, in an implementation of the embodiment of the present invention, step S100 includes the following steps:

[0063] Step S101, pre-setting an underwater camera under the water surface, wherein the shooting angle of the underwater camera is parallel to the water surface, and collecting video data of the oxygen-deficient floating heads of farmed fish;

[0064] Step S102, extracting the video data into multiple pictures, and selecting pictures of fish floating due to lack of oxygen;

[0065] Step S103, defogging the image of the fish's hypoxic floating head and optimizing it through data enhancement;

[0066] Step S104, annotating the optimized fish hypoxia floating head picture with a rectangular frame to obtain the farmed fish hypoxia data set.

[0067] In this embodiment, an underwater camera is pre-set to film the fish hypoxia behavior and generate fish hypoxia video data, and the specifications of the underwater camera equipment used for data collection are not restricted; because fish will gather in groups near the water surface when they are hypoxic, the underwater camera is arranged about 2 cm below the water surface, and the shooting angle is parallel to the water surface to ensure that when the fish has dark floating head or floating head phenomenon, large targets such as the fish body can be directly photographed instead of small targets such as the fish mouth.

[0068] In this embodiment, after collecting fish hypoxia video data taken by an underwater camera, the fish hypoxia video data is extracted into multiple pictures, and pictures containing fish hypoxia floating heads are screened out; the pictures of fish hypoxia floating heads are defogged, and the pictures of fish hypoxia floating heads are optimized by contrast adjustment, flipping, translation and other data enhancement methods to reduce the interference of environmental factors such as water quality and improve the quality of fish hypoxia data; and the hypoxic fish in the picture are marked with a rectangular annotation frame, and the corresponding txt label file is output, which contains the annotation frame x, y coordinates, annotation frame length and width, and category "fish"; batch operations are performed on the label files and pictures in the data set, and the above data set is divided into a training set, a validation set, and a test set at a ratio of 8:1:1 to establish a farmed fish hypoxia data set.

[0069] like Figure 1 As shown, the embodiment of the present invention provides a method for detecting hypoxia in farmed fish based on machine vision technology, which also includes the following steps:

[0070] Step S200, inputting the farmed fish hypoxia data set into a deep learning neural network model for training to obtain a fish recognition neural network model.

[0071] In this embodiment, after preprocessing the collected data, a deep learning neural network model is used to identify fish, and the detection accuracy is improved to more than 96%, which effectively solves the problem of low detection accuracy due to susceptibility to environmental influences and inability to detect stationary targets; the goal of deep learning is to predict unknown variables based on certain given information. The variable to be predicted is generally a single variable. If multiple variables need to be predicted, they are also split into multiple deep learning network models for processing, so each deep learning network model still outputs a single variable; the single variable output by the deep learning network model is called a scalar, and the output of the deep learning network model generally does not involve high-dimensional problems, so compared with the prior art, the deep learning network model is less difficult to implement, and this embodiment loads the fish identification neural network model into the fish hypoxia judgment model to obtain a fish hypoxia detection model, which can more accurately judge the fish hypoxia situation.

[0072] Specifically, in an implementation of the embodiment of the present invention, step S200 includes the following steps:

[0073] Step S201, inputting the farmed fish hypoxia data set into a deep learning neural network model for training to obtain a fish recognition neural network model;

[0074] Step S202, adjusting the hyperparameters of the fish recognition neural network model, and comparing the detection accuracy of the fish recognition network models with different hyperparameters;

[0075] Step S203, selecting a fish recognition neural network model whose detection accuracy meets the requirements according to the detection accuracy.

[0076] In this embodiment, if Figure 3As shown in the figure, CSPDarknet53 is used as the feature extraction network to extract a series of feature maps of different scales of the image; the feature extraction network structure consists of the input end, the Backbone network, the Neck layer, and the Prediction layer. The Backbone network is mainly composed of the Focus structure and three CSP structures (cross-stage local network modules). The Neck layer is composed of the SPP (Spatial Pyramid Pooling) module and the FPN (Feature Pyramid Network) + PAN (Path Aggregation Network instance segmentation neural network) structure; the Backbone network and the Neck are connected in series to form the core part of the feature extraction in the network structure; at the same time, the farmed fish hypoxia dataset is divided into two branches through CSP, and convolution operations are performed separately to halve the number of channels. One branch does not perform any processing, and the other branch performs multiple residual structures, and finally merges the channels of the two branches.

[0077] In this embodiment, in the Backbone network, the Focus structure slices the input 608*608*3 original image, first converting it into a 304*304*12 feature map, and then undergoing a convolution operation with 32 convolution kernels to obtain feature maps of different sizes, and a convolution kernel (CBL) with a size of 3*3 and stride=2 is placed in front of the CSP structure to play a downsampling role; the CSP module first divides the feature map of the base layer into two parts, and then merges them through a cross-stage hierarchy to reduce the amount of calculation while ensuring the accuracy; an SPP module is added after the fourth CBL, and the SPP module converts feature maps of any size into feature vectors of fixed size, and then performs three maximum pooling operations on the feature map of size 19*19, and then splices it with the feature map of size 38*38 to perform multi-scale feature fusion.

[0078] In this embodiment, after the multi-scale features of the image are fused, the FPN+PAN structure will propagate the features from top to bottom starting from the high-resolution feature map, and gradually propagate the features to a lower scale using an upsampling operation; then, through bottom-up feature fusion, the high-resolution features are fused with the low-resolution features to obtain a new feature pyramid; downsampling operation is started at the bottom layer of the new feature pyramid, and a 3*3 convolution is performed on the second-to-last layer of the original feature pyramid with a stride of 2; then a horizontal connection is made with the bottom layer after downsampling, the two are added, and finally, the features are fused through CONV convolution, so that the image has rich graphic features and semantic features.

[0079] In this embodiment, the feature extraction network finally outputs three feature layers, which are called effective feature layers. The three feature layers are located at three different positions of the feature extraction network, namely the middle layer, the lower middle layer, and the bottom layer. The feature pyramid network layer is constructed by extracting the three feature layers to obtain three enhanced feature layers, which are 76*76*255, 38*38*255, and 19*19*255 respectively.

[0080] In this embodiment, the three enhanced feature layers obtained are passed to the Prediction layer. During training, the Prediction layer uses CIOU_Loss as the loss function of the Bounding box, generates three prediction boxes for each feature map pixel of the three enhanced feature layers, and associates them with the feature layer data through the anchor box mechanism to generate an output matrix with target category, category probability and prediction box position; when the detection layer detects a fish target, a prediction box is generated around the fish target, and an nms (non-maximum suppression) operation is performed for the screening of many target boxes, and only one prediction box with the best effect is retained. Correspondingly, a prediction box is drawn for each fish on the original image, thereby completing the training of the deep learning neural network model and obtaining a fish recognition neural network model.

[0081] In this embodiment, the hyperparameters of the fish identification neural network model are adjusted, the fish identification neural network models under different hyperparameters are trained, and the detection accuracy of each model is compared to select the fish identification neural network model with the best detection accuracy, and a target counting and fish hypoxia judgment model matching the fish identification neural network model is constructed based on the fish identification neural network model to build a fish hypoxia detection model.

[0082] like Figure 1 As shown, the embodiment of the present invention provides a method for detecting hypoxia in farmed fish based on machine vision technology, which also includes the following steps:

[0083] Step S300, loading the fish identification neural network model into the fish hypoxia judgment model to obtain a fish hypoxia detection model.

[0084] In this embodiment, the fish recognition network model with the best detection accuracy is first loaded into the fish hypoxia judgment model, and the input image is preprocessed to adapt it to the input requirements of the fish hypoxia judgment model. The fish hypoxia recognition detection model is obtained by loading the fish recognition network model into the fish hypoxia judgment model.

[0085] Specifically, in an implementation of the embodiment of the present invention, step S300 includes the following steps:

[0086] Step S301, establishing a fish hypoxia judgment model corresponding to the fish recognition neural network model;

[0087] Step S302, loading the fish recognition neural network model into the fish hypoxia judgment model to obtain the fish hypoxia detection model;

[0088] Step S303, adjusting the image in the farmed fish hypoxia data set to a size that conforms to the fish hypoxia judgment model, and performing normalization processing on the image;

[0089] Step S304, extracting image features of the processed image, and mapping the image features to anchor box coordinates and category probabilities;

[0090] Step S305, predicting the anchor frame and category of the target on the fish hypoxia judgment model, and retaining the anchor frame that meets the requirements to complete the target detection through non-maximum suppression, so as to obtain the fish hypoxia detection model;

[0091] Step S306, when the number of fish suffering from hypoxia is greater than the threshold value of the number of fish suffering from hypoxia, start recording the duration of fish hypoxia;

[0092] Step S307, when the duration of fish hypoxia is greater than the fish hypoxia time threshold, it is determined that the fish is hypoxic, and a hypoxia alarm is issued;

[0093] Step S308, when the number of the hypoxic fish becomes 0, start recording the hypoxia release time;

[0094] Step S309, when the hypoxia release time is greater than the preset hypoxia release time threshold, it is determined that the fish is not hypoxic, and the hypoxia alarm is released;

[0095] Step S310, outputting a fish hypoxia video with the number of hypoxic fish and a hypoxia alarm.

[0096] In this embodiment, the preprocessing includes adjusting the image to the size required by the fish hypoxia judgment model and normalizing it to scale the pixel values ​​to a fixed range; passing the preprocessed image to the fish hypoxia judgment model; extracting the input image features within the model; mapping the convolution features to the anchor box coordinates and category probabilities, that is, dividing the image into grids, and then predicting the anchor box and category of the target on each grid, and then removing redundant detections through the NMS operation, retaining the anchor box with the highest confidence, and completing the target detection.

[0097] In this embodiment, if Figure 4As shown, draw a frame on the original image and cut out the predicted target, which is saved in the form of a picture under sace_dir / crops; in the counting method module, the number of anchor boxes in each frame is counted, and the total number of anchor boxes with the category "fish" on the anchor boxes in each frame is obtained. Among them, the iteratively detected anchor boxes are involved, and the total number of anchor boxes with the "fish" category represents the number of hypoxic fish n recognized. Write the number of hypoxic fish n to the upper right of the picture before saving the picture; since the phenomenon of fish hypoxia probe is group-based and lasts for a long time, in this embodiment, the degree of fish hypoxia is measured by the number of fish floating on the surface and the floating time.

[0098] In this embodiment, n is the number of hypoxic fish, N is the threshold of the number of hypoxic fish, t is the fish hypoxia time, T is the fish hypoxia time threshold, t0 is the hypoxia relief time, and T0 is the hypoxia relief time threshold, where N, T, and T0 are set according to the actual situation. If there are more than or equal to N fish showing dark floating or floating behavior continuously within a certain time t, it is determined as hypoxia; if there are no more fish showing floating behavior within a certain time t0 after the hypoxia alarm is issued, it is determined that the fish are not hypoxic.

[0099] Specifically, in the fish hypoxia judgment module:

[0100] When n≥N, start recording t;

[0101] When 0 < n < N, stop recording t;

[0102] When n = 0, start recording t0;

[0103] When t is greater than or equal to T, it is determined that the fish are hypoxic, and a hypoxia alarm pops up on the original image; when it is detected that the number of fish n becomes 0 and the duration t0 is greater than or equal to T0, it is determined that the fish are no longer hypoxic, and the hypoxia alarm is解除; finally, a video with anchor boxes, the number of hypoxic fish, and the hypoxia alarm is output to achieve visualization.

[0104] As Figure 1 shown, the embodiment of the present invention provides a method for detecting hypoxia of cultured fish based on machine vision technology, which further includes the following steps:

[0105] Step S400, perform hypoxia detection on the underwater cultured fish through the fish hypoxia detection model, output the detection result, and give an early warning.

[0106] In this embodiment, through the fish hypoxia detection model, the video of the underwater camera is obtained in real time. The fish hypoxia detection model converts the video stream into image frames, then recognizes the number of hypoxic fish in each frame of the image, displays the number of hypoxic fish in the current image, outputs the detection video and the detection result, and gives an early warning to the farmer according to the degree of fish hypoxia.

[0107] Specifically, in an implementation of the embodiment of the present invention, step S400 includes the following steps:

[0108] Step S401, judging the degree of hypoxia of the hypoxic fish according to the position of the underwater camera that detects the hypoxic fish;

[0109] Step S402, outputting the detection result according to the hypoxia degree and issuing an early warning;

[0110] Step S403, when the hypoxia alarm pops up when the fish hypoxia is detected only in the center of the fish pond, it is judged as mild hypoxia;

[0111] Step S404, when the hypoxia alarm pops up only when the fish hypoxia is detected near the fish pond, it is judged as moderate hypoxia;

[0112] Step S405, when hypoxia is detected in the center and the edge of the fish pond at the same time and a hypoxia alarm pops up, it is determined to be severe hypoxia.

[0113] In this embodiment, if Figure 5 As shown, create a cloud server instance, configure the operating system on it, and this embodiment uses Python 3 of the Ubuntu system; connect the underwater camera to the Raspberry Pi, and use socket communication to access the cloud server. Through the public network IP of the cloud server, the purpose of wireless remote transmission of video data can be achieved, and new data sets can be collected in the later stage for model training optimization; in response to different receiving terminals, a web page presenting the hypoxia detection results can be designed, so that different receiving terminals (computers / mobile phones) can be accessed through the public network IP; use third-party software to transfer files with the cloud server, and the present invention uses termius, upload the fish hypoxia detection model, and configure the operating environment for test optimization. Install opencv and ffmpeg appropriate video stream processing libraries on the server. In the fish hypoxia detection model, the underwater camera's shooting video is obtained in real time through video stream communication protocols such as RTSP. The fish hypoxia detection model converts the video stream into image frames, and then identifies the number of hypoxic fish in each frame, and displays the number of hypoxic fish in the current image, and outputs the detection video.

[0114] In this embodiment, the number of hypoxic fish n in each frame is detected by a fish hypoxia detection model. When n is greater than or equal to a threshold value N for the number of hypoxic fish, the duration t of fish hypoxia is recorded. When the fish hypoxia time t is greater than or equal to a preset fish hypoxia time threshold T, the fish is judged to be hypoxic. When n=0, the hypoxia release time t0 is recorded. When t0 is greater than or equal to the hypoxia release time threshold T0, the hypoxia alarm is released. Since the degree of fish hypoxia is related to the position of its floating head, when the fish only floats in the center of the fish pond, it indicates mild hypoxia. When the fish only floats at the edge of the fish pond, it indicates moderate hypoxia and an aerator needs to be turned on for oxygenation. When there are fish floating in the center of the fish pond, the shore and other places, it indicates that there is severe hypoxia and effective measures must be taken to increase oxygen, otherwise the consequences may be serious. Therefore, the degree of fish hypoxia can be judged by underwater cameras in the center and at the edge of the fish pond. When fish hypoxia is detected only in the center of the fish pond and the hypoxia alarm pops up, the fish hypoxia detection model outputs mild hypoxia. When fish hypoxia is detected only at the edge of the fish pond and the hypoxia alarm pops up, the fish hypoxia detection model outputs moderate hypoxia. When hypoxia is detected both in the center and at the edge of the fish pond and the hypoxia alarm pops up, the fish hypoxia detection model outputs severe hypoxia. And hypoxia alarms of corresponding degrees are issued to farmers through multiple ports.

[0115] This embodiment achieves the following technical effects through the above technical solution:

[0116] The present invention sets an underwater camera underwater in advance to more directly collect underwater farmed fish hypoxia data, thereby solving the problem that the dark floating head phenomenon is difficult to capture and the target is too small to detect when the fish floats; and by optimizing the collected farmed fish hypoxia data, the interference of environmental factors such as water quality is reduced, and the quality of fish hypoxia data is improved; the deep learning neural network model is trained by high-quality fish hypoxia data, and the trained deep learning neural network model is used to perform hypoxia recognition and detection on fish, which greatly improves the accuracy of fish hypoxia detection and recognition; at the same time, the present invention judges the fish hypoxia situation by the fish floating head behavior and the fish floating head position, thereby avoiding the problem of large difference in water oxygen content in the water oxygen detection method, making the hypoxia judgment more accurate, and further improving the hypoxia detection effect, so that farmers can make different decisions according to the degree of fish hypoxia; the farmed fish hypoxia detection method based on machine vision technology proposed by the present invention solves the technical problem that the existing fish hypoxia recognition and detection method still has a low accuracy rate of fish hypoxia recognition and detection.

[0117] Exemplary Devices

[0118] Based on the above embodiments, the present invention also provides a terminal, comprising: a processor, a memory, an interface, a display screen and a communication module connected via a system bus; wherein the processor is used to provide computing and control capabilities; the memory includes a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of the operating system and the computer program in the storage medium; the interface is used to connect external devices, such as mobile terminals and computers; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or a mobile terminal.

[0119] When the computer program is executed by the processor, it is used to implement the operation of a method for detecting hypoxia in farmed fish based on machine vision technology.

[0120] It can be understood by those skilled in the art that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] In one embodiment, a terminal is provided, which includes: a processor and a memory, wherein the memory stores a farmed fish hypoxia detection program based on machine vision technology, and the farmed fish hypoxia detection program based on machine vision technology is used to implement the operation of the farmed fish hypoxia detection method based on machine vision technology as described above when executed by the processor.

[0122] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a farmed fish hypoxia detection program based on machine vision technology, and when the farmed fish hypoxia detection program based on machine vision technology is executed by the processor, it is used to implement the operation of the above-mentioned farmed fish hypoxia detection method based on machine vision technology.

[0123] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory.

[0124] In summary, the present invention provides a method and terminal for detecting hypoxia in farmed fish based on machine vision technology, the method comprising: collecting underwater farmed fish hypoxia video data, and optimizing image data in the farmed fish hypoxia video data to obtain a farmed fish hypoxia data set; inputting the farmed fish hypoxia data set into a deep learning neural network model for training to obtain a fish recognition neural network model; loading the fish recognition neural network model into a fish hypoxia judgment model to obtain a fish hypoxia detection model; performing hypoxia detection on underwater farmed fish through the fish hypoxia detection model, outputting detection results, and issuing an early warning; the method for detecting hypoxia in farmed fish based on machine vision technology proposed by the present invention solves the technical problem that the existing fish hypoxia recognition detection method still has a low accuracy rate in fish hypoxia recognition detection.

[0125] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for detecting hypoxia in farmed fish based on machine vision technology, characterized in that: The method for detecting hypoxia in farmed fish based on machine vision technology comprises the following steps: Collecting underwater farmed fish hypoxia video data, and optimizing the image data in the farmed fish hypoxia video data to obtain a farmed fish hypoxia data set; Inputting the farmed fish hypoxia data set into a deep learning neural network model for training to obtain a fish recognition neural network model; Loading the fish recognition neural network model into a fish hypoxia judgment model to obtain a fish hypoxia detection model; Perform hypoxia detection on underwater farmed fish through the fish hypoxia detection model, output the detection results, and issue an early warning; The method of collecting underwater farmed fish hypoxia video data and optimizing the image data in the farmed fish hypoxia video data to obtain the farmed fish hypoxia data set includes: An underwater camera is pre-installed under the water surface, and the shooting angle of the underwater camera is parallel to the water surface to collect video data of the oxygen-deficient floating heads of farmed fish; Extract the video data into multiple pictures, and select the pictures of fish floating due to lack of oxygen; Dehazing the image of the fish's hypoxic floating head and optimizing it through data enhancement; The optimized fish hypoxia floating head picture is marked with a rectangular frame to obtain the farmed fish hypoxia data set; The underwater camera is arranged 2 cm below the water surface; The method of loading the fish identification neural network model into the fish hypoxia judgment model to obtain the fish hypoxia detection model comprises: Establishing a fish hypoxia judgment model corresponding to the fish identification neural network model; Loading the fish recognition neural network model into the fish hypoxia judgment model to obtain the fish hypoxia detection model; The method of establishing a fish hypoxia judgment model corresponding to the fish identification neural network model comprises: When the number of hypoxic fish is greater than the hypoxic fish number threshold, the duration of fish hypoxia begins to be recorded; When the duration of fish hypoxia is greater than the fish hypoxia time threshold, the fish is judged to be hypoxic, and a hypoxia alarm is issued, and the hypoxia relief time is recorded; When the number of the hypoxic fish becomes 0, the hypoxia release time starts to be recorded; When the hypoxia release time is greater than the preset hypoxia release time threshold, it is determined that the fish is not hypoxic, and the hypoxia alarm is released; Output fish hypoxia video with the number of hypoxic fish and hypoxia alarm.

2. The method for detecting hypoxia in farmed fish based on machine vision technology according to claim 1, characterized in that: The method of inputting the farmed fish hypoxia data set into a deep learning neural network model for training to obtain a fish recognition neural network model comprises: Inputting the farmed fish hypoxia data set into a deep learning neural network model for training to obtain a fish recognition neural network model; Adjusting the hyperparameters of the fish identification neural network model, and comparing the detection accuracy of the fish identification network models with different hyperparameters; According to the detection accuracy, the fish recognition neural network model with the detection accuracy meeting the requirements is selected.

3. The method for detecting hypoxia in farmed fish based on machine vision technology according to claim 1, characterized in that: The step of loading the fish identification neural network model into the fish hypoxia judgment model to obtain the fish hypoxia detection model also includes: Adjusting the images in the farmed fish hypoxia dataset to a size that conforms to the fish hypoxia judgment model, and performing normalization processing on the images; Extracting image features of the processed image and mapping the image features to anchor box coordinates and category probabilities; The anchor frame and category of the target are predicted on the fish hypoxia judgment model, and the target detection is completed by retaining the anchor frame that meets the requirements through non-maximum suppression, so as to obtain the fish hypoxia detection model.

4. The method for detecting hypoxia in farmed fish based on machine vision technology according to claim 1, characterized in that: The method of performing hypoxia detection on underwater cultured fish by using the fish hypoxia detection model, outputting the detection result, and issuing an early warning includes: Determining the degree of hypoxia of the anoxic fish according to the position of the underwater camera that detects the anoxic fish; The detection result is output according to the degree of hypoxia, and an early warning is issued.

5. The method for detecting hypoxia in farmed fish based on machine vision technology according to claim 4, characterized in that: The step of determining the degree of hypoxia of the anoxic fish according to the position of the underwater camera detecting the anoxic fish comprises: When the hypoxia alarm pops up when fish hypoxia is detected only in the center of the pond, it is judged as mild hypoxia; When the hypoxia alarm pops up when fish hypoxia is detected only near the fish pond, it is judged as moderate hypoxia; When hypoxia is detected in the center and at the edge of the fish pond at the same time and the hypoxia alarm pops up, it is judged as severe hypoxia.

6. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores a farmed fish hypoxia detection program based on machine vision technology, and when the farmed fish hypoxia detection program based on machine vision technology is executed by the processor, it is used to implement the farmed fish hypoxia detection method based on machine vision technology as described in any one of claims 1-5.

7. A storage medium, characterized in that: The storage medium stores a farmed fish hypoxia detection program based on machine vision technology, and when the farmed fish hypoxia detection program based on machine vision technology is executed by the processor, it is used to implement the farmed fish hypoxia detection method based on machine vision technology as described in any one of claims 1-5.

Images