A fishery law violation fishing behavior discrimination system, method, device and medium based on an improved YOLOv10 algorithm

Abstract

The application discloses a fishery illegal fishing behavior discrimination system, method, equipment and medium based on a YOLOv10 algorithm improvement; the system comprises a water area real-time image acquisition module, an image detection model based on the YOLOv10 algorithm improvement and a detection discrimination module; the Backbone network module of the image detection model based on the YOLOv10 algorithm improvement contains an HWD-ADown module capable of extracting more discriminative features and a C2f module introducing an AM attention module, so that the network can more accurately capture the position of each object. The image detection model based on the YOLOv10 algorithm improvement can identify different scales of ships and personnel, improve the original algorithm calculation redundancy and small target missing detection problem, and improve the detection accuracy of illegal fishing behavior in key water areas.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a fishery illegal fishing behavior discrimination system, method, device and medium based on YOLOv10 algorithm improvement. BACKGROUND

[0002] In recent years, the fishery resources in the Yangtze River mainstream, Poyang Lake, Dongting Lake and other important waters in the middle and lower reaches of the Yangtze River have decreased significantly. After several generations of propagation, artificially bred fish will inevitably experience genetic diversity degradation, so protecting wild fish resources is related to the future of our aquaculture industry. At present, there are illegal fishing behaviors in the Yangtze River basin, and cracking down on illegal fishing is the core of fish resource protection. At present, the patrol work in the Yangtze River basin mainly relies on manual patrol by law enforcement personnel, and law enforcement personnel are arranged to patrol important waters at regular intervals, and the inspection results are recorded in books. From the aspects of manpower and material resources, this method has many disadvantages: first, manual patrol cannot be covered for 24 hours; second, illegal personnel will have the mentality of taking chances and continue to commit crimes. Therefore, by installing Internet of Things devices such as cameras and speakers on the shore of important waters, the target and behavior in the 24-hour monitoring video are analyzed to determine whether it is a fishing behavior, which is a promising fishery law enforcement method.

[0003] With the development of image processing technology, deep learning has made good progress in target detection. At present, the classic target detection algorithm mainly includes single-stage and double-stage, single-stage including YOLO, SDD, Retina-Net, double-stage including R-CNN, Fast R-CNN, Mask R-CNN; among them, YOLO algorithm runs the fastest, and the accuracy is also high, especially the latest version of YOLOv10 model, which has made a major breakthrough in speed and accuracy, but YOLOv10 is prone to miss detection of small targets when applied in water area background, and the recognition effect is not good, which still needs further research. SUMMARY

[0004] The purpose of the present application is to provide a fishery illegal fishing behavior discrimination system, method, device and medium based on YOLOv10 algorithm improvement.

[0005] To achieve the above purpose, the technical scheme provided by the present application is:

[0006] The first aspect of the present application provides a fishery illegal fishing behavior discrimination system based on YOLOv10 algorithm improvement, comprising:

[0007] A water area real-time image acquisition module is configured to acquire water area real-time images through a pan-tilt monitoring camera.

[0008] The image detection model improved based on the YOLOv10 algorithm comprises a Backbone network module for encoding an input image in a feature form, a Neck network module for processing the feature form encoding, and a Head network module for generating a model prediction output;

[0009] The Backbone network module comprises an HWD-ADown module for decomposing a feature map and a C2f module for feature extraction and feature fusion, and the output end of the HWD-ADown module is connected to the C2f module.

[0010] The HWD-ADown module comprises an HWD module and an ADown module; the HWD module is used for obtaining a feature map decomposition processing result through Haar wavelet transformation of an input image; and the ADown module is used for performing down-sampling operation on the feature map decomposition processing result of the HWD module.

[0011] An AM attention module is introduced based on the C2f module connected to the output end of the HWD-ADown module, the AM attention module comprises a context attention module CxAM and a content attention module CnAM, the context attention module CxAM is used for paying attention to a specified local area or block on a feature map, and the content attention module CnAM is used for paying attention to relatively shallow level features on the feature map; and the output features of the context attention module CxAM and the content attention module CnAM are fused to obtain final feature expression of the feature map.

[0012] A detection and discrimination module is used for inputting a water area real-time image into the image detection model improved based on the YOLOv10 algorithm for identification to obtain a detection result, and if there is illegal fishing behavior in the detection result discrimination information of the discrimination model, an alarm function is triggered.

[0013] To optimize the above technical solution, the following specific measures are taken:

[0014] The HWD module comprises a lossless feature encoding block and a feature representation learning block, the lossless feature encoding block comprises a Haar wavelet transformation layer, the Haar wavelet transformation layer is used for performing Haar wavelet transformation on an input image to obtain image decomposition information; and the feature representation learning block is used for extracting discriminative features from the image decomposition information obtained by the Haar wavelet transformation layer to output a feature map decomposition processing result.

[0015] In the ADown module, the input feature map is divided into two branches after an average pooling layer, one branch only passes through a CBS module, and the other branch passes through a maximum pooling layer and a CBS module in sequence, and the results obtained by the two branches are output after feature fusion.

[0016] The process that the Haar wavelet transform layer performs Haar wavelet transform on the input image to obtain image decomposition information is specifically as follows:

[0017] After the image with a resolution of h*w is decomposed by Haar wavelet transform, h represents the image height, w represents the image width, four components are generated, including a low-frequency component A, a high-frequency component H in the horizontal direction, a high-frequency component V in the vertical direction and a high-frequency component D in the diagonal direction, and the size of each component is h / 2 x w / 2, the resolution is reduced to 1 / 4 of the original image resolution, and the number of channels of the feature map is increased by four times;

[0018] The feature representation learning block includes a 1x1 convolution layer, a normalization layer and a ReLU activation function; the process that the feature representation learning block extracts discriminative features from the image decomposition information obtained by the Haar wavelet transform layer and outputs a feature map decomposition processing result is specifically as follows:

[0019] The feature representation learning block uses the 1x1 convolution layer to adjust the number of channels of the feature map to adapt to the subsequent layer and filter redundant information, and then combines the normalization layer and the ReLU activation function to extract discriminative features and obtain a feature map decomposition processing result.

[0020] The process that the context attention module CxAM focuses on a specified local region or block on the feature map is specifically as follows:

[0021] The context attention module CxAM converts the feature map F of the feature map F into a potential space, C is the number of channels, h is the image height, and w is the image width; the convolution layer W C×h×w is used for converting the input feature map into a potential space, and the convolution layer W q is used for converting the input feature map into a potential space; the convolution layer W k is used for converting the input feature map into a potential space, C is the number of channels, h is the image height, and w is the image width; the convolution layer W q is used for converting the input feature map into a potential space, and the convolution layer W k is used for converting the input feature map into a potential space; the conversion formula is as follows:

[0022]

[0023] Wherein, {Q, K} ∈ R C×h×w , the first query space Q and the first key space K are converted into vectors, in order to obtain the relationship between the local regions or blocks, the following relationship matrix is obtained:

[0024] R = Q T K

[0025] Wherein, R ∈ R N×N, N = h x w, and then normalized by a sigmoid activation function and average pooling to obtain an attention matrix R1 e R C×h×w ; then the feature map F is converted into a feature V by using a convolution layer W V , where V e R C ×h×w , and then R1 and V are element-wise multiplied to obtain a final attention representation E, where the i-th feature map attention representation E i of the channel number C is:

[0026] E i = R1 o V i

[0027] where V i represents the feature converted by the feature map convolution of the i-th feature map of the channel number C;

[0028] The process in which the content attention module CnAM focuses on relatively shallow level features in the feature map is specifically as follows:

[0029] The content attention module CnAM converts the relatively shallow level feature map C4 e S C×h×w into a potential space by using a convolution layer W p and a convolution layer W z respectively, where the convolution layer W p is used to guide the focus on relatively shallow level features in the input feature map, and the generated feature vector constitutes a second query space P; the convolution layer W k is used to calculate the relevance or similarity of the relatively shallow level feature map in the second query space P, and the generated feature vector constitutes a second key space Z; the conversion formula is as follows:

[0030]

[0031] where {P, Z} e S C×h×w , and then the second query space P and the second key space Z are converted into vectors; in order to obtain the relationship between each local region or block, the following relationship matrix is obtained:

[0032] S = P T Z

[0033] where S e S N×N , N = h x w, and then normalized by a sigmoid activation function and average pooling to obtain an attention matrix S1 e S C×h×w ; finally, the feature map F is converted into a feature V by using a convolution layer W V , and then S1 and V are element-wise multiplied, and finally a convolution layer W V, to obtain the final attention representation D, the i-th feature map attention representation D of the channel number C i is:

[0034] D i = S1 o V i .

[0035] wherein V i represents the feature mapping convolution transformed features of the i-th feature map of the channel number C.

[0036] The second aspect of the present application provides a fishery illegal fishing behavior discrimination method based on an improved YOLOv10 algorithm, comprising the following steps:

[0037] acquire real-time images of the water area through a pan-tilt monitoring camera;

[0038] construct an image detection model based on the improved YOLOv10 algorithm, and perform model training and evaluation;

[0039] input the real-time images of the water area into the image detection model based on the improved YOLOv10 algorithm for identification, and obtain a detection result, if there is illegal fishing behavior discrimination information in the detection result, trigger an alarm function;

[0040] The model training and evaluation includes: obtaining an illegal fishing image dataset, and dividing it into a training set and a test set; performing iterative training on the training set, and retaining the best network weight; testing the model using the test set;

[0041] The iterative training on the training set and the retention of the best network weight are as follows: input the training set into the Backbone network module through the input layer, and then obtain the feature map of the training set through the Neck network module; perform prediction in the Head network module based on the feature map of the training set, calculate the loss, and update the model parameters; retrain until the loss function converges or the maximum number of iterations is reached, and save the best network weight.

[0042] The illegal fishing images include images of driving a ship to carry out illegal fishing operations in key water areas and images of using live bait to fish on the shore; the illegal fishing images are labeled using a labeling software to obtain a preliminary data set, and then each original picture in the preliminary data set is subjected to expansion processing, and the expansion processing includes simulating weather environment data expansion.

[0043] The third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the fishery illegal fishing behavior discrimination method based on the improved YOLOv10 algorithm is realized.

[0044] The fourth aspect of the present application provides a computer readable storage medium storing a computer program, wherein the computer program enables a computer to execute the improved fishery illegal fishing behavior identification method based on the YOLOv10 algorithm.

[0045] Compared with the prior art, the present application has the following advantages:

[0046] The present application constructs an improved image detection model based on the YOLOv10 algorithm, improves the original YOLOv10 network model, replaces the down-sampling module in the original Backbone network module with an HWD-ADown module, and introduces an AM attention module based on the C2f module in the Backbone network module, trains the model and evaluates the model.

[0047] Since the original down-sampling module may lose information, the present application uses the HWD-ADown module to retain as much image information as possible so that the subsequent layers can extract more discriminative features, thereby improving the performance of the model.

[0048] The HWD-ADown module includes an HWD module and an ADown module, wherein the HWD module includes a lossless feature encoding block and a feature representation learning block, the lossless feature encoding block uses Haar wavelet transform to decompose the image into four components: low-frequency component A, high-frequency component H in the horizontal direction, high-frequency component V in the vertical direction, and high-frequency component D in the diagonal direction, which can effectively reduce the spatial resolution of feature mapping while retaining all information.

[0049] The present application introduces an AM attention module based on the C2f module to generate an attention matrix, the AM attention module can adaptively focus on and weight the specified local area or block on the feature map and the relatively shallow level features on the feature map that contain more details and position information, so that the network can more accurately capture the position of each object to improve the performance of the network.

[0050] The AM attention module includes a context attention module CxAM and a content attention module CnAM, the CxAM mainly focuses on the specified local area or block on the feature map, and the CnAM focuses on the relatively shallow level features on the feature map, thereby reducing the influence of variable convolution on the target position, and the features output by the two sub-modules can be fused to obtain more comprehensive feature expression.

[0051] The present application uses an improved illegal fishing behavior identification model of the Backbone network module of YOLOv10, which can identify ships and personnel of different scales, improve the calculation redundancy and small target missing detection problem of the original algorithm, and improve the detection accuracy of illegal fishing behavior in key water areas. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 Fig. 1 is a flowchart of a method for identifying illegal fishing behavior based on a YOLOv10 algorithm improved by the present application.

[0053] Figure 2 Fig. 2 is a flowchart of a construction process of an image detection model based on a YOLOv10 algorithm improved by the present application.

[0054] Figure 3 Fig. 3 is a structural diagram of an improved Backbone network module of the present application.

[0055] Figure 4 Fig. 4 is a structural diagram of an HWD module in an HWD-ADown module.

[0056] Figure 5 Fig. 5 is a structural diagram of an ADown module in an HWD-ADown module.

[0057] Figure 6 Fig. 6 is a structural diagram of a CxAM module in an AM attention module.

[0058] Figure 7 Fig. 7 is a structural diagram of a CnAM module in an AM attention module.

[0059] Figure 8 Fig. 8 is a schematic diagram of an illegal fishing detection result of an embodiment of the present application. DETAILED DESCRIPTION

[0060] The above content of the present application will be further described in the form of examples, but this should not be understood as limiting the scope of the above subject matter of the present application to the following examples. Any technology realized based on the above content of the present application falls within the scope of the present application.

[0061] Some nouns in the present application are explained as follows:

[0062] The relatively shallow level features on the feature map referred to in this paper refer to features that have fewer processing levels in the neural network. The two are relative concepts, and the division range can be set according to specific needs in application. The features with more processing levels have a larger sense and can capture global context information in the image because they have undergone more layers of processing; while the features processed by fewer levels can retain more detailed information.

[0063] In an embodiment, the present application provides a system for identifying illegal fishing behavior based on a YOLOv10 algorithm improved by the present application, which comprises:

[0064] The water area real-time image acquisition module is configured to acquire real-time images of the water area by using the pan-tilt monitoring camera.

[0065] The improved image detection model based on the YOLOv10 algorithm comprises a Backbone network module for encoding input images in a feature form, a Neck network module for processing the feature form encoding, and a Head network module for generating model prediction output.

[0066] As shown in Figure 3 , the Backbone network module comprises an HWD-ADown module for decomposing feature maps and a C2f module for feature extraction and feature fusion, and the output end of the HWD-ADown module is connected to the C2f module.

[0067] The HWD-ADown module comprises an HWD module and an ADown module; the HWD module is configured to obtain a feature map decomposition processing result by performing Haar wavelet transformation on the input image; and the ADown module is configured to perform down-sampling operation on the feature map decomposition processing result of the HWD module.

[0068] The AM attention module is introduced based on the C2f module connected to the output end of the HWD-ADown module, and the AM attention module comprises a context attention module CxAM and a content attention module CnAM; the context attention module CxAM is configured to focus on a specified local area or block on the feature map, and the content attention module CnAM is configured to focus on relatively shallow level features on the feature map; and the output features of the context attention module CxAM and the content attention module CnAM are fused to obtain the final feature expression of the feature map.

[0069] The detection and discrimination module is configured to input the water area real-time image into the improved image detection model based on the YOLOv10 algorithm for identification to obtain a detection result, and if there is illegal fishing behavior in the detection result discrimination information of the discrimination model, an alarm function is triggered.

[0070] The HWD module, as shown in Figure 4 , comprises a lossless feature encoding block and a feature representation learning block; the lossless feature encoding block comprises a Haar wavelet transformation layer configured to perform Haar wavelet transformation on the input image to obtain image decomposition information; and the feature representation learning block is configured to extract discriminative features from the image decomposition information obtained by the Haar wavelet transformation layer to output a feature map decomposition processing result.

[0071] The ADown module, as shown in Figure 5As shown, the average pooling layer is connected to the two branches at the back end; in the ADown module, the input feature map is divided into two branches after the average pooling layer, one of which only passes through the CBS module, and the other of which passes through the max pooling layer and the CBS module in turn, and the results obtained by the two branches are output after feature fusion; the convolutional layers of the two branches are composed of the Conv module, the BN module and the SiLU module.

[0072] The process of the Haar wavelet transform layer performing Haar wavelet transform on the input image to obtain image decomposition information is specifically as follows:

[0073] After the image with a resolution of h*w is decomposed by Haar wavelet transform, h represents the image height, w represents the image width, four components are generated, including a low-frequency component A, a horizontal high-frequency component H, a vertical high-frequency component V and a diagonal high-frequency component D, and the size of each component is h / 2 x w / 2, the resolution is reduced to 1 / 4 of the original image resolution, and the number of feature mapping channels is increased by four times, while all information is retained, the spatial resolution of the feature mapping can be effectively reduced.

[0074] The feature representation learning block includes a 1x1 convolutional layer, a normalization layer and a ReLU activation function; the feature representation learning block extracts discriminative features from the image decomposition information obtained by the Haar wavelet transform layer, and the process of outputting the feature map decomposition processing result is specifically as follows:

[0075] The feature representation learning block uses a 1x1 convolutional layer to adjust the number of channels of the feature map to adapt to the subsequent layer and filter redundant information, and then combines the normalization layer and the ReLU activation function to extract discriminative features and obtain the feature map decomposition processing result.

[0076] As shown in Figure 6 , the process in which the context attention module CxAM focuses on the specified local area or block on the feature map is specifically as follows:

[0077] The context attention module CxAM converts the feature mapping F e R C×h×w into a potential space using a convolutional layer W q and a convolutional layer W k , C is the number of channels, h is the image height, and w is the image width; the convolutional layer W q is used to guide the focus on the specified local area or block in the input feature mapping, and the generated feature vector constitutes a first query space Q; the convolutional layer W k is used to calculate the relevance or similarity of the feature mapping of the specified local area or block in the first query space Q, and the generated feature vector constitutes a first key space K; the conversion formula is as follows:

[0078]

[0079] wherein {Q, K} ∈ R C×h×w Then the first query space Q and the first key space K are converted into vectors, in order to obtain the relationship between local areas or blocks, the following relationship matrix is obtained:

[0080] R = Q T K

[0081] wherein R ∈ R N×N , N = h x w, and then the R is normalized by a sigmoid activation function and average pooling to obtain an attention matrix R1 ∈ R C×h×w ; Then the feature map F is converted into a feature V by using a convolution layer W V , wherein V ∈ R C ×h×w The R1 and V are multiplied element-wise to obtain the final attention representation E, and the i-th feature map attention representation E i of the channel number C is:

[0082] E i = R1 ⊙ V i

[0083] wherein V i represents the feature converted by the feature map convolution of the i-th feature map of the channel number C.

[0084] The process in which the content attention module CnAM focuses on relatively shallow level features on the feature map is specifically:

[0085] The content attention module CnAM converts the relatively shallow level feature map C4 ∈ S C×h×w on the feature map into a potential space by using a convolution layer W p and a convolution layer W z , wherein the convolution layer W p is used to guide the focus on relatively shallow level features in the input feature map, and the generated feature vector constitutes a second query space P; the convolution layer W k is used to calculate the relevance or similarity of the relatively shallow level feature map in the second query space P, and the generated feature vector constitutes a second key space Z; the conversion formula is as follows:

[0086]

[0087] wherein {P, Z} ∈ S C×h×w , and then the second query space P and the second key space Z are converted into vectors; in order to obtain the relationship between each local area or block, the following relationship matrix is obtained:

[0088] S = P T Z

[0089] Where, S∈S N×N N = h × w, and then S is normalized using the sigmoid activation function and average pooling to obtain the attention matrix S1 ∈ S. C×h×w Finally, the feature map F is processed by a convolutional layer W. V The transformed features V are element-wise multiplied with S1, and finally a convolutional layer W is applied. V The final attention representation D is obtained, where D is the attention representation of the i-th feature map with C channels. i for:

[0090] D i =S1⊙V i .

[0091] Among them, V i The feature map represents the feature transformed by the convolution of the i-th feature map with C channels.

[0092] This invention improves upon the original YOLOv10 network model, such as... Figure 2 As shown, this includes: replacing the downsampling module (SCDown module) in the original YOLOv10 network model structure with the HWD-ADown module, and introducing the AM attention module on the basis of the C2f module after the HWD-ADown module, thus constructing an image detection model based on the improved YOLOv10 algorithm.

[0093] Since the original downsampling module may lose information, this invention uses the HWD-ADown module to preserve image information as much as possible, so that subsequent layers can extract more discriminative features, thereby improving model performance.

[0094] This invention introduces an AM attention module on the basis of the C2f module, which adaptively focuses on and weights specified local regions or blocks on the feature map and relatively shallow features on the feature map that contain more details and location information, so as to improve the performance of the network.

[0095] In some preferred embodiments, the present invention replaces the last downsampling module in the original YOLOv10 network model structure with an HWD-ADown module, and introduces an AM attention module into the C2f module connected to the back end of the HWD-ADown module.

[0096] In another embodiment of this invention, a method for identifying illegal fishing activities by fishery administration based on an improved YOLOv10 algorithm is proposed, such as... Figure 1 As shown, it includes the following steps:

[0097] Real-time images of the water area are captured using a PTZ camera.

[0098] An image detection model based on the improved YOLOv10 algorithm is constructed, and model training and evaluation are performed;

[0099] The real-time image of the water area is input into the image detection model based on the improved YOLOv10 algorithm for identification to obtain a detection result, and if there is illegal fishing behavior identification information in the detection result, an alarm function is triggered.

[0100] The image detection model based on the improved YOLOv10 algorithm is the image detection model based on the improved YOLOv10 algorithm in the above embodiment.

[0101] The model training and evaluation include: obtaining an illegal fishing image dataset, and dividing it into a training set and a test set, iteratively training on the training set, retaining the best network weight, and testing the model using the test set.

[0102] The iterative training on the training set retains the best network weight, and specifically: the training set is input into the Backbone network module through the input layer, and then the feature map of the training set is obtained through the Neck network module; the feature map of the training set is predicted in the Head network module, the loss is calculated, and the model parameters are updated; retraining until the loss function converges or the maximum number of iterations is reached, and the best network weight is saved.

[0103] The illegal fishing images include images of driving a ship to carry out illegal fishing operations in a key water area and images of using live bait fishing on the shore; the illegal fishing images are labeled using LabelImg software to obtain a preliminary dataset, and then each original picture in the preliminary dataset is subjected to augmentation processing, and the augmentation processing includes simulated weather environment data augmentation.

[0104] In another embodiment of the present application, an electronic device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the improved fishing law violation fishing behavior identification method based on the YOLOv10 algorithm is implemented.

[0105] In another embodiment of the present application, a computer readable storage medium is provided, which stores a computer program, and the computer program causes a computer to execute the improved fishing law violation fishing behavior identification method based on the YOLOv10 algorithm.

[0106] In embodiments disclosed herein, the computer storage medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The computer storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer storage medium can include, but are not limited to, an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] The technical solutions of the present application are further described in detail below in combination with specific embodiments:

[0108] Monitoring and discriminating illegal fishing behavior in a water body includes constructing an image detection model improved based on the YOLOv10 algorithm, training and evaluating the model, and the specific implementation process includes the following steps:

[0109] Step 1: Share the illegal fishing images provided by the government department with a size of 960 pixels x 960 pixels using the network platform, which includes images of driving a ship to carry out illegal fishing operations in key water areas and images of using live bait fishing on the shore. According to the standard of PASCAL VOC 2007 dataset, the illegal fishing images are labeled using LabelImg software to obtain the preliminary selected dataset, and then each original picture in the preliminary selected dataset is subjected to two expansion processes; the first data expansion process is to expand the original picture by simulating the weather environment, and the second data expansion process is to expand the data again by using one to three traditional data expansion methods; the dataset is divided into training set and test set according to the ratio of 3:2.

[0110] Step 2: Construct the HWD-ADown module to replace the last SCDown module in the original Backbone network module.

[0111] The HWD-ADown module is obtained by replacing the Conv module in the ADown module with the HWD module (Haar Wavelet Downsampling) structure, wherein the HWD structure refers to the HWD module structure diagram shown in Figure 4 , which includes lossless feature encoding block and feature representation learning block; the ADown module refers to Figure 5The diagram of the ADown module structure shown shows that after one average pooling layer, it splits into two branches. One branch goes through a convolutional layer, and the other branch goes through one max pooling layer and a convolutional layer. The convolutional layer consists of a Conv module, a BN module, and a SiLU module.

[0112] The lossless feature coding block utilizes a Haar wavelet transform layer, such as Figure 4 As shown, an image with a resolution of h*w is decomposed by Haar wavelet transform, where h represents the image height and w represents the image width, resulting in four components: a low-frequency component A, a high-frequency component H in the horizontal direction, a high-frequency component V in the vertical direction, and a high-frequency component D in the diagonal direction. The size of each component is h / 2×w / 2. This effectively reduces the spatial resolution of the feature map while preserving all information.

[0113] The feature representation learning block includes a standard convolutional layer, a normalization layer, and a ReLU activation function.

[0114] Step 3: Introduce the AM attention module into the C2f module at the output of the HWD-ADown module.

[0115] The AM attention module includes the context attention module CxAM and the content attention module CnAM. CxAM mainly focuses on a specified local region or block on the feature map, while CnAM focuses on relatively shallow features on the feature map, thereby reducing the influence of variable convolution on the target location. After the features output by the two sub-modules are fused, a more comprehensive feature representation can be obtained.

[0116] Step 4: Input the training set into the Backbone network through the input layer, and then obtain the feature map of the training set through the Neck network; make predictions based on the feature map of the training set in the Head network module, calculate the loss, and update the model parameters; retrain until the loss function converges or reaches the maximum number of iterations, and save the optimal network weights.

[0117] This embodiment uses the Tensorflow 2.2 framework to build the model, and the graphics card is an NVIDIA GeForce RTX 3060. The environment configuration is CUDA 11.8, cuDNN 7.6.5, Python 3.7, PyTorch 1.10.1, and OpenCV 4.4. The initial learning rate is 0.01, and the momentum factor is 0.9. The learning rate decay strategy is a fixed step size to ensure that the learning rate is appropriately adjusted after each training cycle, specifically by a factor of 0.3.

[0118] Step 5: Evaluate the model. Based on the YOLOv10-based improved illegal fishing discrimination model obtained after training, evaluate the detection accuracy of the model.

[0119] The precision P, recall R, F1 score, mean average precision mAP, and processing time are selected as the verification indexes of the improved YOLOv10 model, and the calculation formula is as follows:

[0120]

[0121] In the formula, TP is the number of defects correctly detected, FP is the number of defects misdetected, FN is the number of defects missed, n is the number of classes, and AP is the surrounding area formed by P and R under different confidence conditions.

[0122] Then, the real-time image of the water area is collected by the pan-tilt monitoring camera, the real-time image of the water area is recognized by using the image detection model improved based on the YOLOv10 algorithm constructed in the embodiment, and the detection result is obtained. If there is illegal fishing behavior in the detection result discrimination information of the discrimination model, the alarm function is triggered, and the specific implementation process is as follows:

[0123] The real-time image of a water area is collected by using the pan-tilt monitoring camera, the images including driving a ship to carry out illegal fishing operation in a key water area and using live bait to fish on the shore are obtained, and data enhancement is performed. The enhanced data set has a total of 1259 images, which are divided into a training set and a test set according to a ratio of 3:2, and are input into the image detection model improved based on the YOLOv10 algorithm and the original YOLOv10 model for training and testing. The average accuracy, average recall, F1 score, mean average precision mAP, and processing time are selected as the verification indexes of the model, the performance of the improved algorithm in the embodiment is compared with that of the model before improvement, and the evaluation comparison result is shown in Table 1. The results show that the performance of the improved YOLOv10 detection method is better than that of other detection methods, and each index is higher than that of the method before improvement. In actual application, the mAP@0.5:0.95 is improved by 3.72%.

[0124] Table 1: Performance evaluation comparison of the model

[0125]

[0126] The illegal fishing image to be detected is input into the trained illegal fishing discrimination model based on the improved YOLOv10 to be recognized, and the detection result is obtained, as shown in the illegal fishing detection result schematic diagram shown in Figure 8

[0127] If there is illegal fishing behavior, the real-time voice alarm function of the intelligent sound box is triggered through the intelligent fishery supervision platform, and relevant personnel are notified by sending a short message to go to law enforcement and disposal. The intelligent fishery supervision platform is a comprehensive system platform developed by China Telecom, which accesses the NB-IoT device of China Telecom through the LWM2M protocol, supports the 24-hour unified control and short message sending of the intelligent sound box.​

[0128] In embodiments disclosed herein, a computer storage medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. Computer storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer storage media can include, but are not limited to, an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] The above description is only the preferred embodiment of the present application, not any form of limitation to the present application, any skilled person in the art, without departing from the technical solution of the present application, according to the technical essence of the present application, any simple modification, equivalent replacement and improvement of the above embodiment, etc., still belongs to the protection scope of the technical solution of the present application.

Claims

1. A fishery law violation fishing behavior discrimination system based on an improved YOLOv10 algorithm, characterized in that, The utility model relates to a water area real-time image acquisition module for acquiring water area real-time images through a pan-tilt monitoring camera, a YOLOv10 algorithm-improved image detection model including a Backbone network module for encoding input images in a feature form, a Neck network module for processing the feature-form encoded images, and a Head network module for generating model prediction outputs, the Backbone network module including an HWD-ADown module for decomposing feature maps and a C2f module for feature extraction and fusion, the output of the HWD-ADown module being connected to the C2f module, the HWD-ADown module including an HWD module for subjecting input images to Haar wavelet transformation to obtain feature map decomposition processing results and an ADown module for performing down-sampling operations on the feature map decomposition processing results of the HWD module, an AM attention module being introduced on the basis of the C2f module connected to the output of the HWD-ADown module, the AM attention module including a context attention module CxAM for focusing on specified local areas or blocks on a feature map and a content attention module CnAM for focusing on relatively shallow-level features on the feature map, the output features of the context attention module CxAM and the content attention module CnAM being fused to obtain final feature expressions of the feature map, and a detection and discrimination module for inputting water area real-time images into the YOLOv10 algorithm-improved image detection model for identification to obtain detection results, the alarm function being triggered if there is illegal fishing behavior in the detection result discrimination information of the discrimination model. The HWD module includes a lossless feature encoding block and a feature representation learning block, the lossless feature encoding block including a Haar wavelet transformation layer for subjecting input images to Haar wavelet transformation to obtain image decomposition information, and the feature representation learning block for extracting discriminative features from the image decomposition information obtained by the Haar wavelet transformation layer to output feature map decomposition processing results. In the ADown module, the input feature map is divided into two branches after passing through an average pooling layer, one branch only passes through a CBS module, and the other branch sequentially passes through a maximum pooling layer and a CBS module, and the results obtained by the two branches are output after feature fusion. The process of the Haar wavelet transformation layer for subjecting input images to Haar wavelet transformation to obtain image decomposition information is as follows: After a h*w resolution image is decomposed by Haar wavelet transformation, h represents the image height, w represents the image width, four components are generated, including a low-frequency component A, a horizontal high-frequency component H, a vertical high-frequency component V, and a diagonal high-frequency component D, each component has a size of h / 2×w / 2, the resolution is reduced to 1 / 4 of the original image resolution, and the number of feature mapping channels is increased by four times. ​ ​ 2. The fishery illegal fishing behavior discrimination system based on the improved YOLOv10 algorithm according to claim 1, characterized in that: ​ 3. The fishery illegal fishing behavior discrimination system based on the improved YOLOv10 algorithm according to claim 1, characterized in that: ​ 4. The fishery illegal fishing behavior discrimination system based on the improved YOLOv10 algorithm according to claim 2, characterized in that: ​ ​ The feature representation learning block includes a 1x1 convolutional layer, a normalization layer, and a ReLU activation function; the feature representation learning block extracts discriminative features from image decomposition information obtained by the Haar wavelet transform layer, and the process of outputting a feature map decomposition processing result is specifically as follows: The feature representation learning block uses a 1x1 convolutional layer to adjust the number of channels of the feature map, and then extracts discriminative features in combination with a normalization layer and a ReLU activation function to obtain a feature map decomposition processing result.

5. The fishery illegal fishing behavior discrimination system based on the improved YOLOv10 algorithm according to claim 1, characterized in that: The process in which the context attention module CxAM focuses on a specified local area or block on the feature map is specifically as follows: The context attention module CxAM maps the features of the feature map F e R C×h×w respectively using the convolution layer W q and the convolution layer W k into the potential space, C is the number of channels, h is the image height, and w is the image width; the convolution layer W q is used to guide the attention to the specified local region or block in the input feature map, and the generated feature vector constitutes the first query space Q; the convolution layer W k is used to calculate the relevance or similarity of the feature map of the specified local region or block in the first query space Q, and the generated feature vector constitutes the first key space K; the conversion formula is as follows: where {Q, K} ∈ R C×h×w Then the first query space Q and the first key space K are converted into vectors, in order to obtain the relationship between the local areas or blocks, the following relationship matrix is obtained: R = Q T K wherein R∈R N×N , N = h x w, and then R is normalized by a sigmoid activation function and average pooling to obtain an attention matrix R1∈R C×h×w ; then a convolution layer W V is used to convert the feature map F into a feature V, wherein V∈R C×h×w , and R1 and V are multiplied element-wise to obtain a final attention representation E, and the i-th feature map attention representation E i of the channel number C is: E i = R1 V i wherein V i represents the feature map convolution transformed features of the i-th feature map of the channel number C; The process in which the content attention module CnAM focuses on relatively shallow level features on the feature map is specifically as follows: The content attention module CnAM maps relatively shallow level features on the feature map C4∈S C×h×w respectively using convolutional layers W p and convolutional layers W z transformed into potential space, the convolutional layers W p for guiding attention to relatively shallow level features in the input feature map, the generated feature vector constitutes a second query space P; the convolutional layers W k for calculating the relevance or similarity of relatively shallow level feature maps in the second query space P, the generated feature vector constitutes a second key space Z; the transformation formula is as follows: where {P, Z}∈S C×h×w Then the second query space P and the second key space Z are converted into vectors; in order to obtain the relationship between each local region or block, the following relationship matrix is obtained: S = P T Z Where, S∈S N×N N = h × w, and then S is normalized using the sigmoid activation function and average pooling to obtain the attention matrix S1 ∈ S. C×h×w Finally, the feature map F is processed by a convolutional layer W. V The transformed features V are element-wise multiplied with S1, and finally a convolutional layer W is applied. V The final attention representation D is obtained, where D is the attention representation of the i-th feature map with C channels. i for: D i = S1 V i wherein V i represents the feature map convolution-transformed features of the i-th feature map of channel number C.

6. A method for identifying illegal fishing behavior based on an improved YOLOv10 algorithm, characterized in that, The method comprises the following steps: Real-time images of the water area are collected by a pan-tilt monitoring camera; An image detection model based on an improved YOLOv10 algorithm is constructed, and model training and evaluation are performed; The real-time images of the water area are input into the image detection model based on the improved YOLOv10 algorithm for identification, and a detection result is obtained; if there is illegal fishing behavior identification information in the detection result, an alarm function is triggered; The image detection model based on the improved YOLOv10 algorithm is the image detection model based on the improved YOLOv10 algorithm according to any one of claims 1-5.

7. The method according to claim 6, wherein the YOLOv10 algorithm is improved. The model training and evaluation comprises: obtaining an illegal fishing image dataset, and dividing the dataset into a training set and a test set; performing iterative training on the training set, and retaining the best network weight; and testing the model using the test set; The iterative training on the training set and the retention of the best network weight are specifically as follows: the training set is input into the Backbone network module through the input layer, and then the feature map of the training set is obtained through the Neck network module; prediction is performed on the feature map of the training set in the Head network module, a loss is calculated, and model parameters are updated; the model is retrained until the loss function converges or the maximum number of iterations is reached, and the best network weight is saved.

8. The method of claim 7, wherein the YOLOv10 algorithm is improved. The illegal fishing images include images of illegal fishing operations by driving a ship in a key water area and images of fishing using live bait on the shore; The illegal fishing images are labeled using labeling software to obtain a preliminary dataset, and each original image in the preliminary dataset is then subjected to expansion processing, which comprises simulating weather environment data expansion.

9. An electronic device, comprising: The method comprises: collecting real-time images of a water area by a pan-tilt monitoring camera; constructing an image detection model based on an improved YOLOv10 algorithm, and performing model training and evaluation; inputting the real-time images of the water area into the image detection model based on the improved YOLOv10 algorithm for identification, and obtaining a detection result; if there is illegal fishing behavior identification information in the detection result, an alarm function is triggered; The image detection model based on the improved YOLOv10 algorithm is the image detection model based on the improved YOLOv10 algorithm according to any one of claims 1-5. The model training and evaluation comprises: obtaining an illegal fishing image dataset, and dividing the dataset into a training set and a test set; performing iterative training on the training set, and retaining the best network weight; and testing the model using the test set; The iterative training on the training set and the retention of the best network weight are specifically as follows: the training set is input into the Backbone network module through the input layer, and then the feature map of the training set is obtained through the Neck network module; prediction is performed on the feature map of the training set in the Head network module, a loss is calculated, and model parameters are updated; the model is retrained until the loss function converges or the maximum number of iterations is reached, and the best network weight is saved. The illegal fishing images include images of illegal fishing operations by driving a ship in a key water area and images of fishing using live bait on the shore; The illegal fishing images are labeled using labeling software to obtain a preliminary dataset, and each original image in the preliminary dataset is then subjected to expansion processing, which comprises simulating weather environment data expansion. The method comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the illegal fishing behavior identification method based on the improved YOLOv10 algorithm according to any one of claims 6-8 when executing the computer program. 10.A computer readable storage medium storing a computer program, wherein the computer program causes a computer to execute the illegal fishing behavior identification method based on the improved YOLOv10 algorithm according to any one of claims 6-8.

Images