ML-YOLOv8 network model structure-based giant salamander juvenile protection behavior automatic identification system
Through the automatic recognition system of giant salamander child protection behavior based on the ML-YOLOv8 network model, combined with multi-scale convolution and large-segment separable nuclear attention mechanism, the problem of time-consuming and subjectiveness of artificial observation in the recognition of giant salamander child protection behavior is solved, and efficient , accurate behavior recognition improves recognition rate and model robustness.
Patent Information
- Application Number
- CN202410944082.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-12
AI Technical Summary
The prior art has time-consuming and susceptible to subjectivity in the recognition of giant salamander child care behavior. The application of deep learning-based methods in amphibian behavior recognition has not been reported, resulting in low recognition efficiency and insufficient accuracy.
The automatic recognition system for the protection of the giant salamander is adopted based on the ML-YOLOv8 network model structure. By integrating the multi-scale convolution of Res2Net, introducing the large separable nuclear attention (LSKA) mechanism, and using the WIoU loss function, the YOLOv8 model is optimized to improve the characteristics. Extraction capability and robustness.
The accuracy and efficiency of the recognition of the behavior of giant salamander guardians has been significantly improved. The recognition rate of the model on mAP50-95 reached 85.7%, which is 2.1% higher than that of the YOLOv8 model, and it reduces interference in complex environments and enhances generalization ability. .
Smart Images

Figure CN118941767A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an automatic animal behavior recognition method, and in particular to an automatic recognition system for giant salamander protecting young based on an ML-YOLOv8 network model structure. Background Art
[0002] The giant salamander (Andrias davidianus) is the largest and oldest amphibian in the world. It is a flagship species of endangered amphibians and a rare species unique to China. The national second-level key protected wild animal is limited to the wild population of giant salamanders, and artificially cultivated giant salamanders can be reasonably used. Giant salamanders have high nutritional value and are rich in 8 essential amino acids required by the human body, with basically no limiting amino acids. They have good development and utilization potential in the fields of food, health care, and medicine, and are known as the "ginseng in water". In the late 1990s, artificial breeding of giant salamanders emerged and developed rapidly. Although the artificial breeding and breeding technology of giant salamanders has become increasingly mature, it still faces many technical bottlenecks, especially the low level of seedling breeding technology and high seedling cost, which greatly limit the economic benefits of the giant salamander breeding industry. Among them, the giant salamander's young protection behavior determines the hatching rate of its eggs, which is crucial to the survival and reproduction of the species. Therefore, accurately grasping the laws of giant salamander breeding behavior is a necessary basis for breaking through the bottleneck of giant salamander breeding technology.
[0003] Research on the breeding behavior of Chinese giant salamanders has developed to the point where infrared imaging monitoring systems are used to obtain videos of the breeding behavior of Chinese giant salamanders (Andrias davidianus) using a digital monitoring system. Animals, 2018, 8, 161.) and the video footage is manually replayed to observe and record the characteristics of the behavior of Chinese giant salamanders protecting their young. Manual observation of such animal behavior is labor-intensive, time-consuming and inefficient; the results of behavioral observations are easily affected by the subjective consciousness of the observer; and due to the complexity and large amount of observation data, data processing and analysis of manual observations are difficult.
[0004] With the development of deep learning, image feature extraction has been automated, and visual features based on deep learning have gradually been applied to animal behavior detection models. The Faster R-CNN model was used to realize the recognition of fish behavior under different ammonia concentrations (Xu W., Zhu Z., Ge F., Han Z., Li J. Analysis of behavior trajectory based on deep learning in ammonia environment for fish. Sensors 2020, 20, 4425.). By designing a new residual structure for the backbone network and introducing the supervisory signal Center Loss to construct a joint classification loss function, Faster R-CNN was optimized, and the accuracy and speed of Faster R-CNN in sow behavior detection were improved. Based on the idea of SKNet network, an adaptive scale attention mechanism was proposed to optimize the Faster R-CNN model, which enhanced the feature extraction capability of multi-scale complex targets. The original feature extraction network VGG-16 was replaced with a lightweight network MobileNet v2, and the receptive field was enhanced through a variable convolutional layer, which improved the detection effect of the SSD model on abnormal crowd behavior (Kang J., Tian Y. Yang G. Research on Crowd Abnormal Behavior Detection Based on Improved SSD. Infrared Technology. 2022, 44, 1316.). The YOLOv3 model was optimized by improving the pooling block and loss function, which improved the accuracy and performance of YOLOv3 on fish behavior. A small target detection framework was added to the YOLOv5 model for multi-scale fusion, which improved the detection accuracy of abnormal behavior of perch. A large-size detection head was added to YOLOv8, and then the ECAM attention mechanism was called to reduce the interference of blurred background, focus on the key features of individual fish, and enhance the recognition ability of blurred fish.
[0005] Although deep learning-based object detection methods have been widely used in the field of animal behavior recognition and have achieved remarkable results, there are no reports on the application of this technology in amphibian behavior recognition, and the research on the recognition of giant salamander guarding behavior is still in its infancy. Therefore, this paper proposes a method for recognizing giant salamander guarding behavior based on the ML-YOLOv8 model structure. Summary of the invention
[0006] The present invention aims to overcome the shortcomings of existing deep learning technology in detecting the protective behavior of giant salamanders. At present, there are several difficulties in the method of relying on infrared monitoring systems to capture and analyze the reproductive behavior of giant salamanders. First, the operator must have the corresponding professional knowledge. In addition, manual observation and analysis consumes a lot of manpower and may introduce subjectivity. The present invention provides an automatic recognition system for the protective behavior of giant salamanders based on the ML-YOLOv8 network model structure.
[0007] In order to solve the above technical problems, the present invention proposes a ML-YOLOv8 model structure for building an automatic recognition system for giant salamander guarding young behavior, including:
[0008] The ML-YOLOv8 model consists of four parts: input end, backbone network, neck network and detection head;
[0009] The input end performs data enhancement and adaptive image scaling. The data enhancement adopts Mosaic enhancement. The Mosaic enhancement part of YOLOv8 introduces the operation of using Mosaic enhancement in the last 10 epochs in YOLOX, which enhances the robustness of the model. The adaptive image scaling uniformly scales the original image to the same size.
[0010] The backbone network is composed of Conv, C2f, C2f-MSConv and SPPF-LSKA modules. The Conv module includes Conv2d, BN, and SLU activation functions. The Conv module performs convolution operations on feature maps. The C2f module is a feature extraction module. The sixth and eighth layers of the backbone network in the C2f module and the neck network are set to C2f-MSConv for multi-scale fusion. The SPPF-LSKA module is used to extract features of different receptive fields.
[0011] The neck network consists of the FPN-PAN structure. FPN is top-down. Through upsampling, the high-level feature information is fused with the bottom-level feature information to calculate the predicted feature map. PAN is an improvement on FPN. It introduces lateral connections to enhance the semantic information of the features, so that the bottom-up feature map can be fused with the top-down feature map.
[0012] The detection head uses an anchor-free matching mechanism, which does not require preset anchors. It only needs to regress the target center point and width and height of feature maps of different scales. By utilizing the rich information from feature maps of different scales, it can accurately obtain the classification and location information of large, medium and small-sized target objects.
[0013] The multi-scale convolution of C2f-MSConv is used for detection tasks. The object information captured under different receptive fields is different. The small receptive field captures the details of the object, which is conducive to detecting small targets. The large receptive field captures the overall structure of the object, which helps the network locate the position of the object.
[0014] The MSConv multi-scale convolution is designed based on the idea of Res2Net. The core structure of Res2Net adopts formula (1):
[0015]
[0016] Feature K2 is processed by a 3×3 convolution layer, which is located at the X3 position in the module architecture. After this initial convolution, K2 is optimized for the second time by another 3×3 convolution. The effect of 5×5 convolution is simulated by the effective combination of two consecutive 3×3 convolutions. Subsequently, K3 is enhanced by integrating the processed features from 3×3 and composite 5×5 receptive fields. Similarly, feature K4 is processed by a 7×7 receptive field, further expanding the scale of feature detection.
[0017] The SPPF-LSKA module captures long-range dependency and adaptability by decomposing large-kernel convolution operations, and decomposes the decomposed 2D convolution kernels into serially connected 1D kernels, thereby reducing computational complexity and memory requirements. The SPPF-LSKA module first decomposes a K×K convolution into (2d-1)×(2d-1) depth convolution, K / d×K / d depth expansion convolution, and 1×1 convolution, and then further decomposes the 2D depth convolution kernel and the depth expansion convolution kernel into 1D horizontal convolution kernel and vertical convolution kernel, and finally serializes the decomposed convolution kernels in sequence.
[0018] The ML-YOLOv8 model adopts a new bounding box regression loss function WIoU. The WloU loss function dynamically reduces the penalty on the geometric metric when the overlap between the anchor box and the target box is high. The expression of the WloU loss function is as shown in formula (2), formula (3), and formula (4):
[0019]
[0020] Where W and H represent the width and height of the bounding box composed of the predicted box and the real box, respectively, and * means separating the W and H of the minimum bounding box from the gradient calculation.
[0021] The experimental operating system is Ubuntu 20.04, PyTorch is used as the framework for developing deep learning models, the input image is normalized to a size of 640 × 640, the batch size is set to 16, the training is performed for 300 cycles, the learning capacity used during model training is 0.01, the SGD momentum is 0.937, the optimizer weight decay is 0.0005, and all other training parameters are set to the default values of the YOLOv8 network.
[0022] Optionally, the performance of the giant salamander guarding behavior detection model is objectively evaluated. The evaluation indicators used include GFLOPS, billion floating-point operations per second, which quantifies the network model execution time and is measured in billion floating-point operations per second. The parameter is used to evaluate the size and complexity of the model; FPS, frames per second, is used to measure the detection speed of the model and is measured in the number of frames transmitted per second; mAP, mean average precision, is used to evaluate the accuracy of the model.
[0023] Precision is the ratio of the number of target boxes correctly predicted as positive by the model to the number of all target boxes predicted as positive by the model, and is defined as formula (5):
[0024]
[0025] Among them, TP represents true positives, which are the number of target frames correctly predicted as positive, FP represents false positives, which are the number of target frames incorrectly predicted as positive, and FN represents false negatives, which are the number of target frames that are actually in the positive class but are incorrectly predicted as negative.
[0026] Recall is the ratio of the number of target frames correctly predicted by the model as positive to the number of target frames in all actual positive classes, defined as formula (6):
[0027]
[0028] AP, average precision, is the area under the precision-recall curve, representing the average precision of the model at different recall rates, and is defined as formula (7):
[0029]
[0030] mAP, the mean average precision, is a comprehensive indicator used to evaluate the performance of object detection models on multiple categories. The average precision AP of each category is calculated, and then these AP values are averaged to measure the performance of the model, which is defined as formula (8):
[0031]
[0032] The i in formula (8) indicates the type of behavior. Formula (8) means that the AP value of the first type of behavior is accumulated to the AP value of the Nth type, and then divided by N to obtain the average accuracy of the N types of behaviors.
[0033] Where N represents the number of categories in the dataset, and N is taken as 6. The higher the mAP value, the better the performance of the model.
[0034] The present application also includes a device for a system for automatically identifying the protection behavior of giant salamanders based on the ML-YOLOv8 network model, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program, the computer program encoding realizing the above-mentioned system for automatically identifying the protection behavior of giant salamanders based on the ML-YOLOv8 network model;
[0035] It also includes matching image taking equipment and display equipment.
[0036] Through the above network structure system, the present invention designs and constructs a system and device for automatically identifying the giant salamander's young protection behavior network model based on the improved YOLOv8s. First, by integrating the design concept of Res2Net, a multi-scale convolution (MSConv) is proposed, which is used in combination with the C2f module to significantly enhance the feature extraction ability of the model. Secondly, the idea of Large Separable Kernel Attention (LSKA) is introduced to optimize the spatial pyramid pooling (SPPF) layer, which effectively reduces the interference factors in complex environments. In addition, in order to solve the problem of low quality of captured images, WIoU is used to replace CIoU in the original YOLOv8 model, optimize the loss function and improve the robustness of the network. Experimental results show that the model is 2.1% higher than the YOLOv8 model in mAP50-95. Compared with other mainstream models, the overall performance of this model is much better.
[0037] This paper constructs a dataset of images of giant salamanders protecting their young, which is the first dataset of amphibian behavior images. The following factors are considered:
[0038] 1. Combining the idea of Res2Net, this paper proposes a multi-scale fusion convolution (MSConv), which is combined with the C2f module to form C2f-MSConv. Experiments show that this module can effectively improve the feature extraction ability of the model.
[0039] 2. Combining LSKA with the SPPF (Spatial Pyramid Pooling Fusing) module in YOLOv8 enhances the long-distance dependency in the input image and reduces the interference of complex background on the detection of giant salamander guarding behavior.
[0040] 3. The use of WIoU loss function effectively solves the large positioning error and missed detection problems caused by low-quality data collected on edge devices, thereby enhancing the generalization ability of the model.
[0041] 4. For the first time, a deep learning model was used to automatically identify the parent behavior of giant salamanders, which optimized the method of observing the protective behavior of giant salamanders, promoted the application of information technology in the study of amphibian behavioral ecology, and provided a reference for the study of other amphibians and aquatic animals. Mastering the protective behavior characteristics of giant salamanders is crucial to optimizing their breeding techniques and improving the hatching rate of offspring. However, since giant salamanders are nocturnal and burrowing, they are active at night and often live in dark caves. Their behavioral image data can be obtained through infrared image monitoring, but the manual identification and statistics of their behavioral image data are huge, time-consuming and inefficient, and are subject to the subjectivity of the observer. In order to solve this problem, the present invention proposes a system for automatic identification of giant salamander protective behavior based on the ML-YOLOv8 network model, which is a method for automatically identifying giant salamander protective behavior based on the improved YOLOv8 model. This method includes three key components. First, a multi-scale convolution (MSConv) is proposed by integrating the design concept of Res2Net, which is used in combination with the C2f module to significantly enhance the feature extraction capability of the model. Secondly, the idea of Large Separable Kernel Attention (LSKA) is introduced to optimize the spatial pyramid pooling (SPPF) layer, which effectively reduces the interference factors in complex environments. In addition, in order to solve the problem of low quality of captured images, WIoU is used to replace CIoU in the original YOLOv8 model, which optimizes the loss function and improves the robustness of the network. Experimental results show that the recognition rate of the model on mAP50-95 is 85.7%, which is 2.1% higher than the YOLOv8 model. Compared with other mainstream models, the overall performance of this model is much better. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to make the technical problems solved by the present invention, the technical means adopted and the technical effects achieved more clearly, the specific embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, it should be noted that the drawings described below are only drawings of exemplary embodiments of the present invention, and those skilled in the art can obtain drawings of other embodiments based on these drawings without creative work.
[0043] Figure 1 It shows the camera and water quality monitoring system for the ecological giant salamander breeding pool and its behavior monitoring. Figure 1 (a) is the camera system for monitoring the behavior of giant salamanders protecting their young, and (b) is the water quality monitoring system.
[0044] Figure 2 This is an example of a dataset showing parental care behavior of a giant salamander. Figure 2 (a) Fanning the tail; (b) Stirring; (c) Vibrating; (d) Eating; (e) Entering the cave; (f) Exiting the cave.
[0045] Figure 3 is a schematic diagram showing a standard YOLOv8 model.
[0046] Figure 4 The figure shows the ML-YOLOv8 network model structure of the automatic identification system for giant salamander protecting young in the present invention.
[0047] Figure 5 Schematic diagram showing the core structure of Res2Net of the present invention, wherein: (a) Res2Net module; (b) MSConv structure.
[0048] Figure 6 FIG. 1 shows the decomposition process of a large kernel convolution of the present invention.
[0049] Figure 7 FIG. 1 is a diagram showing the process of decomposing a 2D convolution into 1D in the present invention.
[0050] Figure 8 FIG. 1 is a diagram illustrating the LSKA attention mechanism of the present invention.
[0051] Fig. 9 is a structural diagram showing the SPPF-LSKA module of the present invention.
[0052] Fig.10 Graph 1 shows the relevant parameter definitions of the WIoU loss function of the present invention.
[0053] Fig.11 It shows the basic model YOLOv8s and the network model structure system for automatically identifying the giant salamander's young protection behavior based on the ML-YOLOv8 network model structure of the present invention.
[0054] Comparison chart of the detection of giant salamander. Fig.11 Figures (a), (c), and (e) are the detection results of YOLOv8. Fig.11 (b), (d), and (f) are the detection results of the same image by the automatic recognition system for giant salamander protecting young with an ML-YOLOv8 network model structure of the present invention. DETAILED DESCRIPTION
[0055] The exemplary embodiments of the present invention are described more fully with reference to the accompanying drawings. Although each exemplary embodiment can be implemented in a variety of specific ways, it should not be understood that the present invention is limited to the embodiments described herein. On the contrary, these exemplary embodiments are provided to make the content of the present invention more complete and to facilitate the comprehensive communication of the inventive concept to those skilled in the art.
[0056] Under the premise of conforming to the technical concept of the present invention, the structure, performance, effect or other characteristics described in a specific embodiment may be combined with one or more other embodiments in any appropriate manner.
[0057] In the process of introducing specific embodiments, the detailed description of the structure, performance, effect or other characteristics is to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can implement the present invention with a technical solution that does not contain the above-mentioned structure, performance, effect or other characteristics under certain circumstances.
[0058] The flowchart in the accompanying drawings is only an exemplary process demonstration, and does not mean that the solution of the present invention must include all the contents, operations and steps in the flowchart, nor does it mean that it must be executed in the order shown in the figure. For example, some operations / steps in the flowchart can be decomposed, some operations / steps can be combined or partially combined, etc. The execution order shown in the flowchart can be changed according to the actual situation without departing from the inventive spirit of the present invention.
[0059] Frames in the attached figure Figure 1 Generally, it refers to functional entities, which do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0060] The same reference numerals in the drawings represent the same or similar elements, components or parts, and thus the repeated description of the same or similar elements, components or parts may be omitted below. It should also be understood that although the first, second, third and other attributives representing the numbers may be used herein to describe various devices, elements, components or parts, these devices, elements, components or parts should not be limited by these attributives. In other words, these attributives are only used to distinguish one from another. For example, the first device may also be called the second device, but it does not deviate from the essential technical solution of the present invention. In addition, the terms "and / or" and "and / or" refer to all combinations including any one or more of the listed items.
[0061] The materials and methods of the automatic identification system of giant salamander protecting young based on the ML-YOLOv8 network model structure of the present invention are as follows.
[0062] The origin of the dataset.
[0063] The images were taken from surveillance videos of four pairs of giant salamanders that were successfully bred at the ecological breeding base of Zhangjiajie Zhuyuan Giant Salamander Biotechnology Co., Ltd. in Tangxiyu Village, Kongkeshu Township, Sangzhi County, Zhangjiajie City from August to October each year from 2020 to 2022. The base's simulated ecological breeding pool consists of artificial streams and caves, distributed on both sides of the artificial stream, such as Figure 1 As shown, a natural giant salamander breeding pool is simulated. Figure 2 This is an example of a dataset showing parental care behavior of giant salamanders. Figure 2 (a) fanning tail, (b) stirring, (c) shaking, (d) eating, (e) entering the hole, and (f) leaving the hole. Using video editing software (Format Factory), the key frames of the giant salamander's young protection behavior were randomly selected from the video to create an image dataset. The quality of the images in the dataset will significantly affect the training effect of the model. The images extracted were checked one by one to delete the low-quality images. Finally, the behavioral image dataset (3000 images) of this study was obtained, including 6 behaviors, namely fanning tail (600 images), stirring (700 images), shaking (500 images), eating eggs (700 images), entering the hole (250 images), and leaving the hole (250 images). The "LabelImg" image annotation tool was used to manually define and annotate the image dataset. The evaluation criteria for each behavior are shown in Table 1. The training set and the validation set were randomly selected at a ratio of 8:2.
[0064] Table 1 Dataset classification
[0065] Behavior Type Judging Criteria Label Sample size fantail The tail is beside or inside the egg pile, swinging left and right shanwei 600 agitation Head inserted into the egg mass or body passed through the egg mass jiaodong 700 shock The head or body straddles the top or the vicinity of the egg pile and swings left and right or up and down zhendong 500 Eat eggs Open mouth to hold the egg or shake head chsihi 700 Enter the cave Only the head appears at the hole entrance jindong 250 Exit Only the tail appears at the hole entrance chudong 250
[0066] In order to solve the problems existing in the YOLOv8 model, the present invention proposes an ML-YOLOv8 network model structure for identifying the young protecting behavior of giant salamander.
[0067] like Figure 4 As shown in the figure, it is the structure diagram of ML-YOLOv8. First, C2f-MSConv is used to replace the sixth and eighth layers in the backbone network and the C2f module in the neck network; secondly, the SPPF in the backbone network is replaced by SPPF-LSKA; finally, the original CIoU loss function in YOLOv8 is replaced by WIoU. The C2f-MSConv module performs multi-scale fusion to enhance the ability to extract the characteristics of giant salamander protection behavior at different scales. The SPPF-LSKA module is to reduce the interference of irrelevant background features on the detection of giant salamander protection behavior and improve the accuracy of the model. Replace the loss function to enhance the generalization ability of the model.
[0068] Propose C2f-MSConv multi-scale convolution.
[0069] Multi-scale features have always been an important aspect in detection tasks. Since the introduction of zero convolution, the multi-scale pyramid model based on zero convolution has achieved milestone results in detection tasks. The object information captured under different receptive fields is different. Small receptive fields can capture more details of the object, which is also very beneficial for detecting small targets. In contrast, large receptive fields can capture the overall structure of the object and help the network locate the position of the object. Combining details and positions can better extract clear object information.
[0070] The core structure of Res2Net is Figure 5 As shown in Figure 2, feature K2 is processed through a 3×3 convolution layer, which is located at position X3 in the module architecture. After this initial convolution, K2 is refined a second time through another 3×3 convolution, which simulates the effect of a 5×5 convolution through the effective combination of two consecutive 3×3 convolutions. Subsequently, K3 is enhanced by integrating the processed features from the 3×3 and composite 5×5 receptive fields. Similarly, feature K4 is processed through a 7×7 receptive field, further expanding the scale of feature detection. The formula is as follows:
[0071]
[0072] Based on the concept of Res2Net, we designed a new type of multi-scale convolution (MSConv). Figure 5 As shown in the figure, the number of original input channels is divided into four parts using grouped convolution, and features of different scales are extracted through 1×1, 3×3, 5×5, and 7×7 convolution operations. Finally, a 1×1 convolution layer is used to fuse multi-scale features, achieving comprehensive feature extraction and efficient integration.
[0073] By combining MSConv with the C2f module, C2f-MSConv is proposed, which replaces part of the original C2f in YOLOv8. It not only significantly reduces feature redundancy and improves the efficiency of feature extraction, but also makes full use of features of different scales, thereby further enhancing the performance of the neural network.
[0074] SPPF-LSKA module.
[0075] Due to the complex environment in which giant salamanders live, in order to improve the model's ability to extract key features of giant salamanders' young-guarding behavior, the LSKA (Large Separable Kernel Attention) attention mechanism is introduced into the SPPF module in the YOLOv8 backbone network, so that the network ignores irrelevant background information interference and pays attention to more effective giant salamanders' young-guarding behavior feature information. Previous attention mechanisms such as self-attention and Large Kernel Attention (LKA) have many shortcomings. Self-attention has strong long-range dependence and adaptability, but ignores the two-dimensional structure of the image. LSA attention solves the problem that self-attention ignores the two-dimensional structure of the image, but its computational complexity is too large when processing large-size convolution kernels. The LSKA attention mechanism introduced in this paper solves the problem that LSA attention has too much computational complexity when processing large-size convolution kernels, and achieves higher performance at a lower computational cost.
[0076] LSKA (Large Separable kernel Attention) is an innovative large separable kernel attention module that captures long-range dependencies and adaptability by decomposing large kernel convolution operations, and decomposes the decomposed 2D convolution kernels into serialized 1D kernels, reducing computational complexity and memory requirements. The process structure is as follows: Figure 6 As shown in FIG. 1 , the decomposition process of a large kernel convolution of the present invention is shown. Figure 7 As shown, the process of decomposing the 2D convolution into 1D of the present invention is shown. Figure 8 FIG. 1 is a diagram illustrating the LSKA attention mechanism of the present invention. Fig. 9 is a structural diagram showing the SPPF-LSKA module of the present invention. Fig.10 It shows the relevant parameter definitions of the WIoU loss function of the present invention. First, a K×K convolution is decomposed into (2d-1)×(2d-1) depth convolution, K / d×K / d depth expansion convolution and 1×1 convolution, then the 2D depth convolution kernel and the depth expansion convolution kernel are further decomposed into 1D horizontal (horizontal) convolution kernel and vertical (longitudinal) convolution kernel, and finally the decomposed convolution kernels are connected in series.
[0077] Improvements in loss functions.
[0078] In the field of object detection, the YOLOv8 model uses CIoU (Complet Intersection over Union) as its default loss calculation method. The CIoU loss function not only considers the overlapping area between the predicted bounding box and the true bounding box during the calculation process, but also introduces the distance (DIoU) and aspect ratio between the two, so that the loss function can pay more attention to the shape characteristics of the bounding box. However, when collecting and annotating data on edge devices, due to environmental and conditional limitations, the data obtained often contains some low-quality samples. The presence of these low-quality samples, if evaluated using traditional geometric metrics (such as distance and aspect ratio), will over-amplify the impact of these samples, resulting in a decrease in the model's generalization performance.
[0079] To solve this problem, the improved network model uses a new bounding box regression loss function - WloU (Wise Intersection over Union). The core idea of the WloU loss function is to dynamically reduce the penalty for geometric measurements when the overlap between the anchor box and the target box is high. This method enables the model to maintain good generalization capabilities when facing low-quality data collected on edge devices. The expression of the WloU loss function is as follows:
[0080]
[0081] Where W and H represent the width and height of the bounding box composed of the predicted box and the true box, respectively. * means separating the W and H of the minimum bounding box from the gradient calculation to reduce the adverse effects on model training.
[0082] The experimental operating system used in this paper is ubuntu20.04, and PyTorch is used as the framework for developing deep learning models. The detailed specifications of the experimental environment are outlined in Table 2. The input image is normalized to a size of 640×640, the batch size is set to 16, and the training is performed for 300 cycles. The learning capacity used during model training is 0.01, the SGD momentum is 0.937, and the optimizer weight decay is 0.0005. All other training parameters are set to the default values of the YOLOv8 network, as shown in Table 2.
[0083] Table 2 Training environment and hardware platform parameters
[0084] type Experimental environment CPU 16vCPU Intel(R)Xeon(R)Gold 6430 GPU RTX A5000 24G operating system ubuntu20.04 Deep Learning Frameworks Pytorch 1.11.0 programming language Python 3.8
[0085] Evaluation index of the present invention.
[0086] In order to objectively evaluate the performance of the giant salamander guarding behavior detection model, the evaluation indicators used include GFLOPS (billion floating-point operations per second), which quantifies the network model execution time, measured in billion floating-point operations per second. Parameters, used to evaluate the size and complexity of the model. FPS (frames per second), used to measure the detection speed of the model, measured in the number of frames transmitted per second. mAP (mean average precision), used to evaluate the accuracy of the model.
[0087] Among them, TP represents the number of positive samples predicted as positive examples, FP represents the number of negative samples predicted as positive examples, FN represents the number of positive samples predicted as negative examples; N is the number of predicted categories.
[0088] Precision is the ratio of the number of positive samples predicted by the model as positive examples to the number of all sample boxes predicted by the model as positive classes, defined as:
[0089]
[0090] Recall is the ratio of the number of positive samples predicted as positive examples to the number of actual positive examples, defined as:
[0091]
[0092] AP (average precision) is the area under the precision-recall curve, which represents the average accuracy of the model at different recall rates. It is defined as:
[0093]
[0094] mAP (mean average precision) is a comprehensive metric used to evaluate the performance of object detection models on multiple categories. It calculates the average precision (AP) for each category and then averages these AP values to measure the performance of the model. It is defined as:
[0095]
[0096] Where N represents the number of categories in the dataset, and in this paper N is equal to 6. The higher the mAP value, the better the performance of the model.
[0097] Based on the giant salamander dataset, an ablation experiment was conducted to explore the effect of improving the overall model performance. Starting from YOLOv8 as the base model, we successively replaced C2f with C2f-MSConv and introduced the LSKA attention mechanism into the SPPF layer.
[0098] The WIoU loss function was used for modification. The experimental results of the model on the giant salamander dataset are shown in Table 3.
[0099] Table 3 Ablation experiments of the present invention.
[0100]
[0101]
[0102] Referring to Table 2 and Table 3, we can see that:
[0103] 1. Through the comparative analysis between the first and second groups of experiments, it can be found that the proposed MSConv module shows significant advantages on this dataset. It not only effectively reduces the number of parameters and computational load (measured in GFLOPs) of the model, but also successfully improves the mAP of the model.
[0104] 2. Further, in the comparison between the second and third groups of experiments, the LSKA mechanism was introduced into the SPPF layer. Although this led to a slight increase in the number of model parameters and computational load, it significantly enhanced the model's ability to extract characteristic behaviors in complex backgrounds, with a 0.5% increase in mAP.
[0105] 3. Finally, by comparing the third and fourth groups of experiments, the CIoU loss function of the original model is replaced by the WIoU loss function. This change strengthens the generalization ability of the model and makes its performance more stable and reliable in different scenarios.
[0106] In order to more intuitively observe the detection effect of the improved model, the present invention uses the YOLOv8 model and the automatic recognition system of giant salamander protection behavior based on the ML-YOLOv8 network model structure to detect giant salamanders. The effect comparison diagrams are shown as follows: Fig.11 The detection results of YOLOv8 are shown as follows. Fig.11 In (a), (c), and (e), the detection results of the automatic recognition system for giant salamander protecting young based on the ML-YOLOv8 network model structure of the present invention are shown as follows Fig.11 (b), (d) and (f) in the figure.
[0107] Fig.11 Figures (a), (c), and (e) are the detection results of YOLOv8s. Fig.11 (b), (d), and (f) are the detection results of the same image by an automatic recognition system for giant salamander protecting young based on the ML-YOLOv8 network model structure.
[0108] Here, the present invention also includes a device for a system for automatically identifying the behavior of giant salamander protecting young based on the ML-YOLOv8 network model, including:
[0109] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program, and the computer program code implements the above-mentioned automatic recognition system for giant salamander protecting young based on the ML-YOLOv8 network model; and also includes a matching image taking device and a display device.
[0110] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device or electronic device, and various general devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.
Claims
1. An automatic recognition system for giant salamander protecting young based on ML-YOLOv8 network model structure, characterized in that: include: The ML-YOLOv8 model consists of four parts: input end, backbone network, neck network and detection head; The input end performs data enhancement and adaptive image scaling. The data enhancement adopts Mosaic enhancement. The Mosaic enhancement part of YOLOv8 introduces the operation of using Mosaic enhancement in the last 10 epochs in YOLOX, which enhances the robustness of the model. The adaptive image scaling uniformly scales the original image to the same size. The backbone network is composed of Conv, C2f, C2f-MSConv and SPPF-LSKA modules. The Conv module includes Conv2d, BN, and SLU activation functions. The Conv module performs convolution operations on feature maps. The C2f module is a feature extraction module. The sixth and eighth layers of the backbone network in the C2f module and the neck network are set to C2f-MSConv for multi-scale fusion. The SPPF-LSKA module is used to extract features of different receptive fields. The neck network is composed of the FPN-PAN structure. FPN is top-down. Through upsampling, the high-level feature information is fused with the bottom-level feature information to calculate the predicted feature map. PAN is an improvement on FPN. It introduces lateral connections to enhance the semantic information of the features, so that the bottom-up feature map can be fused with the top-down feature map. The detection head uses an anchor-free matching mechanism, which does not require preset anchors. It only needs to regress the target center point and width and height of feature maps of different scales. By utilizing the rich information from feature maps of different scales, it can accurately obtain the classification and location information of large, medium and small-sized target objects.
2. The automatic identification system for giant salamander protecting young based on the ML-YOLOv8 network model structure according to claim 1 is characterized in that: The multi-scale convolution of C2f-MSConv is used for detection tasks. The object information captured under different receptive fields is different. The small receptive field captures the details of the object, which is conducive to detecting small targets. The large receptive field captures the overall structure of the object, which helps the network locate the position of the object. The MSConv multi-scale convolution is designed based on the idea of Res2Net. The core structure of Res2Net adopts formula (1): Feature K2 is processed by a 3×3 convolution layer, which is located at the X3 position in the module architecture. After this initial convolution, K2 is optimized for the second time by another 3×3 convolution. The effect of 5×5 convolution is simulated by the effective combination of two consecutive 3×3 convolutions. Subsequently, K3 is enhanced by integrating the processed features from 3×3 and composite 5×5 receptive fields. Similarly, feature K4 is processed by a 7×7 receptive field, further expanding the scale of feature detection.
3. The automatic identification system for giant salamander protecting young based on the ML-YOLOv8 network model structure according to claim 1 is characterized in that: The SPPF-LSKA module captures long-range dependency and adaptability by decomposing large-kernel convolution operations, and decomposes the decomposed 2D convolution kernels into serially connected 1D kernels, thereby reducing computational complexity and memory requirements. The SPPF-LSKA module first decomposes a K×K convolution into (2d-1)×(2d-1) depth convolution, K / d×K / d depth expansion convolution, and 1×1 convolution, and then further decomposes the 2D depth convolution kernel and the depth expansion convolution kernel into 1D horizontal convolution kernel and vertical convolution kernel, and finally serializes the decomposed convolution kernels in sequence.
4. The automatic identification system for giant salamander protecting young based on the ML-YOLOv8 network model structure according to claim 1 is characterized in that: The ML-YOLOv8 model uses a new bounding box regression loss function WIoU. The WloU loss function dynamically reduces the penalty for geometric metrics when the overlap between the anchor box and the target box is high. The expression of the WloU loss function is as shown in formula (2), formula (3), and formula (4): LwIoU=RwIouLI0U (4) Where W and H represent the width and height of the bounding box composed of the predicted box and the real box, respectively, and * means separating the W and H of the minimum bounding box from the gradient calculation.
5. The automatic identification system for giant salamander protecting young based on the ML-YOLOv8 network model structure according to claim 1 is characterized in that: The experimental operating system is Ubuntu 20.04, PyTorch is used as the framework for developing deep learning models, the input image is normalized to a size of 640 × 640, the batch size is set to 16, the training is performed for 300 cycles, the learning capacity used during model training is 0.01, the SGD momentum is 0.937, the optimizer weight decay is 0.0005, and all other training parameters are set to the default values of the YOLOv8 network.
6. The automatic identification system for giant salamander protecting young based on the ML-YOLOv8 network model structure according to claim 1 is characterized in that: Objectively evaluate the performance of the giant salamander guarding behavior detection model. The evaluation indicators used include GFLOPS, billion floating-point operations per second, which quantifies the network model execution time and is measured in billion floating-point operations per second. The parameter is used to evaluate the size and complexity of the model. FPS, frames per second, is used to measure the detection speed of the model and is measured in the number of frames transmitted per second. mAP, mean average precision, is used to evaluate the accuracy of the model. Precision is the ratio of the number of positive samples predicted by the model as positive examples to the total number of positive samples predicted by the model, and is defined as formula (5): Among them, TP represents the number of positive samples predicted as positive examples, FP represents the number of negative samples predicted as positive examples, FN represents the number of positive samples predicted as negative examples; N is the number of predicted categories.
7. The automatic identification system for giant salamander protecting young based on the ML-YOLOv8 network model structure according to claim 6 is characterized in that: Recall is the ratio of the number of positive samples predicted as positive examples to the number of actual positive examples, which is defined as formula (6): The average precision (AP) is the area under the precision-recall curve, which represents the average accuracy of the model at different recall rates and is defined as formula (7): In formula (7), Precision represents accuracy, Recall represents recall, and d(Recall) represents the integral of Recall.
8. The automatic identification system for giant salamander protecting young based on the ML-YOLOv8 network model structure according to claim 7 is characterized in that: The mean average precision (mAP) is a comprehensive indicator used to evaluate the performance of object detection models on multiple categories. The average precision (AP) of each category is calculated, and then the average of all AP values is taken to measure the performance of the model, which is defined as formula (8): The i in formula (8) indicates the type of behavior. Formula (8) means that the AP value of the first type of behavior is accumulated to the AP value of the Nth type, and then divided by N to obtain the average accuracy of N types of behaviors. Where N represents the number of categories in the dataset, N is equal to 6, and the higher the mAP value, the better the performance of the model.
9. The device of the automatic identification system of giant salamander protecting young based on the ML-YOLOv8 network model structure according to any one of claims 1 to 8, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program, and the computer program code implements the ML-YOLOv8 model structure for automatic identification of giant salamander protecting young as described in any one of claims 1 to 9.
10. The device according to claim 9, characterized in that: It also includes matching image taking equipment and display equipment.
