A method for detecting a laver cultivation scene by fusing YOLO and a transformer

Abstract

The application provides a kind of fusion YOLO and the detection method of laver culture scene of transformer, belong to image recognition and intelligent agricultural technology field, solve the technical problem of low efficiency, insufficient precision of traditional laver culture monitoring.The technical scheme is: S1, collect and pretreat laver culture image, construct the annotation dataset containing laver, inserted rod and raft net target;S2, construct the detection model of fusion YOLO module and transformer module;S3, based on training dataset, the model parameters are iteratively trained using optimization algorithm until performance is up to standard;S4, the preprocessed image to be tested is input into the trained model for detection, and then the non-maximum suppression algorithm is used to filter out overlapping boxes to obtain the final result.The application can accurately monitor laver, inserted rod, raft net and other key targets, improve the automation level and accuracy of monitoring.

Description

Technical Field

[0001] This invention relates to the fields of image recognition and smart agriculture, specifically to a method for detecting seaweed farming scenes that integrates YOLO and Transformer. Background Technology

[0002] The laver farming industry currently faces three major challenges that severely restrict its intelligent upgrading and economic benefits. First, there is the issue of strong environmental dependence. Traditional manual inspections are limited by weather and tides, making operation impossible in foggy weather or at night, resulting in untimely information acquisition. Second, there is the problem of delayed condition assessment. Existing sensors can only monitor physical parameters such as water temperature and salinity, and cannot directly identify laver shedding or disease, leading to an average delay of 5-7 days in disease detection, missing the optimal intervention window. Furthermore, insufficient detection accuracy is also a major bottleneck. Conventional image processing methods have a false detection rate of over 40% under wave interference and cannot effectively distinguish between pole tilting and normal displacement, resulting in a persistently high misjudgment rate.

[0003] Currently, there are still significant shortcomings in the field of laver aquaculture monitoring. On the one hand, traditional image processing methods use edge detection and morphological manipulation to identify laver, but they cannot effectively distinguish laver from wave interference, and the false detection rate increases sharply in rainy weather, resulting in poor robustness. On the other hand, deep learning-based methods, such as "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows," can improve detection accuracy, but existing algorithms are usually computationally intensive, making it difficult to run in real time on edge devices. Furthermore, their ability to detect small targets still needs improvement, and there is a lack of effective modeling of the spatial relationship between laver and aquaculture facilities such as rafts and nets. This leads to a high false negative rate for small targets in complex environments, and the overall detection accuracy is insufficient to meet the needs of practical applications.

[0004] Existing target detection algorithms have significant shortcomings when dealing with seaweed farming scenarios, such as poor environmental adaptability and susceptibility to weather interference from waves, changes in light, etc.; and low accuracy in recognizing multiple targets, especially small targets such as seaweed and poles. Therefore, there is an urgent need for an intelligent detection method that combines environmental robustness with high-precision multi-target and small-target detection capabilities to improve the efficiency of monitoring and early warning of seaweed farming status. Summary of the Invention

[0005] This invention aims to address the technical problems of low efficiency and high subjectivity in traditional laver farming monitoring, as well as the insufficient accuracy and real-time performance of existing image recognition methods in complex environments for multi-target identification. It provides a laver farming scene detection method that integrates YOLO and Transformer. Applied to smart laver farming scenarios, this method can accurately and quickly identify key elements such as laver, poles, and rafts, effectively cope with interference from complex environments, significantly improve the automation, accuracy, and real-time performance of monitoring, reduce labor costs, and provide technical support for the precise and intelligent management of laver farming.

[0006] The inventive concept of this invention is as follows: First, images of seaweed farming areas are collected and preprocessed; then, the images are labeled to construct a high-quality training dataset containing seaweed, poles, and raft nets as targets; next, a YOLO-Transformer target detection model integrating Swin Transformer, BiFPN, and attention mechanisms is constructed, and a composite loss function is used for optimization training; finally, model inference and NMS post-processing are performed on the images to be tested to output accurate detection results.

[0007] This invention is achieved through the following measures: a method for detecting seaweed farming scenarios that integrates YOLO and Transformer, comprising the following steps:

[0008] S1. Collect digital images of the seaweed farming environment. The images contain at least one key target among seaweed, raft nets, and poles. Perform preprocessing on the images, including denoising, image enhancement, and size normalization. Use an image annotation tool to annotate the preprocessed images to generate a training dataset containing target bounding boxes and category labels.

[0009] S2. Construct a YOLO-Transformer fusion detection model, in which the YOLO-Transformer fusion detection model is used to achieve fast target detection, and the self-attention mechanism of the YOLO-Transformer fusion detection model is used to enhance the model's ability to extract image features and understand context.

[0010] S3. Based on the training dataset described in step S1, a composite loss function including localization loss, classification loss and confidence loss is adopted, and an optimization algorithm is selected to iteratively train the network parameters of the YOLO-Transformer fusion detection model described in step S2 until the performance indicators of the YOLO-Transformer fusion detection model on the seaweed farming target recognition task meet the preset requirements, and the trained detection model is obtained.

[0011] S4. Perform the same preprocessing as step S1 on the image of the seaweed farming area to be detected. Input the preprocessed image into the detection model trained in step S3 for forward inference to obtain the initial detection result containing the target detection box, category and confidence. Apply the non-maximum suppression algorithm to the initial detection result to merge and filter overlapping detection boxes for the same seaweed, raft net or pole target to obtain the final detection result.

[0012] Furthermore, the specific steps of step S2 are as follows:

[0013] Step S2.1: Construct a Swing Transformer network as the feature extraction backbone and extract hierarchical features; the Swing Transformer network replaces the CSPDarknet in the original YOLO model; the Swing Transformer network is composed of multiple stacked Swing Transformer modules, which output feature maps of different resolutions at different stages through its hierarchical structure. The Swing Transformer modules internally utilize a multi-head self-attention mechanism based on shifted windows for feature learning, where the calculation of the window attention mechanism follows the formula:

[0014]

[0015] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k B is the dimension of the key vector, and B is a learnable relative position encoding bias used to explicitly model the spatial relative relationships of the aquaculture targets such as seaweed and stalks in the image. Through this backbone network, the Swing Transformer module is configured to output feature maps of at least three stages, which have different receptive fields and semantic levels.

[0016] Step S2.2: Construct a multi-scale feature fusion network. Between the multi-scale feature maps output by the Swin Transformer network, construct a bidirectional feature pyramid network structure. This allows for bidirectional information flow from top to bottom and bottom to top between feature maps from different levels containing information on seaweed morphology, raft structure, and pole size. This information is then fused to each input feature map F. i Assign a learnable weight w i The weighted fusion is performed, and the fusion calculation follows the formula:

[0017]

[0018] Among them, F out For the fused output feature map, F i For the i-th input feature map, w iw represents the learnable weights corresponding to the i-th input feature map. j Let be the learnable weight corresponding to the j-th input feature map, and ∈ be a preset numerical stability constant. This weighted fusion idea is specifically implemented in the node calculation of the bidirectional path. For the i-th layer of the pyramid, its intermediate feature P on the top-down path is... i td and the output feature P of the bottom-up path i out It can be calculated using the following formula:

[0019]

[0020] Among them, P i in is the input feature of the i-th layer of the backbone network. The Resize operation upsamples or downsamples according to the path direction. w1, w2 and w′1, w′2, w′3 are learnable weights. Conv is the convolution operation.

[0021] Step S2.3: Insert a Cross-stage Partial Attention (C2PSA) module into one or more nodes in the feature fusion path of the bidirectional feature pyramid network. C2PSA first processes the feature map of the input node... It is divided into two parts along the channel dimension. and Where C = C1 + C2, then a coordinate attention mechanism is applied to the first feature part X1 to generate a position-sensitive attention map for the slender target pole and the raft mesh structure; the calculation of coordinate attention begins with coordinate information embedding, which aggregates spatial information by performing one-dimensional global pooling on feature X1 along the horizontal and vertical directions respectively, to obtain a direction-aware feature vector:

[0022]

[0023] in, and These are the outputs of the c-th channel at height h and width w, respectively, where H is the feature map height, W is the feature map width, and x is the width. c (h,i) is the feature value of the c-th channel at position (h,i), x c (j,w) represents the feature value of the c-th channel at position (j,w); next, the coordinate attention generation step is performed, where the feature vectors of the two directions are concatenated, transformed by a shared convolutional layer F1 and a nonlinear activation function δ, and then the result is decomposed into two independent tensors f. h and f w And through independent convolutional layers F h F wAnd the attention weights g generated by the Sigmoid function σ. h and g w :

[0024]

[0025] Finally, the generated attention weights are applied to the features. Then, it is concatenated and fused with the unprocessed feature X2 to obtain the final output Y of the C2PSA module:

[0026]

[0027] Wherein, Concat is the tensor concatenation operation;

[0028] Step S2.4: Based on the enhanced feature map output in step S2.3, configure detection heads corresponding to different scale features such as P2, P3, P4, and P5 respectively; the detection head at the P2 level is used to process the high-resolution feature map to detect small-sized targets such as poles in the image, while the detection heads at the P3 to P5 levels perform bounding box regression and category prediction for medium and large-sized targets such as seaweed and raft nets on the feature maps of the corresponding scales.

[0029] Furthermore, the specific steps of step S3 are as follows:

[0030] Step S3.1: Before training begins, first load the seaweed farming target training dataset generated in step S1. This dataset contains images with bounding boxes and category labels for seaweed, rafts, and poles. At the same time, load the YOLO-Transformer fusion detection model network architecture constructed in step S2 to initialize the model network parameters. The initialization method is to load the weights obtained by pre-training on a large-scale public dataset.

[0031] Step S3.2: Define a composite loss function for target detection of seaweed, raft nets, and poles. This function is composed of a weighted average of localization loss, classification loss, and confidence loss. The calculation of the composite loss function follows the formula:

[0032]

[0033] in, To pinpoint the loss, For classifying losses, To determine whether there is a confidence loss for any target within the predicted bounding box, λ box , λ cls , λ obj The preset weighting coefficients; the positioning loss Focal-CIoU loss is used, denoted as For each predicted bounding box of laver, raft net, or pole, the loss value is calculated using the formula:

[0034]

[0035] Wherein, IoU is the intersection-union ratio between the predicted bounding box of laver, raft net, or pole and its actual labeled bounding box. p is the preset CIoU loss value. t It is a numerical value related to the IoU value or classification confidence, and γ is a preset focusing parameter; classification loss. With confidence loss Both use a binary cross-entropy loss function. Its calculation follows the formula:

[0036]

[0037] When calculating the classification loss, y i For the true category label of laver, raft net, or pole, To calculate the confidence loss, y represents the model's predicted probability for that category. i To determine whether the prediction box contains the true labels of seaweed, raft nets, or pole targets. The prediction confidence level corresponding to the model; finally, based on this composite loss function The calculation results are used to calculate the gradient through the backpropagation algorithm, and the network parameters of the YOLO-Transformer fusion detection model are iteratively updated using stochastic gradient descent or its variant optimization algorithm.

[0038] Step S3.3: During iterative training, the performance of the current model is periodically evaluated using a reserved validation dataset, which also contains labeled images of seaweed, raft nets, and poles. The convergence status of the model is monitored by calculating the mean precision, accuracy, and recall of the model on the validation set for the three target classes of seaweed, raft nets, and poles. When the performance indicators reach a preset threshold or no longer improve within several consecutive training cycles, the iterative training is terminated, and the network parameters of the model with the best current or validation performance are saved as the final trained detection model.

[0039] Furthermore, the specific steps of step S4 are as follows:

[0040] Step S4.1: After performing the same preprocessing operation as in step S1, the image of the seaweed farming area to be detected is input into the YOLO-Transformer fusion detection model trained in step S3 for forward inference to obtain the initial detection result containing multiple initial detection boxes, where each initial detection box is associated with a target category label as seaweed, raft net or pole and the corresponding confidence score.

[0041] Step S4.2: Apply a category-aware nonmaximum suppression method, namely the NMS algorithm, to the initial detection results. This method first sorts all initial detection boxes in descending order according to their confidence scores, and then processes them iteratively. In each iteration, the detection box with the highest confidence score is selected as the reference box, and its intersection-union ratio (IU) with all other detection boxes to be processed is calculated. Differential suppression is performed based on the IU and the category relationship. When a detection box to be processed has the same category as the reference box, such as both being laver or both being stalks, and the IU is greater than a first preset threshold, the detection box to be processed is suppressed. When the detection box to be processed has a different category from the reference box, such as corresponding to raft net and laver, or stalk and raft net respectively, no suppression is performed. This iterative process is repeated until all detection boxes are processed to form the final detection result.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] 1. This invention employs a YOLO-Transformer model architecture, combining YOLO and Transformer. First, the YOLO module efficiently extracts multi-scale features and rapidly generates candidate targets. Then, the Transformer module's self-attention mechanism performs deep processing on the features to enhance global contextual understanding, enabling accurate identification of aquaculture targets of varying shapes and sizes, such as seaweed, stalks, and raft nets. This model architecture improves the accuracy and robustness of multi-target detection in complex aquatic environments, laying a solid visual data foundation for subsequent accurate assessment of aquaculture status.

[0044] 2. This invention enhances the model's feature representation based on an optimized feature fusion network and attention mechanism. By weighted fusion and bidirectional information transfer of feature maps at different levels, the model is guided to adaptively focus on key regions and important feature channels in the image, thereby optimizing the feature representation of targets such as details of seaweed growth, minute displacements of poles, and minor damage to the raft net. This method effectively enhances the model's ability to identify small targets, dense targets, and detailed features, improving the precision of detection and its anti-interference performance.

[0045] 3. This invention employs a composite loss function, including advanced localization losses such as Focal-CIoU, to drive model training. While comprehensively measuring the geometric similarity between predicted and ground truth bounding boxes, it focuses on learning and optimizing difficult-to-identify samples, achieving dynamic adjustment of model parameters and efficient convergence. This strategy improves the localization accuracy of target bounding boxes and the stability of model training, significantly enhancing the overall accuracy and reliability of seaweed farming status detection. It provides crucial technical support for achieving precise and intelligent farming management and has significant prospects for widespread application. Attached Figure Description

[0046] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0047] Figure 1 This is a flowchart of a seaweed farming scene detection method integrating YOLO and Transformer according to the present invention.

[0048] Figure 2 The diagram shown is a model architecture block diagram of a seaweed farming scene detection method that integrates YOLO and Transformer according to the present invention.

[0049] Figure 3 The diagram shown is a block diagram of a seaweed farming scene detection method that integrates YOLO and Transformer according to the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0051] Example 1

[0052] See Figures 1 to 3 This embodiment provides a method for detecting seaweed farming scenarios by integrating YOLO and Transformer, including the following steps:

[0053] S1. Collect digital images of the seaweed farming environment. The images contain at least one key target among seaweed, raft nets, and poles. Perform preprocessing on the images, including denoising, image enhancement, and size normalization. Use an image annotation tool to annotate the preprocessed images to generate a training dataset containing target bounding boxes and category labels.

[0054] S2. Construct a YOLO-Transformer fusion detection model, in which the YOLO module is used to achieve fast target detection, and the self-attention mechanism of the Transformer module is used to enhance the model's ability to extract image features and understand context.

[0055] S3. Based on the training dataset described in step S1, a composite loss function including localization loss, classification loss and confidence loss is adopted, and an optimization algorithm is selected to iteratively train the network parameters of the fusion detection model described in step S2 until the performance indicators of the model on the seaweed farming target recognition task meet the preset requirements, and the trained detection model is obtained.

[0056] S4. Perform the same preprocessing as step S1 on the image of the seaweed farming area to be detected. Input the preprocessed image into the detection model trained in step S3 for forward inference to obtain the initial detection result containing the target detection box, category and confidence. Apply the non-maximum suppression algorithm to the initial detection result to merge and filter overlapping detection boxes for the same seaweed, raft net or pole target to obtain the final detection result.

[0057] Specifically, the steps of step S2 are as follows:

[0058] Step S2.1: Construct a Swing Transformer network as the feature extraction backbone and extract hierarchical features; the Swing Transformer network replaces the CSPDarknet in the original YOLO model; the Swing Transformer network is composed of multiple stacked Swing Transformer modules, which output feature maps of different resolutions at different stages through its hierarchical structure. The Swing Transformer modules internally utilize a multi-head self-attention mechanism based on shifted windows for feature learning, where the calculation of the window attention mechanism follows the formula:

[0059]

[0060] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k B is the dimension of the key vector, and B is a learnable relative position encoding bias used to explicitly model the spatial relative relationships of the aquaculture targets such as seaweed and stalks in the image. Through this backbone network, the Swing Transformer module is configured to output feature maps of at least three stages, which have different receptive fields and semantic levels.

[0061] In this embodiment, we use a smaller version of the Swing Transformer. Taking the calculation of one attention head as an example, an input 7×7 image window undergoes a linear transformation to obtain a query matrix Q, a key matrix K, and a value matrix V, all of which have a dimension of 49×32. Here, the key vector dimension d... k =32, then B is a learnable relative position encoding matrix with a dimension of 49×49; finally, the feature maps of the three stages C3, C4 and C5 are output, with dimensions of 80×80×384, 40×40×768 and 20×20×1536 respectively.

[0062] Step S2.2: Construct a multi-scale feature fusion network. Between the multi-scale feature maps output by the Swin Transformer network, construct a bidirectional feature pyramid network structure. This allows for bidirectional information flow from top to bottom and bottom to top between feature maps from different levels containing information on seaweed morphology, raft structure, and pole size. This information is then fused to each input feature map F. i Assign a learnable weight w i The weighted fusion is performed, and the fusion calculation follows the formula:

[0063]

[0064] Among them, F out For the fused output feature map, F i For the i-th input feature map, w i w represents the learnable weights corresponding to the i-th input feature map. j Let be the learnable weight corresponding to the j-th input feature map, and ∈ be a preset numerical stability constant. This weighted fusion idea is specifically implemented in the node calculation of the bidirectional path. For the i-th layer of the pyramid, its intermediate feature P on the top-down path is... i td and the output feature P of the bottom-up path i out It can be calculated using the following formula:

[0065]

[0066] Among them, P i in These are the input features of the i-th layer of the backbone network. The Resize operation upsamples or downsamples the data according to the path direction. w1, w2 and w′1, w′2, w′3 are learnable weights. Conv is the convolution operation.

[0067] In this embodiment, when calculating the intermediate features of the top-down path in the 4th layer... At that time, the feature size was 40×40, which was derived from the same layer feature from the backbone network, with a size of 40×40×768. Features from deeper layers, whose size changes from 20×20 to 40×40 after a 2x upsampling. Let the learnable weights w1 and w2 be initialized to 1, ∈ = 0.0001. The fusion process involves summing the two input features element-wise with weights, dividing by the sum of the weights, and finally refining the features through a 3×3 convolutional layer to obtain intermediate features. Next, use the intermediate features that were just calculated. Calculate the final output features of the bottom-up path. The computation here incorporates features from three sources: firstly, the original backbone network features. Second, the newly generated top-down path features Third, bottom-up path features from shallower layers, after being downsampled by 2 times. Similarly, learnable weights w′1, w′2, and w′3 are assigned to these three input features. Through weighted summation, normalization, and final convolution, a final output feature that integrates information from three directions and has stronger semantic representation capabilities is generated. Similarly, by fusing upsampled high-level features with low-level features of the backbone network in a top-down path, and performing the same bidirectional weighted fusion operation on all levels in both paths, a fused feature map with a uniform number of 256 channels can be obtained, whose members are as follows: and

[0068] Step S2.3: Insert a Cross-stage Partial Attention (C2PSA) module into one or more nodes in the feature fusion path of the bidirectional feature pyramid network. C2PSA first processes the feature map of the input node... It is divided into two parts along the channel dimension. and Where C = C1 + C2, then, a coordinate attention (CoordAtt) mechanism is applied to the first feature part X1 to generate a position-sensitive attention map for the slender target pole and the raft mesh structure; the calculation of CoordAtt begins with the embedding of coordinate information, which aggregates spatial information by performing one-dimensional global pooling on feature X1 along the horizontal and vertical directions respectively, to obtain a direction-aware feature vector:

[0069]

[0070] in, and These are the outputs of the c-th channel at height h and width w, respectively, where H is the feature map height, W is the feature map width, and x is the width. c (h,i) is the feature value of the c-th channel at position (h,i), x c (j,w) represents the feature value of the c-th channel at position (j,w); next, the coordinate attention generation step is performed, where the feature vectors of the two directions are concatenated, transformed by a shared convolutional layer F1 and a nonlinear activation function δ, and then the result is decomposed into two independent tensors f. h and f w And through independent convolutional layers F h F w And the attention weights g generated by the Sigmoid function σ. h and g w :

[0071]

[0072] Finally, the generated attention weights are applied to the features. Then, it is concatenated and fused with the unprocessed feature X2 to obtain the final output Y of the C2PSA module:

[0073]

[0074] Wherein, Concat is the tensor concatenation operation;

[0075] In this embodiment, after the output node of BiFPN, a C2PSA attention module is inserted into the feature map of each level to further enhance the expressive power of the features. Here, we take... Taking the input feature map as an example, it is first segmented into and Applying CoordAtt to X1 and performing pooling using formula (5) yields... and Then, the attention weight is calculated according to formula (6). and Finally, according to formula (7), the weights are applied to X1 and concatenated with X2 to obtain the final output enhanced feature map. Similarly, this C2PSA module is applied to other layer-by-layer fusion feature maps generated in step S2.2, i.e. as well as This results in an enhanced feature map set that remains unchanged in size and is ultimately used for detection.

[0076] Step S2.4: Based on the enhanced feature map output in step S2.3, configure detection heads corresponding to features at different scales such as P2, P3, P4, and P5, where the P2 detection head is used to process the high-resolution feature map. P5 detector head processes low-resolution feature maps To identify large targets that occupy the main area of ​​the image, such as large raft nets, the P3 and P4 detection heads are also configured on their respective final enhanced feature maps to detect medium-sized targets. All detection heads perform bounding box regression and class prediction in parallel on their respective feature maps, thus forming a complete detection model that can accurately identify targets of different sizes simultaneously.

[0077] Specifically, the steps of step S3 are as follows:

[0078] Step S3.1: Before training begins, first load the laver farming target training dataset generated in step S1. This dataset contains images with bounding boxes and category labels for laver, raft nets, and poles. At the same time, load the YOLO-Transformer fusion detection model network architecture constructed in step S2 to initialize the model network parameters. The initialization method includes loading weights obtained from pre-training on a large-scale public dataset.

[0079] Step S3.2: Define a composite loss function for target detection of seaweed, raft nets, and poles. This function is composed of a weighted average of localization loss, classification loss, and confidence loss. The calculation of the composite loss function follows the formula:

[0080]

[0081] in, To pinpoint the loss, For classifying losses, To determine whether there is a confidence loss for any target within the predicted bounding box, λ box , λ cls , λ obj The preset weighting coefficients; the positioning loss Focal-CIoU loss is used, denoted as For each predicted bounding box of laver, raft net, or pole, the loss value is calculated using the formula:

[0082]

[0083] Wherein, IoU is the intersection-union ratio between the predicted bounding box of laver, raft net, or pole and its actual labeled bounding box. p is the preset CIoU loss value. t It is a numerical value related to the IoU value or classification confidence, and γ is a preset focusing parameter; classification loss. With confidence loss Both use a binary cross-entropy loss function. Its calculation follows the formula:

[0084]

[0085] When calculating the classification loss, y i For the true category label of laver, raft net, or pole, To calculate the confidence loss, y represents the model's predicted probability for that category. i To determine whether the prediction box contains the true labels of seaweed, raft nets, or pole targets. The prediction confidence level corresponding to the model; finally, based on this composite loss function The calculation results are used to calculate the gradient through the backpropagation algorithm, and the network parameters of the YOLO-Transformer fusion detection model are iteratively updated using stochastic gradient descent or its variant optimization algorithm.

[0086] In this embodiment, a specific prediction sample is used as an example to illustrate the calculation process of the total loss. Assume the model predicts a region where the true target is "seaweed," and its true bounding box B... gt = [0.2, 0.3, 0.4, 0.5], predict bounding box B p = [0.22, 0.31, 0.45, 0.52], and the calculated Intersection over Union (IoU) is 0.93. The true class y of this target... i =1 ("seaweed"), the model predicts the probability that it is "seaweed". The target actually exists within the predicted bounding box, therefore the true label is y. i =1, the confidence level of the model in predicting the existence of the target. We set the loss weight coefficient λ box =0.05,λ cls =0.5, λ obj =1, focusing parameter γ = 0.5. First, calculate the positioning loss λ according to formula (9). box Its value is 0.93. 0.5 ×(1-0.93)≈0.0675. Next, the classification loss λ is calculated according to formula (10). cls Its value is -[1·log(0.9)+0·log(0.1)]≈0.1054. Similarly, the confidence loss L is calculated according to formula (10). obj Its value is -[1·log(0.95)+0·log(0.05)]≈0.0513. Finally, according to formula (8), the three are weighted and summed to obtain the total loss L of the sample. total The loss value is (0.05×0.0675)+(0.5×0.1054)+(1.0×0.0513)≈0.1074. This loss value will be used for backpropagation in this iteration to update the model weights.

[0087] Step S3.3: During iterative training, the performance of the current model is periodically evaluated using a reserved validation dataset, which also contains labeled images of seaweed, raft nets, and poles. The convergence status of the model is monitored by calculating the mean precision, accuracy, and recall of the model on the validation set for the three target classes of seaweed, raft nets, and poles. When the performance indicators reach a preset threshold or no longer improve within several consecutive training cycles, the iterative training is terminated, and the network parameters of the model with the best current or validation performance are saved as the final trained detection model.

[0088] In this embodiment, we use the AdamW optimizer for training, setting the initial learning rate to 0.001, the weight decay to 0.05, the momentum to 0.937, and a total of 100 epochs. The batch size is 16. We monitor the average precision on the validation set. If the metric does not improve within 10 consecutive epochs, we trigger the early stop mechanism to terminate the training.

[0089] Specifically, the steps of step S4 are as follows:

[0090] Step S4.1: After preprocessing the image of the seaweed farming area to be detected in the same way as in Step S1, input it into the YOLO-Transformer fusion detection model trained in Step S3 for forward inference to obtain initial detection results containing multiple initial detection boxes. Each initial detection box is associated with a target category label as seaweed, raft net, or pole, and a corresponding confidence score. Specifically, an image to be detected is scaled to 640×640 and normalized before being input into the model. After forward inference, for the input image, the model may output approximately 2000 initial prediction boxes containing location coordinates, confidence scores, and scores for the three specific categories of seaweed, raft net, and pole. This batch of preliminary detection results, containing a large amount of redundant information, will be used as input for the next step.

[0091] Step S4.2: The generated preliminary detection results are processed using a category-aware nonmaximum suppression algorithm to filter out redundant detection boxes, forming the final detection results. This algorithm first sorts all initial detection boxes in descending order based on their confidence scores, and then iteratively filters them. In one iteration, the algorithm selects the detection box with the highest current confidence score from the sorted list as the baseline box, for example, "Seaweed A" with a confidence score of 0.95. Subsequently, the algorithm calculates the intersection-union ratio (IU) of this baseline box with all other unprocessed detection boxes and performs differential suppression based on category relationships. If an unprocessed detection box "Seaweed B" has a confidence score of 0.88 and belongs to the same category as the baseline box (both are seaweed), and their IU of 0.7 is greater than the preset first threshold of 0.5, then "Seaweed B" will be considered a redundant detection of "Seaweed A" and suppressed. However, the algorithm's category-aware characteristic is reflected in its handling of targets of different categories: when encountering another detection box "Stick C" with a confidence score of 0.92, even though it is spatially adjacent to "Seaweed A," resulting in an intersection-union ratio as high as 0.8, the system will determine that they are two independent targets because they belong to different categories, and therefore will not suppress "Stick C." By selecting reference boxes one by one and repeating the above suppression process until all detection boxes have been processed, a set of non-redundant and accurate detection results can be obtained, accurately preserving different targets such as "Seaweed A" and its adjacent "Stick C."

[0092] To further clarify and fully explain the specific configuration used in this embodiment, the key parameters used throughout the model building, training and inference process, their values ​​in various embodiments, and the preferred range determined by the author through a large number of experiments are summarized as shown in Table 1.

[0093] Table 1 Key design parameters, values ​​of embodiments, and preferred ranges

[0094]

[0095] The target detection method proposed in this embodiment accurately identifies and locates key targets in the seaweed farming environment, enabling automated and intelligent monitoring of the farming status. This embodiment constructs a seaweed farming scene dataset containing 2000 images, using 1600 images as the training set and 400 images as the test set. Table 2 shows the experimental results of target detection applied to the test set using the method of this embodiment.

[0096] Table 2. Experimental results of the method in this embodiment on the laver aquaculture test set.

[0097]

[0098] This embodiment achieved significant experimental results in the target detection task of seaweed farming using the method described in this embodiment. On a test set of 400 images, the model achieved a precision of 0.85, a recall of 0.82, an F1-score of 0.835, and the key comprehensive evaluation index mAP@0.5 reached 0.84.

[0099] These results demonstrate that the method described in this embodiment can accurately and efficiently identify key elements in the laver cultivation process, providing crucial technical support for automated growth monitoring, yield prediction, and disease control, and has extremely high application value.

[0100] Example 2

[0101] To further verify the robustness and generalization ability of the technical solution in this embodiment under complex and variable environments, this embodiment, under the same experimental environment, model structure, and evaluation metrics as Embodiment 1, uses an additional independent test set containing 500 images for rigorous testing. This test set specifically selects images taken under complex conditions such as strong backlighting, water surface reflection, fog, partial target occlusion, and low light at dawn or dusk. The model trained in Embodiment 1 is directly applied to this complex environment test set, and its performance is shown in Table 3.

[0102] Table 3 shows the model's performance on the complex environment test set.

[0103]

[0104] As shown in Table 3, when facing challenging scenarios such as poor lighting conditions, shading, and weather interference, the performance indicators of the method in this embodiment show a slight and reasonable decrease compared to the ideal conditions in Embodiment 1. However, its mAP@0.5 still reaches a relatively high level of 0.81. This result strongly demonstrates that the method in this embodiment is not only applicable to specific, idealized scenarios. The deep abstract features extracted through deep learning networks have a strong resistance to environmental interference factors such as changes in lighting and partial shading, and can effectively capture the essential characteristics of targets such as seaweed, raft nets, and poles.

[0105] Therefore, this embodiment demonstrates that the technical solution of this embodiment has excellent robustness and good generalization ability, and can be stably and reliably applied to diverse and complex real seaweed farming environments, thus possessing practical application value.

[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting seaweed farming scenarios integrating YOLO and Transformer, characterized in that, Includes the following steps: S1. Collect digital images of the seaweed farming environment. The images contain at least one key target among seaweed, raft nets, and poles. Perform preprocessing on the images, including denoising, image enhancement, and size normalization. Use an image annotation tool to annotate the preprocessed images to generate a training dataset containing target bounding boxes and category labels. S2. Construct a YOLO-Transformer fusion detection model, in which the self-attention mechanism of the YOLO-Transformer fusion detection model is used to enhance its ability to extract image features and understand context. S2 includes the following steps: Step S2.1: Construct a sliding window-based visual Transformer network as the feature extraction backbone and extract hierarchical features; the sliding window visual Transformer network is used to replace the cross-stage local network in the original YOLO model; Step S2.2: Construct a multi-scale feature fusion network. Between the multi-scale feature maps output by the sliding window visual Transformer network, construct a bidirectional feature pyramid network structure. This allows for bidirectional information flow from top to bottom and bottom to top between feature maps from different levels containing information on seaweed morphology, raft structure, and pole size. This information is then fused to each input feature map. Assign a learnable weight Perform weighted fusion; Step S2.3: Insert cross-stage local attention modules into one or more nodes in the feature fusion path of the bidirectional feature pyramid network. The cross-stage local attention modules first input the feature maps of the nodes. It is divided into two parts along the channel dimension. and ,in Subsequently, the first feature portion A coordinate attention mechanism is applied to generate attention maps for slender targets like poles and raft mesh structures. The computation of coordinate attention begins with the embedding of coordinate information, which is achieved through the analysis of features... Spatial information is aggregated by performing one-dimensional global pooling along both the horizontal and vertical directions to obtain orientation-aware feature vectors. （5）； in, and The first The passage at height and width The output on For feature map height, Feature map width, For the first The passage is in the location eigenvalues, For the first The passage is in the location The feature values ​​are then analyzed; next, the coordinate attention generation step is performed, where the feature vectors from the two directions are concatenated and then passed through a shared convolutional layer. and nonlinear activation functions Perform the transformation, and then decompose the result into two independent tensors. and And through independent convolutional layers , and the Sigmoid function Generate attention weights and : （6）； Finally, the generated attention weights are applied to the features. and unprocessed features The components are then stitched together to obtain the final output of the C2PSA module. : （7）； in, This is a tensor splicing operation; Step S2.4: Based on the enhanced feature map output in step S2.3, configure detection heads corresponding to different scales of features at levels P2, P3, P4, and P5 respectively; the detection head at level P2 is used to process the high-resolution feature map to detect small-sized targets such as poles in the image, while the detection heads at levels P3 to P5 perform bounding box regression and category prediction for medium and large-sized targets such as seaweed and raft nets on the feature maps of the corresponding scales respectively; S3. Based on the training dataset described in step S1, a composite loss function including localization loss, classification loss and confidence loss is adopted, and an optimization algorithm is selected to iteratively train the network parameters of the YOLO-Transformer fusion detection model described in step S2 until the performance indicators of the YOLO-Transformer fusion detection model on the seaweed farming target recognition task meet the preset requirements, and the trained detection model is obtained. The positioning loss Focal-CIoU loss is used; S4. Perform the same preprocessing as in step S1 on the image of the seaweed farming area to be detected. Input the preprocessed image into the detection model trained in step S3 for forward inference to obtain the initial detection result containing the target detection box, category and confidence. Apply the non-maximum suppression algorithm to the initial detection result to merge and filter overlapping detection boxes for the same seaweed, raft net or pole target to obtain the final detection result.

2. The method for detecting seaweed farming scenarios integrating YOLO and Transformer as described in claim 1, characterized in that, Step S2 includes the following steps: In step S2.1, the sliding window visual Transformer network is composed of multiple stacked sliding window visual Transformer modules. Through its hierarchical structure, it outputs feature maps of different resolutions at different stages. Internally, the sliding window visual Transformer module utilizes a multi-head self-attention mechanism based on shifted windows for feature learning. The calculation of the window attention mechanism follows the formula: （1）； in, For querying the matrix, The key matrix, For value matrices, Let be the dimension of the key vector. The offset is encoded as a learnable relative position; In step S2.1, the fusion calculation follows the formula: （2）； in, The output feature map after fusion. For the input of the first Each feature map For the first Learnable weights corresponding to each input feature map For the first Learnable weights corresponding to each input feature map The preset numerical stability constant is used; for the pyramid's... Layers, the intermediate features of their top-down path Output features of bottom-up paths Calculated using the following formula: （3）； （4）； in, It is the backbone network The input features of the layer The operation performs upsampling or downsampling based on the path direction. , and , , These are learnable weights. This is a convolution operation.

3. The method for detecting seaweed farming scenarios integrating YOLO and Transformer as described in claim 1, characterized in that, Step S3 includes the following steps: Step S3.1: Before training begins, first load the seaweed farming target training dataset generated in step S1. This dataset contains images with bounding boxes and category labels for seaweed, rafts, and poles. At the same time, load the YOLO-Transformer fusion detection model network architecture constructed in step S2 to initialize the model network parameters. The initialization method is to load the weights obtained by pre-training on a large-scale public dataset. Step S3.2: Define a composite loss function for target detection of seaweed, raft nets, and poles. The function is composed of a weighted average of localization loss, classification loss, and confidence loss. The calculation of the composite loss function follows the formula: （8）； in, To pinpoint the loss, For classifying losses, To determine whether there is a confidence loss for any target within the predicted bounding box, , , The preset weighting coefficients; the positioning loss Focal-CIoU loss is used, denoted as For each predicted bounding box of laver, raft net, or pole, the loss value is calculated using the formula: （9）； in, The intersection-union ratio (IoU) of the predicted bounding box of laver, raft net, or pole net with its actual labeled bounding box. The preset CIoU loss value, It is a with Values ​​or numerical values ​​related to classification confidence. Preset focusing parameters; classification loss With confidence loss Both use a binary cross-entropy loss function. Its calculation follows the formula: （10）； When calculating classification loss, For the true category label of laver, raft net, or pole, To calculate the confidence loss, the model predicts the probability of that category. To determine whether the prediction box contains the true labels of seaweed, raft nets, or pole targets. The prediction confidence level corresponding to the model; finally, based on this composite loss function The calculation results are used to calculate the gradient through the backpropagation algorithm, and the network parameters of the YOLO-Transformer fusion detection model are iteratively updated using stochastic gradient descent or its variant optimization algorithm. Step S3.3: During iterative training, the performance of the current model is periodically evaluated using a reserved validation dataset, which also contains labeled images of seaweed, raft nets, and poles. The convergence status of the model is monitored by calculating the mean precision, accuracy, and recall of the model on the validation set for the three target classes of seaweed, raft nets, and poles. When the performance indicators reach a preset threshold or no longer improve within several consecutive training cycles, the iterative training is terminated, and the network parameters of the current or validation optimal model are saved as the final trained detection model.

4. The method for detecting seaweed farming scenarios integrating YOLO and Transformer as described in claim 1, characterized in that, Step S4 includes the following steps: Step S4.1: After performing the same preprocessing operation as in step S1, the image of the seaweed farming area to be detected is input into the YOLO-Transformer fusion detection model trained in step S3 for forward inference to obtain the initial detection result containing multiple initial detection boxes, where each initial detection box is associated with a target category label as seaweed, raft net or pole and the corresponding confidence score. Step S4.2: Apply the initial detection results A category-aware nonmaximum suppression method is proposed. This method first sorts all initial detection boxes in descending order based on their confidence scores, and then processes them iteratively. In each iteration, the detection box with the highest confidence score is selected as the baseline box, and its intersection-union ratio (IU) with all other unprocessed detection boxes is calculated. Differential suppression is then performed based on the IU and category relationships. If an unprocessed detection box shares the same category as the baseline box and its IU is greater than a first preset threshold, the unprocessed detection box is suppressed. If the unprocessed detection box differs from the baseline box in category, no suppression is performed. This iterative process is repeated until all detection boxes have been processed, resulting in the final detection result.

Images