A method for identifying key elements of the work environment

Abstract

A method for identifying key elements of a work environment includes: S1, collecting and filtering work environment scene images to construct a work environment image dataset; S2, preprocessing the work environment image dataset constructed in S1 to generate a training set, a validation set, and a test set; S3, constructing a work environment scene recognition model, improving the original DeepLabV3+ model to construct a P-Net model; S4, training the P-Net model constructed in S3 using the training set and validation set generated in S2; and S5, testing and evaluating the P-Net model using the test set generated in S2.

Description

Technical Field

[0001] This invention relates to the field of computer image processing technology, and in particular to a method for identifying key elements of the working environment. Background Technology

[0002] To achieve intelligent agricultural operations, it is necessary to develop specialized intelligent agricultural machinery. This machinery should be able to autonomously perform tasks such as plant protection, ditching, and fertilization. As a key technology for intelligent agricultural machinery, the perception methods of key elements in operation (such as crops or fruit trees, roads, row paths, people, and other obstacles) can guide the machinery in path planning and autonomous obstacle avoidance, which is of great significance for improving the level of intelligence of agricultural machinery.

[0003] Traditional computer vision methods are widely used in workplace environment perception due to their stability, interpretability, and low computational cost, but they face challenges in accurate detection and localization in complex workplace environments. High-performance deep learning methods, such as Faster R-CNN and the YOLO series, perform excellently in object detection and path planning in orchard environments, but they still have limitations in capturing 3D shapes and boundary details.

[0004] Pixel-level semantic recognition methods are gradually emerging in orchard information perception, and vehicle-mounted image sensors combined with sophisticated image recognition methods offer significant advantages in vehicle navigation and environmental perception. However, the complex lighting conditions and noise interference in orchards pose challenges to environmental perception, and computational cost and recognition rate are also key issues.

[0005] DeepLabV3 is an advanced pixel-level semantic recognition model that uses Spatial Pyramid Pooling (ASPP) technology to capture multi-scale contextual information in complex scenes, improving recognition performance. DeepLabV3+ builds upon this by introducing a decoder module, further enhancing the recognition accuracy of boundary regions. However, DeepLabV3+ still faces some challenges when recognizing key elements of the working environment: high computational complexity and low recognition speed, making it difficult to meet the needs of real-time applications; reduced recognition accuracy under complex lighting conditions; and susceptibility to false positives and false negatives in noisy environments.

[0006] It is not difficult to see that there are still many problems in the existing technology, and it is necessary to improve the identification methods of key elements in the existing working environment in order to overcome the shortcomings of the existing technology. Summary of the Invention

[0007] Therefore, in order to solve the above-mentioned problems in the prior art, the present invention proposes a method for identifying key elements of the working environment.

[0008] The present invention solves the above problems through the following technical means:

[0009] A method for identifying key elements of a work environment, comprising:

[0010] S1. Collect and filter images of the work environment to construct a work environment image dataset;

[0011] S2. Preprocess the work environment image dataset constructed in S1 to generate a training set, a validation set, and a test set;

[0012] S3. The DeeplabV3+ model is improved by adopting the feature extraction network MobileNetV2, the selective attention mechanism α module and the lightweight feature extraction mechanism β module to construct the P-Net work environment scene recognition model.

[0013] S4. Use the training set and validation set generated in S2 to train the P-Net model constructed in S3;

[0014] S5. Using the test set generated in S2, perform test evaluation and model optimization on the P-Net model.

[0015] Furthermore, S1 includes:

[0016] S11. Image acquisition of the work environment: Travel along the work path and use image acquisition equipment to acquire RGB images in the direction of travel;

[0017] S12. The images acquired in S11 are filtered and processed to reduce the size of the original acquired images to d1*d2 pixels and stored to construct a work environment image dataset.

[0018] Furthermore, S2 includes:

[0019] S21. Data annotation: Annotate the key elements in the images of the work environment image dataset constructed in S1.

[0020] S22. Image enhancement: Perform data enhancement processing on the images in the work environment image dataset constructed in S1 to improve the model's generalization ability.

[0021] S23. Dataset partitioning: The dataset enhanced in S22 is partitioned into training set, validation set, and test set in a ratio of 7:1:2.

[0022] Furthermore, S3 includes:

[0023] S31. Using the DeepLabV3+ model as the base model, replace the Xception feature extraction network in the encoder part of the DeepLabV3+ model with the MobileNetv2 feature extraction network to reduce the model size.

[0024] S32. The lightweight feature extraction mechanism β module is used as the feature extraction algorithm of the multi-scale feature extraction layer DP structure, and the ASPP structure in the DeepLabV3+ model is replaced with the DP structure to improve the model recognition speed and further improve the model's ability to recognize multi-scale targets in the context of noise interference.

[0025] S33. Selective attention mechanism α module is used to extract key information at the output end of the MobileNetv2 feature extraction network and the output end of the DP structure, respectively, to further improve the model’s recognition ability under complex lighting conditions.

[0026] S34. The lightweight feature extraction mechanism β module is used to replace the standard convolution in the decoder part of the original DeepLabV3+ model to quickly restore the spatial resolution of the feature map and improve the recognition effect.

[0027] S35. The DF loss function is used as the loss function of P-Net to handle the imbalance problem between target information pixels and background pixels.

[0028] Furthermore, S32 includes: the DP structure first receives the output features X of the MobileNetv2 feature extraction network, and performs feature extraction using a lightweight feature extraction mechanism β module with different convolutional kernels K and dilation rates r. Finally, global average pooling is used to capture global context information; wherein the calculation method of the β module is as follows:

[0029]

[0030] In the formula, C in It is the number of input channels, Y(h, w, c) out ) is the output feature map of the β module. X(h+r·i, w+r·j, c) is the element in the input feature map X after its position is adjusted by dilated convolution, where r is the dilation rate, and i and j are the spatial indices of the convolution kernel. W dw (i, j, c) is a depthwise convolution kernel of size K×K, with an independent kernel for each input channel c. pw (c,c out ) is a pointwise convolution kernel with a size of 1×1×C. in ×C out .

[0031] Furthermore, S33 includes: a selective attention mechanism α module that, through the input feature map... The process involves assigning a weight to each channel, where H×W×C represents the height, width, and number of channels of the feature map, respectively.

[0032] First, global average pooling is performed on each channel to obtain a C-dimensional vector. Then, the pooled feature vector is processed by a one-dimensional convolution operation Conv1D(·) to generate a channel vector. Next, the convolution result is normalized using the Sigmoid activation function σ(·) to obtain the final channel weights. Finally, the weights are applied channel by channel to the original input feature map X to obtain the weighted feature map X′, thereby selectively enhancing the features of important channels and suppressing the information of unimportant channels.

[0033]

[0034] In the formula, X ijc X represents ijc The value at position (i, j) and channel c.

[0035] Furthermore, S34 includes: replacing the standard convolution operation in the decoder part of the original DeepLabV3+ model with the lightweight feature extraction mechanism β module; during the feature extraction process, the computational cost of the β module is 1 / K of the computational cost of the standard convolution. 2 Where K is the size of the convolution kernel; the computational complexity formulas are as follows:

[0036] computational load β =C in ×K 2 ×H×W+C in ×C out ×H×W

[0037] computational load 标准卷积 =C in ×C out ×K 2 ×H×W

[0038] computational load β / computational complexity 标准卷积 =1 / K 2 +1 / C out ≈1 / K 2 .

[0039] Furthermore, S35 includes: using the DF loss function as the loss function of P-Net to handle the imbalance problem between target information pixels and background pixels;

[0040] First, by optimizing the overlap between the target region and the predicted region, the segmentation accuracy of the target region is enhanced. Second, by weighting the contribution of the background region, the interference of background pixels on the training process is reduced, allowing the model to focus more on learning the target region. The DF calculation formula is as follows:

[0041]

[0042] In the formula, ∈ is a very small constant, and N is the total number of samples. and These are the true label and the predicted result for the i-th sample, respectively.

[0043] Furthermore, S4 includes:

[0044] S41. Configure training hyperparameters, inputting the image resolution of the training set as d1*d2, the batch size as B, and the number of iterations as E rounds;

[0045] S42. Start training the model. In each training iteration, calculate the model loss value and accuracy value, and update the model weights.

[0046] S43. During training, the P-Net model periodically outputs training metrics, including loss values ​​and accuracy metrics, and validates them on the validation set to obtain the model's performance metrics.

[0047] Furthermore, the test evaluation described in S5 includes the following metrics:

[0048] Pixel accuracy PA is calculated using the following formula: P ii P is the total number of pixels that belong to class i and are predicted to be class i. ij K+1 is the total number of pixels belonging to class i and predicted as class j, where K+1 is the number of classes.

[0049] Average pixel accuracy (MPA) is calculated using the following formula: P ii P is the total number of pixels that belong to class i and are predicted to be class i. ij K+1 is the total number of pixels belonging to class i and predicted as class j, where K+1 is the number of classes.

[0050] The formula for calculating the Mean Intersection over Union (MIOU) is as follows: P ii P is the total number of pixels that belong to class i and are predicted to be class i. ji K+1 represents the total number of pixels that belong to class j but are predicted as class i, and K+1 represents the number of classes.

[0051] The model optimization in S5 includes:

[0052] First, based on the evaluation results, the model's performance across various categories is analyzed, with particular attention paid to regions with low object detection and segmentation accuracy. Second, the weights in the loss function are dynamically adjusted to enhance the optimization effect on difficult samples and small targets. Then, a learning rate scheduler and early stopping strategy are employed to prevent overfitting and improve the model's generalization ability. Finally, the structure or hyperparameters of the P-Net model are fine-tuned based on the evaluation results to ensure optimal accuracy in identifying key elements in the operational environment and to improve the model's practical application performance and stability.

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0054] This invention significantly reduces model size and improves recognition speed by replacing the Xception feature extraction network in the encoder part of DeepLabV3+ with a MobileNetv2 feature extraction network. A lightweight feature extraction mechanism β module is used as the feature extraction algorithm for the multi-scale feature extraction layer DP structure, and the ASPP structure in the DeepLabV3+ model is replaced with the DP structure to improve model recognition speed and further enhance the model's ability to recognize multi-scale targets in noisy environments. Selective attention mechanism α modules are used at the outputs of the MobileNetv2 feature extraction network and the DP structure to extract key information, further improving the model's recognition ability under complex lighting conditions. The lightweight feature extraction mechanism β module replaces the standard convolution in the decoder part of the original DeepLabV3+ model to quickly recover the spatial resolution of the feature map, improving recognition performance. Finally, the DF loss function is used as the loss function for the constructed P-Net model to address the imbalance between target information pixels and background pixels.

[0055] Compared to recognition models such as DeepLabV3+, U-Net, PSPNet, Segformer, and HRNet, P-Net has higher recognition accuracy and speed when detecting images of the work environment. It can effectively identify key elements in the work environment. Furthermore, the P-Net model has a small memory footprint, high accuracy, and fast speed, which can well support the autonomous navigation and obstacle avoidance of the work machine. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1This is the original network structure diagram of DeepLabV3+ as described in this invention;

[0058] Figure 2 This is a network structure diagram of the P-Net described in this invention;

[0059] Figure 3 This is a schematic diagram of the DP structure described in this invention; Figure 4 This is a schematic diagram of the α module described in this invention;

[0060] Figure 5 This is a schematic diagram comparing the standard convolutional and β-module structures described in this invention;

[0061] Figure 6 This is a diagram showing the training results of the P-Net described in this invention;

[0062] Figure 7 This is a diagram showing the recognition effect of the P-Net described in this invention on pomelo orchard images under different lighting conditions;

[0063] Figure 8 This is a diagram showing the recognition effect of the P-Net described in this invention on pomelo orchard images with different levels of noise. Detailed Implementation

[0064] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the described embodiments are merely some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0065] Example

[0066] The following uses the identification of key elements in an orchard work environment as an example to illustrate the method of the present invention in detail: A method for identifying key elements in a work environment, comprising:

[0067] S1. Collect and filter images of the work environment to construct a work environment image dataset; that is, collect and filter orchard scene images to construct an orchard image dataset; preferably, S1 includes:

[0068] S11. Image acquisition of the working environment: traveling along the working path, using an image acquisition device to acquire RGB images in the direction of travel; that is, orchard image acquisition: traveling along the path between the rows of fruit trees in the orchard, using an image acquisition device to acquire RGB images in the direction of travel; it should be noted that, in this invention, it is preferable that the working vehicle travels along the path between the rows of fruit trees in the orchard, while simultaneously calling the camera installed on the working vehicle to acquire RGB images in front of the working vehicle.

[0069] S12. The images acquired in S11 are filtered and processed to reduce the original image size to d1*d2 pixels and stored to construct a work environment image dataset. It should be noted that the d1*d2 pixels can be selected according to the actual application. That is, the images acquired in S11 are filtered and processed to reduce the original image size to 576×324 pixels and stored to construct an orchard image dataset. Generally, the RGB image size acquired by the camera mounted on the work vehicle is 2208×1242 pixels, so it needs to be scaled to 576×324 pixels to improve the model detection speed. Preferably, this invention compresses the original image size to 576×324 pixels and stores it. Because larger image sizes result in slower detection speeds, while smaller sizes result in faster detection speeds, but the recognition accuracy may decrease. Furthermore, if the scaling is not proportional, it will lead to image distortion; therefore, this invention preferably uses proportional image scaling.

[0070] S2. Preprocess the operational environment image dataset constructed in S1 to generate a training set, a validation set, and a test set; that is, preprocess the orchard image dataset constructed in S1 to generate a training set, a validation set, and a test set; preferably, S2 includes:

[0071] S21. Data annotation: Key elements in the images of the operational environment image dataset constructed in S1 are annotated; specifically, the images in the orchard image dataset constructed in S1 are annotated. Preferably, this invention uses the open-source software Labelme to annotate the orchard image dataset constructed in S12, selecting the JSON annotation format to mark fruit trees, roads, paths between fruit trees, people, and obstacles in the images. The advantage of the JSON format is that structured data easily expresses and parses object information in images, and it widely supports various deep learning frameworks, facilitating subsequent model training and processing. The model proposed in this invention supports multiple dataset formats, commonly including COCO, Pascal VOC, and TFRecord formats. The specific choice depends on the type, size, and project requirements of the dataset.

[0072] S22. Image Enhancement: Data enhancement processing is performed on the images in the operational environment image dataset constructed in S1 to improve the model's generalization ability. Specifically, data enhancement is performed on the images in the orchard image dataset constructed in S1 to improve the model's generalization ability. It should be noted that the methods for data enhancement of the images in the orchard image dataset include, but are not limited to: rotation, translation, flipping, noise addition, and brightness adjustment to alleviate model overfitting and improve model generalization ability. It should also be noted that, in addition to rotation, translation, flipping, noise addition, and brightness adjustment, image enhancement methods such as scaling, cropping, color dithering, blurring and sharpening, random cropping, channel swapping, blending enhancement, occlusion enhancement, elastic deformation, and gamma correction can also be used. These enhancement methods help improve the model's adaptability to different scenes and changes, prevent overfitting, and enhance generalization ability.

[0073] S23. Dataset partitioning: The dataset enhanced in S22 is partitioned into training set, validation set, and test set in a ratio of 7:1:2.

[0074] S3. Improve the DeepLabV3+ model by using the feature extraction network MobileNetV2, the selective attention mechanism α module, and the lightweight feature extraction mechanism β module to construct the working environment scene recognition model P-Net; that is, construct the orchard scene recognition model and build the original DeepLabV3+ model into the P-Net model.

[0075] It should be noted that the original DeepLabV3+ model includes an encoder and a decoder:

[0076] In the encoder section, DeepLabV3+ employs the Xception network. The Xception network effectively captures spatial features and contextual information of the image, providing high-quality feature maps for subsequent feature processing. However, due to the large number of layers, the overall computational complexity remains high. At the end of the encoder, DeepLabV3+ introduces the Spatial Pyramid Pooling (ASPP) module, which captures multi-scale contextual information through dilated convolutions at different scales, enhancing the model's perception capability at different scales. However, multi-scale dilated convolutions increase computational cost.

[0077] In the decoder section, firstly, the decoder performs channel-wise dimensionality reduction on the low-level feature map using a 1×1 standard convolution, reducing the number of channels from 256 to 48. Next, the feature map from ASPP is interpolated and upsampled to the same size as the dimensionality-reduced low-level feature map. This step ensures spatial alignment of the feature maps, facilitating subsequent fusion processing. The channel-wise dimensionality-reduced low-level feature map is concatenated with the interpolated and upsampled feature map and fed into a set of 3×3 standard convolutional blocks for further processing. Through feature fusion, the decoder can combine low-level and high-level features; however, noise in both low-level and high-level features may affect recognition accuracy. Finally, linear interpolation upsampling is performed to restore the feature map to the same resolution as the original image, thereby generating the final prediction map.

[0078] Preferably, S3 includes:

[0079] S31. Using the DeepLabV3+ model as the base model, the Xception feature extraction network in the encoder part of the DeepLabV3+ model is replaced with the MobileNetv2 feature extraction network to reduce the model size; that is, replacing the Xception feature extraction network in the encoder part of the DeepLabV3+ model with the MobileNetv2 feature extraction network can improve feature extraction efficiency. It should be noted that because MobileNetv2 is a lightweight network, its design uses depthwise separable convolutions and inverse residual structures, which significantly reduces the number of parameters and computational cost, while maintaining good feature extraction capabilities, enabling feature extraction to be completed with lower computational resources and latency. In contrast, the Xception network is more complex, using more residual connections and a deep architecture, resulting in a larger number of parameters and computational cost.

[0080] S32. A lightweight feature extraction mechanism β module is adopted as the feature extraction algorithm for the multi-scale feature extraction layer DP structure. The ASPP structure in the DeepLabV3+ model is replaced with the DP structure to improve the model's recognition speed and further enhance the model's ability to recognize multi-scale targets in the context of noise interference. Preferably, the DP structure first receives the output features X of the MobileNetv2 feature extraction network, and then uses the lightweight feature extraction mechanism β module with different convolutional kernels K and dilation rates r for feature extraction. Finally, global average pooling is used to capture global contextual information. The calculation method of the β module is as follows:

[0081]

[0082] In the formula, C in It is the number of input channels, Y(h, w, c) out) is the output feature map of the β module. X(h+r·i, w+r·j, c) is the element in the input feature map X after its position is adjusted by dilated convolution, where r is the dilation rate, and i and j are the spatial indices of the convolution kernel. W dw (i, j, c) is a depthwise convolution kernel of size K×K, with an independent kernel for each input channel c. pw (c,c out ) is a pointwise convolution kernel with a size of 1×1×C. in ×C out .

[0083] S33. A selective attention mechanism α module is used at the output ends of the MobileNetv2 feature extraction network and the DP structure to extract key information, further improving the model's recognition ability under complex lighting conditions. Specifically, a selective attention mechanism α module is used at the output ends of the MobileNetv2 feature extraction network and the DP structure to extract key information under different lighting conditions, thereby further improving the model's recognition ability under complex lighting conditions. Preferably, the selective attention mechanism α module extracts key information from the input feature map... The process involves assigning a weight to each channel, where H×W×C represents the height, width, and number of channels of the feature map, respectively. First, global average pooling is performed on each channel to obtain a C-dimensional vector. Next, a one-dimensional convolution operation, Conv1D(·), is used to process the pooled feature vector, generating a channel vector. Then, the convolution result is normalized using the Sigmoid activation function σ(·) to obtain the final channel weights. Finally, these weights are applied channel-by-channel to the original input feature map X, resulting in a weighted feature map X′, thus selectively enhancing the features of important channels while suppressing information from less important channels.

[0084]

[0085] In the formula, X ijc X represents ijc The value at position (i, j) and channel c.

[0086] S34. A lightweight feature extraction mechanism β module is used to replace the standard convolution in the decoder part of the original DeepLabV3+ model to quickly restore the spatial resolution of the feature map and improve the recognition effect. Preferably, the lightweight feature extraction mechanism β module is used to replace the standard convolution operation in the decoder part of the original DeepLabV3+ model; during feature extraction, the computational cost of the β module is 1 / K of the computational cost of the standard convolution. 2 Because of C out The computational cost of the β module is very large, therefore it is only 1 / K of the computational cost of a standard convolution.2 Where K is the size of the convolution kernel; the computational complexity formulas are as follows:

[0087] computational load β =C in ×K 2 ×H×W+C in ×C out ×H×W

[0088] computational load 标准卷积 =C in ×C out ×K 2 ×H×W

[0089] computational load β / computational complexity 标准卷积 =1 / K 2 +1 / C out ≈1 / K 2 .

[0090] S35. The DF loss function is used as the loss function for P-Net to address the imbalance between target information pixels and background pixels. First, by optimizing the overlap between the target region and the predicted region, the segmentation accuracy of the target region is enhanced, preventing the background region from having an excessive impact on the total loss, especially when the target region has few pixels. Second, by weighting the contribution of the background region, the interference of background pixels on the training process is reduced, allowing the model to focus more on learning the target region. The DF calculation formula is as follows:

[0091]

[0092] In the formula, ∈ is a very small constant, and N is the total number of samples. and These are the true label and the predicted result for the i-th sample, respectively.

[0093] It should be noted that in the P-Net model:

[0094] In the encoder section, the Xception feature extraction network in the encoder section of the DeepLabV3+ model is replaced with the MobileNetv2 feature extraction network to reduce the model size; a lightweight feature extraction mechanism β module is used as the feature extraction algorithm for the multi-scale feature extraction layer DP structure, and the ASPP structure in the DeepLabV3+ model is replaced with the DP structure to improve the model recognition speed and further improve the model's ability to recognize multi-scale targets in noisy backgrounds. Furthermore, a selective attention mechanism α module is used at the output of the MobileNetv2 feature extraction network and the output of the DP structure to extract key information, further improving the model's recognition ability under complex lighting conditions. In the decoder section, the standard convolution in the decoder section of the original DeepLabV3+ model is replaced with the lightweight feature extraction mechanism β module to quickly recover the spatial resolution of the feature map and improve the recognition effect. In addition, the DF loss function is used as the loss function of P-Net to handle the imbalance problem between target information pixels and background pixels. S4: The P-Net model constructed in S3 is trained using the training set and validation set generated in S2; preferably, S4 includes:

[0095] S41. Configure training hyperparameters, inputting the image resolution of the training set as d1*d2, the batch size as B, and the number of iterations as E rounds; that is, configure training hyperparameters, inputting the image resolution of the training set as 576×324, the batch size as 8, and the number of iterations as 500 rounds.

[0096] S42. Start training the model. In each training iteration, calculate the model loss value and accuracy value, and update the model weights.

[0097] S43. During training, the P-Net model periodically outputs training metrics, including loss values ​​and accuracy metrics, and validates them on the validation set to obtain the model's performance metrics.

[0098] S5. Using the test set generated in S2, perform test evaluation and model optimization on the P-Net model.

[0099] Preferably, the test evaluation in S5 includes the following indicators:

[0100] Pixel accuracy is calculated using the following formula: P ii P is the total number of pixels that belong to class i and are predicted to be class i. ij K+1 is the total number of pixels belonging to class i and predicted as class j, where K+1 is the number of classes.

[0101] Average pixel accuracy (MPA) is calculated using the following formula: P iiP is the total number of pixels that belong to class i and are predicted to be class i. ij K+1 is the total number of pixels belonging to class i and predicted as class j, where K+1 is the number of classes.

[0102] The formula for calculating the Mean Intersection over Union (MIOU) is as follows: P ii P is the total number of pixels that belong to class i and are predicted to be class i. ji K+1 represents the total number of pixels that belong to class j but are predicted as class i, and K+1 represents the number of classes.

[0103] Preferably, the model optimization in S5 includes: first, analyzing the model's performance across various categories based on the evaluation results, with particular attention to regions with low object detection and segmentation accuracy; second, dynamically adjusting the weights in the loss function to enhance the optimization effect on difficult samples and small targets; then, preventing overfitting and improving the model's generalization ability by employing a learning rate scheduler and an early stopping strategy; and finally, fine-tuning the structure or hyperparameters of the P-Net model based on the evaluation results to ensure optimal accuracy in identifying key elements in the working environment and to improve the model's practical application performance and stability.

[0104] To make the present invention easier to understand, the method mentioned in the present invention will be further described below using a pomelo orchard as an example:

[0105] S1. Collect and filter images of orchard scenes to construct an orchard image dataset; that is, use a camera to collect images of pomelo orchard scenes to construct a pomelo orchard image dataset.

[0106] S11. Orchard image acquisition: The vehicle travels along the path between the rows of fruit trees in the orchard and uses image acquisition equipment to acquire RGB images in the direction of travel; the vehicle is controlled to travel along the path between the rows of fruit trees in the pomelo orchard, while the camera installed on the vehicle is used to acquire RGB images in front of the vehicle.

[0107] S12. The images acquired in S11 are filtered and processed, and the original image size is reduced to 576×324 pixels and stored to construct an orchard image dataset. The acquired images are filtered and the original image size of 2208×1242 pixels is scaled down to 576×324 pixels to improve the model detection speed.

[0108] S2. Preprocess the orchard image dataset constructed in S1 to generate a training set, a validation set, and a test set; that is, preprocess the pomelo orchard dataset including annotation and image enhancement, and divide it into a training set, a validation set, and a test set.

[0109] S21. Data annotation: Annotate the images in the orchard image dataset constructed in S1; that is, use the open-source software Labelme to annotate the images, select the JSON annotation format, and mark the pomelo trees, roads, fruit tree rows, people, signs and streetlights in the images.

[0110] S22. Image enhancement: Data enhancement is performed on the images in the orchard image dataset constructed in S1 to improve the model's generalization ability; that is, data enhancement is performed on the images, including rotation, translation, flipping, noise addition and brightness adjustment to alleviate model overfitting and improve model generalization ability.

[0111] S23. Dataset partitioning: The dataset enhanced in S22 is partitioned into training set, validation set, and test set in a ratio of 7:1:2.

[0112] S3. Construct an orchard scene recognition model, transforming the original DeepLabV3+ model into a P-Net model; that is, construct a pomelo orchard scene recognition model, including improving the original DeepLabV3+ model to obtain P-Net.

[0113] like Figure 1 As shown, the original DeepLabV3+ network structure includes an encoder part and a decoder part.

[0114] The encoder's feature extraction backbone network is Xception. The feature map output by Xception is divided into two parts: one part is the feature map output from the last convolutional layer, and the other part is the feature map of the intermediate low-level features. The first part of Xception's output is fed into the ASPP module, and the second part is fed into the decoder. The ASPP module takes the first part of Xception's output as input and uses four dilated convolutional blocks with different dilation rates (including convolution, BN, and activation layers) and a global average pooling block (including pooling, convolution, BN, and activation layers) to obtain a total of five sets of feature maps. After concatenating these, the maps are passed through a 1×1 convolutional block (including convolution, BN, activation, and dropout layers) and finally fed into the decoder.

[0115] The decoder first takes low-level features from the shallow layers of the encoder and performs a 1×1 convolution, then fuses them with the encoder's output features. Low-level features typically retain more spatial detail, such as edges and textures. Next, the encoder's output feature map is upsampled by 4x to restore the low-resolution feature map to a higher resolution. The upsampled feature map is then concatenated with the low-level feature map along the channel dimension to form a new feature map. This new feature map is processed through multiple convolutional layers to further fuse and refine the features. The final feature map is again upsampled to restore the original resolution of the input image. A 3×3 convolutional layer is used to convert the feature map to the same number of channels as the number of classes, generating a pixel-level classification probability map. The classification probability map determines the class of each pixel, and the final output is a recognition map of the same size as the input image, with each pixel labeled as a specific class.

[0116] The encoder and decoder of DeepLabV3+ were improved respectively to obtain the P-Net model, as follows: Figure 2 As shown. Specific improvements include:

[0117] Improved encoder section:

[0118] 1) Using the DeepLabV3+ model as the base model, the Xception feature extraction network in the encoder part of the DeepLabV3+ model is replaced with the MobileNetv2 feature extraction network to reduce the model size; for example... Figure 3 As shown, a lightweight feature extraction mechanism β module is used as the feature extraction algorithm for the multi-scale feature extraction layer DP structure. The ASPP structure in the DeepLabV3+ model is replaced with the DP structure to improve the model's recognition speed and further enhance its ability to recognize multi-scale targets in noisy environments. Figure 4 As shown, selective attention mechanism α modules are used at the output of the MobileNetv2 feature extraction network and the output of the DP structure to extract key information, so as to further improve the model's recognition ability under complex lighting conditions.

[0119] 2) Improved decoder section:

[0120] like Figure 5 As shown, a lightweight feature extraction mechanism, the β module, is used to replace the standard convolution in the decoder part of the original DeepLabV3+ model to quickly recover the spatial resolution of the feature map and improve the recognition effect. The DF loss function is used as the loss function of P-Net to handle the imbalance between target information pixels and background pixels, ensuring that the model has robust performance.

[0121] S4. Use the training set and validation set generated in S2 to train the P-Net model constructed in S3; that is, use the pomelo orchard dataset to train the P-Net model.

[0122] S41. Configure training hyperparameters, inputting the image resolution of the training set as 576×324, the batch size as 8, and the number of iterations as 500 rounds;

[0123] S42. Start training the model. In each training iteration, calculate the model loss value and accuracy value, and update the model weights.

[0124] S43. During training, the P-Net model periodically outputs training metrics, including loss and accuracy, and validates the model on a validation set to obtain performance metrics. For example... Figure 6 The figures show the mIoU and loss curves of P-Net on the validation set. The mIoU value of P-Net rapidly rises to 70% after the start of training, and begins to converge quickly around 40 epochs, eventually reaching approximately 89% after multiple iterations. Furthermore, the loss value of P-Net also decreases rapidly after the start of training, initially converging to around 0.25, and finally converging to below 0.18 after 500 iterations of model training. The results show that the training process of P-Net is stable, with no signs of overfitting or underfitting.

[0125] S5. Using the test set generated in S2, test and evaluate the P-Net model.

[0126] Ablation experiments were conducted on the test set to verify the impact of various improvement methods on P-Net. The results are shown in Table 1. DeepLabV3+ achieved mIoU and mPA of 80.8% and 84%, respectively, on the test set. However, with the integration of MobileNetv2, the DP structure, and the α module, DeepLabV3+ performance was significantly enhanced. mIoU increased to 86.2%, 88.4%, and 89.3%, respectively, and mPA improved to 90.8%, 93.8%, and 95.1%, respectively. Notably, P-Net exhibited superior recognition accuracy, with an 8.5% improvement in mIoU and an 11.1% improvement in mPA compared to the original DeepLabV3+. Furthermore, P-Net demonstrated advantages in computational complexity and model size. Compared to DeepLabV3+, the number of parameters, FLOPs, and volume were reduced by 94.8%, 94.3%, and 94.6%, respectively, while achieving a detection speed of 32 FPS.

[0127] Table 1

[0128]

[0129] The test set was used to evaluate P-Net's recognition performance on orchard images under different lighting conditions. For example... Figure 7 As shown in Table 2, P-Net achieves better recognition results compared to U-Net, PSPNet, Segformer, and HRNet. Compared to U-Net, PSPNet, Segformer, and HRNet, P-Net improves mIoU by 2.3%, 5.1%, 3.6%, and 2.7%, respectively, and mPA by 1.5%, 4.9%, 1.3%, and 2.7%, respectively. Furthermore, while maintaining a lightweight footprint, P-Net achieves a detection speed of 32 FPS. The results demonstrate that P-Net exhibits good robustness and generalization ability under different lighting conditions.

[0130] Table 2

[0131] Model Number of images mIoU / % mPA / % Parameter quantity / M FLOPs / G Volume / MB Speed / FPS P-Net 424 89.8 95.2 2.87 9.93 11.2 32 U-Net 424 87.5 93.7 24 159 94.9 27 PSPNet 424 84.7 90.3 2.37 2.20 9.3 23.2 Segformer 424 86.2 93.9 3.72 4.88 14.2 17.2 HRNet 424 87.1 92.5 9.64 13.49 37.5 20.5

[0132] The P-Net recognition performance on orchard images with varying levels of noise was tested using a test set. The recognition results of P-Net on orchard images with different levels of noise are as follows: Figure 8 As shown in Table 3, the recognition results show that P-Net performs well in recognizing images with slight noise, achieving a PA exceeding 91.3% for each perceptual element, and mIoU and mPA reaching 90.2% and 95.4%, respectively. When recognizing images with moderate and severe noise, P-Net achieves mIoUs of 86.2% and 84.7%, and mPAs of 93.2% and 90.9%, respectively. Compared to the preferred U-Net, P-Net demonstrates higher mIoUs (1.6%, 1.8%, and 6.6% higher) and higher mPAs (6.4%, 9.1%, and 11.1% higher), respectively, when recognizing images with slight, moderate, and severe noise. These results indicate that P-Net has good recognition capabilities for orchard images with varying degrees of noise.

[0133] Table 3

[0134]

[0135] This invention significantly reduces the model size and improves recognition speed by replacing the Xception feature extraction network in the encoder part of DeepLabV3+ with the MobileNetv2 feature extraction network. A lightweight feature extraction mechanism β module is used as the feature extraction algorithm for the multi-scale feature extraction layer DP structure, and the ASPP structure in the DeepLabV3+ model is replaced with the DP structure to improve the model's recognition speed and further enhance its ability to recognize multi-scale targets in noisy environments. Selective attention mechanism α modules are used at the outputs of the MobileNetv2 feature extraction network and the DP structure to extract key information, further improving the model's recognition ability under complex lighting conditions. In the encoder part, the lightweight feature extraction mechanism β module replaces the standard convolution in the decoder part of the original DeepLabV3+ model to quickly recover the spatial resolution of the feature map, improving recognition performance. Finally, the DF loss function is used as the loss function for the constructed P-Net model to address the imbalance between target information pixels and background pixels.

[0136] Compared to recognition models such as DeepLabV3+, U-Net, PSPNet, Segformer, and HRNet, P-Net offers higher accuracy and speed in orchard image recognition. It can effectively identify key elements in orchards (fruit trees, roads, fruit tree rows, people, and other obstacles). Furthermore, the P-Net model is characterized by its small memory footprint, high accuracy, and fast speed, making it well-suited for supporting autonomous navigation and obstacle avoidance for intelligent agricultural machinery in orchards.

[0137] In this specification, terms such as "one embodiment," "another embodiment," "embodiment," and "preferred embodiment" refer to specific features, structures, or characteristics described in connection with that example, which are included in at least one embodiment generally described in this application. The appearance of the same expression in multiple places in the specification does not necessarily refer to the same embodiment. Furthermore, when a specific feature, structure, or characteristic is described in connection with any embodiment, the intention is to suggest that implementing such a feature, structure, or characteristic in conjunction with other embodiments also falls within the scope of this invention. Although the invention has been described herein with reference to several illustrative examples, it should be understood that those skilled in the art can devise many other modifications and implementations that fall within the scope and spirit of the principles disclosed in this application. More specifically, in addition to variations and improvements to the subject matter and its components and / or layout within the scope of this application, drawings, and claims, other uses will be apparent to those skilled in the art.

Claims

1. A method for identifying key elements of a work environment, characterized in that, include: S1. Collect and filter images of the work environment to construct a work environment image dataset; S2. Preprocess the work environment image dataset constructed in S1 to generate a training set, a validation set, and a test set; S3. The DeeplabV3+ model is improved by adopting the feature extraction network MobileNetV2, the selective attention mechanism α module and the lightweight feature extraction mechanism β module to construct the P-Net work environment scene recognition model. S4. Use the training set and validation set generated in S2 to train the P-Net model constructed in S3; S5. Using the test set generated in S2, perform test evaluation and model optimization on the P-Net model; The β module is calculated as follows: In the formula, h and w represent the spatial location indices of the feature map, respectively, and C out Indicates the number of output channels, C in It is the number of input channels, Y(h,w,c) out ) is the output feature map of the β module, and X(h+r·i,w+r·j,c) is the element in the input feature map X after being repositioned by dilated convolution, where r is the dilation rate, i and j are the spatial indices of the convolution kernel, and W... dw (i,j,c) is a depthwise convolution kernel of size K×K, and there is an independent convolution kernel for each input channel c. pw (c,c out ) is a pointwise convolution kernel with a size of 1×1×C. in ×C out ; The selective attention mechanism α module, through the input feature map The process involves assigning a weight to each channel, where H, W, and C represent the height, width, and number of channels of the feature map, respectively. First, global average pooling is performed on each channel to obtain a C-dimensional vector. Then, the pooled feature vector is processed by a one-dimensional convolution operation Conv1D(·) to generate a channel vector. Next, the convolution result is normalized using the Sigmoid activation function σ(·) to obtain the final channel weights. Finally, the weights are applied channel by channel to the original input feature map X to obtain the weighted feature map X', thereby selectively enhancing the features of important channels and suppressing the information of unimportant channels. In the formula, X ijc This represents the value of the input feature map X at position (i, j) and channel c, and ⊙ represents channel-wise multiplication, which applies weights to the corresponding channels of the input feature map.

2. The method for identifying key elements of the working environment according to claim 1, characterized in that, S1 includes: S11. Image acquisition of the work environment: Travel along the work path and use image acquisition equipment to acquire RGB images in the direction of travel; S12. The images acquired in S11 are filtered and processed to reduce the size of the original acquired images to d1*d2 pixels and stored to construct a work environment image dataset.

3. The method for identifying key elements of the working environment according to claim 1, characterized in that, S2 includes: S21. Data annotation: Annotate the key elements in the images of the work environment image dataset constructed in S1. S22. Image enhancement: Perform data enhancement processing on the images in the work environment image dataset constructed in S1 to improve the model's generalization ability. S23. Dataset partitioning: The dataset enhanced in S22 is partitioned into training set, validation set, and test set in a ratio of 7:1:

2.

4. The method for identifying key elements of the working environment according to claim 1, characterized in that, S3 includes: S31. Using the DeepLabV3+ model as the base model, replace the Xception feature extraction network in the encoder part of the DeepLabV3+ model with the MobileNetv2 feature extraction network to reduce the model size. S32. The lightweight feature extraction mechanism β module is used as the feature extraction algorithm of the multi-scale feature extraction layer DP structure, and the ASPP structure in the DeepLabV3+ model is replaced with the DP structure to improve the model recognition speed and further improve the model's ability to recognize multi-scale targets in the context of noise interference. S33. Selective attention mechanism α module is used to extract key information at the output end of the MobileNetv2 feature extraction network and the output end of the DP structure, respectively, to further improve the model’s recognition ability under complex lighting conditions. S34. The lightweight feature extraction mechanism β module is used to replace the standard convolution in the decoder part of the original DeepLabV3+ model to quickly restore the spatial resolution of the feature map and improve the recognition effect. S35. The DF loss function is used as the loss function of P-Net to handle the imbalance problem between target information pixels and background pixels.

5. The method for identifying key elements of the working environment according to claim 4, characterized in that, The S32 includes: the DP structure first receives the output features X of the MobileNetv2 feature extraction network, and performs feature extraction using a lightweight feature extraction mechanism β module with different convolutional kernels K and dilation r, and finally uses global average pooling to capture global context information.

6. The method for identifying key elements of the working environment according to claim 5, characterized in that, S34 includes: replacing the standard convolution operation in the decoder part of the original DeepLabV3+ model with a lightweight feature extraction mechanism β module; during feature extraction, the computational cost of the β module is 1 / K of the computational cost of the standard convolution. 2 Where K is the size of the convolution kernel; the computational complexity formulas are as follows: computational load β =C in ×K 2 ×H×W+C in ×C out ×H×W computational load 标准卷积 =C in ×C out ×K 2 ×H×W computational load β / computational complexity 标准卷积 =1 / K 2 +1 / C out ≈1 / K 2 .

7. The method for identifying key elements of the working environment according to claim 6, characterized in that, S35 includes: using the DF loss function as the loss function of P-Net to handle the imbalance problem between target information pixels and background pixels; First, by optimizing the overlap between the target region and the predicted region, the segmentation accuracy of the target region is enhanced. Second, by weighting the contribution of the background region, the interference of background pixels on the training process is reduced, allowing the model to focus more on learning the target region. The DF calculation formula is as follows: In the formula, ∈ is a very small constant, and N is the total number of samples. and These are the true label and the predicted result for the i-th sample, respectively.

8. The method for identifying key elements of the working environment according to claim 3, characterized in that, S4 includes: S41. Configure training hyperparameters, inputting the image resolution of the training set as d1*d2, the batch size as B, and the number of iterations as E rounds; S42. Start training the model. In each training iteration, calculate the model loss value and accuracy value, and update the model weights. S43. During training, the P-Net model periodically outputs training metrics, including loss values ​​and accuracy metrics, and validates them on the validation set to obtain the model's performance metrics.

9. The method for identifying key elements of the working environment according to claim 1, characterized in that, The test evaluation described in S5 includes the following metrics: The pixel accuracy PA is calculated using the following formula: P ii P is the total number of pixels that belong to class i and are predicted to be class i. ij k+1 is the total number of pixels belonging to class i and predicted as class j, and k+1 is the number of classes. Average pixel accuracy (MPA) is calculated using the following formula: P ii P is the total number of pixels that belong to class i and are predicted to be class i. ij k+1 is the total number of pixels belonging to class i and predicted as class j, and k+1 is the number of classes. The formula for calculating the Mean Intersection over Union (MIOU) is as follows: P ii P is the total number of pixels that belong to class i and are predicted to be class i. ji k+1 represents the total number of pixels belonging to class j but predicted as class i, where k+1 is the number of classes. The model optimization in S5 includes: First, based on the evaluation results, the model's performance across various categories is analyzed, focusing on regions with low object detection and segmentation accuracy. Second, the weights in the loss function are dynamically adjusted to enhance the optimization effect on difficult samples and small targets. Then, an early stopping strategy and a learning rate scheduler are employed to prevent overfitting and improve the model's generalization ability. Finally, the structure or hyperparameters of the P-Net model are fine-tuned based on the evaluation results to ensure optimal accuracy in identifying key elements in the operational environment and to improve the model's practical application performance and stability.

Images