Vegetable inter-ridge visual navigation method combining feature weighting and semantic alignment
By combining feature weighting and semantic alignment, the segmentation accuracy and universality of visual navigation between agricultural machinery ridges is improved, the problems of model complexity and resource limitation are solved, and more efficient navigation line extraction is achieved.
Patent Information
- Application Number
- CN202410962563.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-12
AI Technical Summary
The existing visual navigation method between agricultural machinery ridges has problems such as high complexity, lack of universality and low segmentation accuracy, and it is difficult to deploy on agricultural machinery equipment with limited computing resources, and it is impossible to accurately segment the ridges of different vegetables, resulting in navigation line errors. big.
Using a visual navigation method between vegetable ridges combining feature weighting and semantic alignment, a cascaded decoding structure with multi-scale feature weighting fusion is designed, and the feature fusion quality is improved through feature weighting modules and stream alignment modules, and the coordinate attention modules are embedded to enhance the features of high-frequency regions. Extract and use a lightweight Mobilenet-v2 network to reduce the amount of model parameters.
The segmentation performance of the model for edge areas is improved, segmentation accuracy and versatility is enhanced, the number of model parameters is reduced, the detection speed is improved, and it is adapted to agricultural machinery equipment with limited hardware resources, providing more accurate navigation line fitting results.
Smart Images

Figure CN118936470A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of automatic navigation of agricultural machinery, and in particular to a method for visual navigation between vegetable ridges combining feature weighting and semantic alignment. Background Art
[0002] In order to improve the efficiency of ridge tillage and reduce the labor intensity of inter-ridge operations, the realization of automatic navigation technology for agricultural machinery between ridges has become an important direction for the development of modern agriculture. At present, the main methods for automatic navigation of agricultural machinery are global navigation satellite system (GNSS), inertial navigation system (INS) and machine vision (MV) navigation system. However, the global satellite navigation system has the disadvantages of poor positioning accuracy and lack of environmental perception function in extreme environments or unstructured agricultural environments. Similarly, due to the drift caused by integration, the inertial navigation system cannot guarantee navigation accuracy during long-term work and cannot effectively monitor the current operation.
[0003] In contrast, machine vision (MV) is a technology that simulates human eyes to measure and judge. It acquires and processes operation information by analyzing images. It has the characteristics of high real-time performance and can well make up for the shortcomings of the above navigation methods. It is being widely used in agricultural automatic navigation. However, limited by the unstructured environment of agriculture, the existing visual detection methods still have the problem of poor robustness and are easily affected by factors such as lighting and interference. In order to solve the defects of mainstream agricultural machinery navigation methods, it is urgent to develop an agricultural machinery ridge navigation method to improve navigation accuracy, reduce the number of algorithm parameters, speed up model inference time and reduce navigation costs.
[0004] At present, there are two main methods for visual navigation between ridges of agricultural machinery. One is the traditional algorithm, which extracts the region of interest from the image based on grayscale, color channel and texture shape. Han Changjie et al. proposed the "Design and Experiment of Cabbage Crop Row Identification and Row Spraying Control System", which extracts color by improving the ExG algorithm, collects feature points based on the vertical projection of the limited threshold in the strip segmented ROI, and obtains the center line of the crop row by linear fitting of the feature points through the least squares method to realize the extraction of navigation line.
[0005] The other type is based on deep learning algorithms. Based on deep learning, the convolutional layer continuously abstracts and learns image features, extracts image features, and classifies and segments the image at the pixel level. Han Zhenhao et al. proposed "Orchard Visual Navigation Path Recognition Method Based on U-Net Network", using the U-net network to train the data set after data enhancement, extract information based on the generated mask map, fit the midpoint, and perform multi-segment cubic B-spline curve fitting on the fitting midpoint based on the spline curve fitting principle to complete path recognition. Compared with traditional image segmentation algorithms, deep learning algorithms have higher accuracy and universality.
[0006] The above algorithms all realize the extraction of navigation lines between ridges of target crops. Although good experimental results have been achieved, the accuracy and practicality of segmentation still need to be improved. Summary of the invention
[0007] The existing navigation line extraction method has the following problems:
[0008] 1) The complexity of the model needs to be reduced. Although complex models can effectively improve the accuracy of image segmentation, the embedded additional modules also significantly increase the resources required for model calculation, reduce the detection speed, and are difficult to deploy on agricultural machinery that is more sensitive to computing resources.
[0009] 2) The model lacks versatility. Most models focus only on the extraction of navigation lines for a single crop, and lack testing for the detection of different crops of the same type. The development of a more generalizable algorithm is obviously more in line with actual application needs.
[0010] 3) Segmentation accuracy needs to be improved. For different types of vegetables, there are obvious differences in ridge height and mulch coverage. The existing method cannot accurately segment the ridge area of different vegetables. The edge information and ridge front information are seriously lost, and the segmentation accuracy is low. The navigation line fitted by this result has a large error, which is not conducive to practical application in agricultural scenarios.
[0011] In order to solve the above problems, the present invention proposes a vegetable ridge visual navigation method combining feature weighting and semantic alignment, designs a cascade decoding structure of multi-scale feature weighted fusion, and adaptively weights the features to adjust the weights during feature fusion; introduces a stream alignment module to alleviate the feature offset problem caused by up and down sampling operations in the semantic segmentation process, and improves the segmentation performance of the model for edge areas; embeds a coordinate attention module in the backbone network to suppress the channel feature response containing invalid or repeated information, and increases the backbone network's extraction of interesting features; replaces the backbone network with Mobilenet-v2 to reduce the number of model parameters and improve model efficiency, so as to better adapt to agricultural machinery with limited hardware resources.
[0012] The technical solution adopted by the present invention is as follows:
[0013] A vegetable ridge visual navigation method combining feature weighting and semantic alignment, comprising:
[0014] Collect vegetable image data, annotate vegetable images, and create training sets and validation sets;
[0015] Constructing a vegetable image segmentation model, setting model hyperparameters and a learning optimizer; the vegetable image segmentation model includes a backbone network based on a coordinate attention mechanism and a cascade decoding structure with multi-scale feature weighting;
[0016] Adjust the model hyperparameters and input the training set images into the vegetable image segmentation model for training;
[0017] Determine whether the vegetable image segmentation model converges and meets the segmentation index requirements. If so, input the test image into the vegetable image segmentation model and output the segmentation result; otherwise, adjust the model hyperparameters and continue training;
[0018] The navigation line is fitted according to the output segmentation results to realize visual navigation between vegetable ridges.
[0019] Furthermore, the multi-scale feature weighted cascade decoding structure includes a feature weighting module and a flow alignment module. The feature weighting module dynamically generates a set of weighting coefficients according to the input feature maps of different sizes and weights the feature maps; the flow alignment module semantically aligns the high-level features with the low-level features, and cascades the weighted feature maps with the high-level features to fully complement the detail information in the low-level features and the semantic information in the high-level features.
[0020] Furthermore, the feature weighting module is configured to perform the following steps:
[0021] For the different levels of features {F1, F2, F3} output by the backbone network, the features {F2, F3} are adjusted to have the same size parameters as the feature F1 through upsampling operations, and the features {F1, F2, F3} are obtained. 22 ,F 33}; A convolution block consisting of 3×3 convolution and 1×1 convolution is used to reshape the features {F1,F 22 ,F 33} Perform regularization operation; concatenate these features in the channel dimension, apply adaptive average pooling operation to the concatenated feature F, and obtain a one-dimensional feature vector M∈R 3c , where c represents the spliced channel;
[0022] For the one-dimensional feature vector M, 1×1 convolution blocks are used continuously to reduce the dimension, and the Softmax function is used to activate the output weight before output; the output weight vector is expressed as {μ1, μ2, μ3}, which represents the weight size when the features at different levels are fused.
[0023] Furthermore, applying an adaptive average pooling operation to the concatenated feature F includes:
[0024]
[0025] Where M(k) is the value of the corresponding feature layer after adaptive average pooling; W is the width dimension of the input feature layer, and H is the height dimension of the input feature layer; F k (i,j) is the value of position (i,j) on the kth channel.
[0026] Further, the flow alignment module is configured to perform the following steps:
[0027] When the lower-level features F of the adjacent level l and high-level features F l-1 After the input stream alignment module, a 1×1 convolution pair is used to align the low-level features F. l With high-level features F l-1 The processed features are concatenated and the semantic flow field Δ is generated through 3×3 convolution. l-1 ;
[0028] The high-level features F l-1 Through the Warp operation, it is upsampled to the low-level feature F l Same resolution; in the Warp operation, the high-resolution feature map is obtained by bilinear interpolation of adjacent pixels in the low-resolution feature map, and the neighborhood of the pixel is defined according to the learned semantic flow field;
[0029] The low-level features F with the same resolution l With high-level features F l-1 The final output is obtained by summing the results.
[0030] Furthermore, the backbone network based on the coordinate attention mechanism is configured to perform the following steps:
[0031] Given an input, each channel is encoded along the horizontal and vertical axes respectively;
[0032] Aggregate the features generated by the encoding in a concatenated manner, and use batch normalization and nonlinear activation to obtain the aggregated features f;
[0033] The feature f is divided into two independent tensors along the spatial dimension, and the original image feature is multiplied by the weight to achieve weighting.
[0034] Furthermore, the backbone network based on the coordinate attention mechanism adopts the Mobilenet-v2 network as the backbone feature extraction network, and the Mobilenet-v2 network adopts deep separable convolution and inverted residual structure to reduce the model parameters and computational complexity.
[0035] Furthermore, fitting the navigation line according to the output segmentation result includes:
[0036] Scan the mask line by line to obtain the non-zero image set of each line;
[0037] Traverse the pixel set, search for the leftmost and rightmost pixels in each row, and calculate the midpoint;
[0038] Select a number of points in the mask height direction at even intervals to form a point set;
[0039] The intercept and slope of the fitting line are calculated using the least squares method, and the fitting navigation line is output.
[0040] Furthermore, the vegetable images are labeled to prepare a training set and a verification set, including:
[0041] The collected vegetable images are semantically annotated using a visual image annotation tool to obtain a JSON file, which is then converted into a grayscale image to make a label. The annotated images are then made into a VOC format dataset, which is divided into a training set and a validation set according to a preset ratio for training and testing.
[0042] Furthermore, the segmentation indicators of the vegetable image segmentation model include average intersection-over-union ratio, average pixel accuracy and floating-point operations; the vegetable image segmentation model uses model inference speed, model memory occupancy and parameter quantity to evaluate the real-time performance of the model.
[0043] The beneficial effects of the present invention are:
[0044] 1. In view of the problems of low segmentation accuracy and poor real-time performance of existing semantic segmentation models for areas such as ridge edges and front ends that have significant impact on navigation line extraction, the present invention designs a cascade decoding structure of multi-scale feature weighted fusion, which complements the detail features and semantic information between low-level features and high-level features in a step-by-step manner through adaptive weighted feature fusion.
[0045] 2. The cascade decoding structure of multi-scale feature weighted fusion performs weighted fusion processing on the segmented image through the feature weighted module, which can improve the feature fusion quality, enhance the high-frequency feature response, and improve the model segmentation ability.
[0046] 3. In view of the problem of inaccurate edge segmentation caused by semantic offset of semantic segmentation features during sampling, the cascade decoding structure of multi-scale feature weighted fusion improves the segmentation accuracy of the algorithm for ridge edges by embedding a stream alignment module. The present invention uses stream alignment to replace the traditional interpolation method for upsampling, which to a certain extent alleviates the problem of semantic information misalignment caused by upsampling, realizes more flexible and dynamic feature registration, and further improves the segmentation performance of the algorithm.
[0047] 4. To address the problem that conventional segmentation algorithms tend to ignore high-frequency areas such as the far ends and edges of ridges, the present invention embeds a coordinate attention mechanism (CA) module after the highest-level feature extraction block of the backbone network to strengthen the network's extraction of information channel features containing high-frequency areas, thereby activating the response of the features in this area.
[0048] 5. The present invention adopts lightweight Mobilenet-v2 as the backbone network to extract features, which can significantly reduce the number of model parameters and improve its detection speed, thereby improving modeling efficiency.
[0049] 6. Based on the segmentation results, the present invention uses the least square method to extract navigation lines, providing support for the automatic navigation of agricultural machinery in the field. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 It is a flow chart of a method for visual navigation between vegetable ridges that combines feature weighting and semantic alignment according to an embodiment of the present invention.
[0051] Figure 2 It is a structural diagram of the vegetable image segmentation model according to an embodiment of the present invention.
[0052] Figure 3 It is a structural diagram of a feature weighting module according to an embodiment of the present invention.
[0053] Figure 4 4 is a structural diagram of a flow alignment module according to an embodiment of the present invention. DETAILED DESCRIPTION
[0054] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, the specific implementation methods of the present invention are now described. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the embodiments described are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.
[0055] Example 1
[0056] like Figure 1 and Figure 2 As shown, this embodiment provides a vegetable ridge visual navigation method combining feature weighting and semantic alignment, including:
[0057] Collect vegetable image data, annotate vegetable images, and create training sets and validation sets;
[0058] Constructing a vegetable image segmentation model, setting model hyperparameters and a learning optimizer; the vegetable image segmentation model includes a backbone network based on a coordinate attention mechanism and a cascade decoding structure with multi-scale feature weighting;
[0059] Adjust the model hyperparameters and input the training set images into the vegetable image segmentation model for training;
[0060] Determine whether the vegetable image segmentation model converges and meets the segmentation index requirements. If so, input the test image into the vegetable image segmentation model and output the segmentation result; otherwise, adjust the model hyperparameters and continue training;
[0061] The navigation line is fitted according to the output segmentation results to realize visual navigation between vegetable ridges.
[0062] Specifically, the vegetable ridge visual navigation method of this embodiment can be implemented by the following steps.
[0063] 1. Dataset Construction
[0064] Collect and process vegetable images, and use a visual image annotation tool (such as Labelme) to annotate the collected vegetable images. In this embodiment, a total of 720 vegetable ridge images were collected during the shooting, including 142 lettuce ridge images, 188 cabbage ridge images, and 390 green vegetable ridge images. After semantic annotation of the vegetable images, a json file is obtained, which is converted into an 8-bit grayscale image to make a label, and the annotated image is made into a VOC format data set. Finally, the data set is divided into a training set and a validation set according to a preset ratio (for example, 8:2) for training and testing, including 576 training sets and 144 validation sets.
[0065] 2. Lightweight backbone network based on coordinate attention mechanism
[0066] Since the original Deeplabv3+ model uses Xception as the backbone network, it has the disadvantage of a large number of parameters. When it is applied to agricultural machinery with limited resources, the long inference time will affect the response speed of the equipment. In order to improve computing efficiency, this embodiment uses Mobilenet-v2 as the backbone feature extraction network. Compared with the Xception backbone network, this network has fewer parameters. Unlike Xception, which uses a linear stacking structure of deep separable convolutions, Mobilenet-v2 mainly uses deep separable convolutions and inverted residual structures, which effectively reduces the model parameters and computational complexity while ensuring better feature extraction capabilities. In addition, this embodiment uses the outputs of the 2nd, 4th, 6th, and 7th Bottleneck in the Mobilenet-v2 network as 4 valid intermediate layer outputs.
[0067] The attention mechanism allows the model to pay more attention to the detail areas in the segmentation task. In order to improve the feature extraction capability of the encoder and enhance the segmentation performance of the model for detail areas such as the edge or front end of the ridge, this embodiment introduces the coordinate attention mechanism. The coordinate attention mechanism is a new type of attention mechanism that pays attention to both feature space information and channel information. It can capture both types of information at the same time to help more accurately locate and identify the target of interest.
[0068] Given an input x∈R C×H×W In the case of , each channel is encoded along the horizontal and vertical coordinates respectively, and the output of the cth channel with a width of w and a height of h can be expressed as
[0069]
[0070] In the formula, is the output in width and height directions; W, H are the width and height of the image.
[0071] The features generated by the encoding are aggregated in a concatenated manner, and batch normalization and nonlinear activation are used to obtain the aggregated features f, that is,
[0072] f=δ(F1([z h ,z w ]))
[0073] Where F1 is the convolution transformation function; δ is the nonlinear activation function; f∈R C / r×(H+W) Then, f is split into two independent tensors along the spatial dimension, namely
[0074] g h =σ(F h (f h ))
[0075] g w =σ(Fw (f w ))
[0076] In the formula, f h ,f w is the component of feature f along the h, w direction; F h ,F w is the convolution transformation function along the h, w direction; δ is the Sigmoid activation function.
[0077] Finally, the original image features are multiplied by the weights to achieve weighting, that is:
[0078]
[0079] In the formula, is the attention weight; x c (i,j) is the original image feature.
[0080] 3. Cascade decoding structure of multi-scale feature weighting
[0081] Low-level features refer to shallow features extracted by the backbone network, which are rich in detail features but lack semantic information. After undergoing multiple convolutional downsampling operations, high-level features have the characteristics of large receptive field and strong response to semantic information, which are very complementary to low-level features. Because the decoder of Deeplabv3+ first performs a 4-fold bilinear upsampling on the input high-level encoding features in a step-by-step manner. In this process, the decoder processing features are spliced with the same-level features output by the backbone network to obtain structural information other than semantic response. However, there may be differences between encoding and decoding features (channel misalignment, semantic offset), and aggregating features in a simple fusion method such as splicing may suppress the semantic activation of high-frequency features and the response of precise structural information.
[0082] In response to the above problems, this embodiment designs a cascade decoding structure of multi-scale feature weighted fusion, which consists of two parts: Feature Weighting Module (FWM) and Flow Align Module (FAM). The feature weighting module dynamically generates a set of weighting coefficients according to the input feature maps of different sizes, and weights the input feature maps. The flow alignment module is mainly used to semantically align high-level features with low-level features. By cascading the weighted feature map with the high-level features, the detail information in the low-level features and the semantic information in the high-level features are fully complemented.
[0083] (1) Feature Weighting Module
[0084] In order to improve the quality of feature fusion and enhance the response of high-frequency features, this embodiment designs a feature weighting module (FWM), which adjusts the fusion weights of different features by dynamically generating a set of weighting coefficients when fusing features of different levels. Its structure is as follows: Figure 3 As shown, the overall workflow is as follows.
[0085] For the different levels of features {F1, F2, F3} output by the backbone network, we first use an upsampling operation to adjust the {F2, F3} features to have the same size parameters as the F1 feature, and obtain {F1, F 22 ,F 33}. A convolution block consisting of 3×3 convolution and 1×1 convolution is used to regularize the reshaped features. These features are concatenated in the channel dimension. Adaptive Average Pooling (AVG) is applied to the concatenated features F to obtain a one-dimensional feature vector M∈R 3c , where c represents the concatenated channel, the operation can be expressed as follows:
[0086]
[0087] Where M(k) is the value of the corresponding feature layer after adaptive average pooling; W is the width dimension of the input feature layer, and H is the height dimension of the input feature layer; F k (i,j) is the value of position (i,j) on the kth channel.
[0088] For the one-dimensional feature vector M, 1×1 convolution blocks are continuously used to reduce its dimension, and the output weight is activated by the Softmax function before output. The output weight vector is expressed as {μ1,μ2,μ3}, which represents the weight size when the features at different levels are fused.
[0089] (2) Stream alignment module
[0090] In the process of feature fusion, it is necessary to use up-sampling and down-sampling operations based on interpolation to align the sizes of features of different scales. Since interpolation sampling will lead to the loss of target edge structure information in the image, there will be a certain semantic misalignment between adjacent feature maps.
[0091] To address this problem, this embodiment uses a flow alignment module (FAM) to fuse adjacent feature maps. The flow alignment module is based on the idea of optical flow and achieves a more flexible and dynamic feature alignment. Its structure is as follows: Figure 4 As shown. The features F of the adjacent levels l and F l-1After the input stream alignment module, they are first processed using 1×1 convolution to align the low-level features F l With the higher level feature F l-1 The processed features are concatenated and a semantic stream is generated through 3×3 convolution. The process can be expressed by the following formula:
[0092] Δ l-1 =conv 3×3 (cat(F l ',F l ' -1 ))
[0093] In the formula, Δ l-1 is the semantic flow field (FlowSemantic); F l '=conv 1×1 (F l );F l ' -1 =up(conv 1×1 (F l-1 ))
[0094] Get the semantic flow field Δ l-1 After that, the high-level feature F l-1 Through the Warp operation, it is upsampled to the low-level feature F l Same resolution. In the Warp operation, the high-resolution feature map is obtained by bilinear interpolation of adjacent pixels in the low-resolution feature map, and the neighborhood of the pixel is defined according to the learned semantic flow field.
[0095] Finally, two features with the same resolution are summed to obtain the final output. The process is expressed as follows:
[0096] output=F l ⊕Warp(Δ l-1 ,F l-1 )
[0097] The flow alignment module aligns the positions between high-level feature maps and low-level feature maps by predicting an offset field that contains high-level and low-level features. This method is conducive to making full use of the rich details in the low-level features and enhancing the response of detailed features such as edges.
[0098] 4. Path Fitting
[0099] According to the segmentation results of the ridges, the navigation lines for visual navigation are extracted. The least squares method finds the best matching function of the data by minimizing the sum of squares of the errors. Since the boundaries of the vegetable ridges are linearly distributed as a whole, the least squares method is used in this embodiment to fit the navigation lines. The fitting steps are as follows:
[0100] 1) Scan the mask line by line to obtain the set of non-zero images of each line.
[0101] 2) Traverse the pixel set, search for the leftmost and rightmost pixels in each row, and calculate the midpoint.
[0102] 3) Select 30 points in the mask height direction at an even interval to form a point set.
[0103] 4) Use the least squares method to calculate the intercept and slope of the fitting line and output the fitting line.
[0104] 5. Algorithm training and testing
[0105] The algorithm training and testing of this embodiment are carried out on a computer equipped with a Window 10 operating system, where the CPU is an Intel i5-12400f, the memory is 32G, the GPU is a GeForce RTX 3080, and the video memory is 12G.
[0106] During the experiment, the training and testing environment of the proposed algorithm and comparative experiments consisted of Python3.8+pytorch2.0.1+cuda11.7, and initial training and testing were performed on the open source framework MMsegmentation1.2.1.
[0107] During the training process, the input image size is 1280×720, and random flipping and random exposure are set. The batch size is set to 2, the number of iterations is set to 30 cycles, the optimizer uses the stochastic gradient descent optimizer (SGD), the optimization momentum is set to 0.9, the initial learning rate is 0.001, and the weight decay is set to 0.0005.
[0108] The mean intersection over union (MIoU), mean pixel accuracy (MPA) and floating point operations (Flops) are used as evaluation indicators of the performance of the semantic segmentation algorithm. The model inference speed (Frames per second, FPS), model memory usage and parameter amount (Params) are used to evaluate the real-time performance of the model.
[0109] VI. Experimental Analysis
[0110] In order to evaluate the performance of the algorithm for vegetable ridge segmentation, this embodiment selects U-net, PSPnet, PSAnet and SegFormer algorithms for comparative experiments on the constructed data set.
[0111] The selected algorithms are trained and tested on the open source framework MMsegmentation1.2.1, and the test results are shown in Table 1. From the results in Table 1, it can be found that the algorithm of this embodiment achieves the best effect.
[0112] Table 1 - Comparison results of semantic segmentation experiments
[0113]
[0114] According to the visualization results of the semantic segmentation comparison experiment, compared with the improved model, the original Deeplabv3+ model has the problem of inaccurate segmentation of ridge edges and front ends, which is specifically manifested in uneven edge segmentation and missed detection of distant areas, and the overall segmentation result is not ideal; SegFormer has significant differences in segmentation performance under different ridge environments, which is specifically manifested in over-prediction of ridge edges in lettuce images with more occlusions. In addition, the PSPnet and PSAnet models will misidentify background interference when processing ridge extraction in green vegetable images, which will affect the extraction of navigation lines based on the least squares method. Compared with the above methods, the improved model of this embodiment achieved the best performance in ridge extraction in different images, the ridge boundary segmentation results are smooth, and the ridge front area segmentation is accurate.
[0115] In order to evaluate the quantitative impact of the segmentation results on the extraction of the navigation line, this embodiment uses the least squares method to extract the corresponding navigation line based on the segmentation results, and uses the average pixel deviation and the average forward angle deviation as evaluation indicators. Among them, the pixel deviation is the pixel offset between the coordinates of the midpoint of the extracted navigation line and the coordinates of the midpoint of the real navigation line; the forward angle deviation is the deviation angle between the extracted navigation line and the real navigation line. In the test, 10 reference points are selected on the extracted navigation line at an average interval to calculate the pixel deviation, and the average value is taken as the calculation standard of the pixel offset.
[0116] The visualization results of navigation line extraction based on different segmentation models show that the pixel deviations of the navigation lines fitted by the model proposed in this embodiment in the ridges of lettuce, cabbage and green vegetables are 2.0, 1.7 and 3.8 pixels respectively, and their forward angle deviations are 0.21°, 0.12° and 1.19° respectively. Therefore, the overall average pixel deviation is 2.5 pixels, and the average forward angle deviation is 0.51°.
[0117] Example 2
[0118] This embodiment is based on embodiment 1:
[0119] This embodiment provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the vegetable ridge visual navigation method combining feature weighting and semantic alignment in embodiment 1 are implemented. The computer program may be in source code form, object code form, executable file, or some intermediate form.
[0120] Example 3
[0121] This embodiment is based on embodiment 1:
[0122] The present embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the vegetable row visual navigation method combining feature weighting and semantic alignment of Embodiment 1. The computer program may be in source code form, object code form, executable file, or some intermediate form. The storage medium includes: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the storage medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the storage medium does not include electric carrier signals and telecommunication signals.
[0123] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
Claims
1. A vegetable ridge visual navigation method combining feature weighting and semantic alignment, characterized in that: include: Collect vegetable image data, annotate vegetable images, and create training sets and validation sets; Constructing a vegetable image segmentation model, setting model hyperparameters and a learning optimizer; the vegetable image segmentation model includes a backbone network based on a coordinate attention mechanism and a cascade decoding structure with multi-scale feature weighting; Adjust the model hyperparameters and input the training set images into the vegetable image segmentation model for training; Determine whether the vegetable image segmentation model has converged and meets the segmentation index requirements. If so, input the test image into the vegetable image segmentation model and output the segmentation result; otherwise, adjust the model hyperparameters and continue training; Navigation lines are fitted according to the output segmentation results to achieve visual navigation between vegetable ridges.
2. A vegetable ridge visual navigation method combining feature weighting and semantic alignment according to claim 1, characterized in that: The multi-scale feature weighted cascade decoding structure includes a feature weighting module and a stream alignment module. The feature weighting module dynamically generates a set of weighting coefficients based on the input feature maps of different sizes and weights the feature maps; the stream alignment module semantically aligns high-level features and low-level features, and by cascading and fusing the weighted feature maps with high-level features, the detail information in the low-level features and the semantic information in the high-level features are fully complementary.
3. The vegetable ridge visual navigation method combining feature weighting and semantic alignment according to claim 2 is characterized in that: The feature weighting module is configured to perform the following steps: For the different level features {F1, F2, F3} output by the backbone network, the features {F2, F3} are adjusted to have the same size parameters as the feature F1 through upsampling operation to obtain the features {F1, F 22 ,F 33 }; A convolution block consisting of 3×3 convolution and 1×1 convolution is used to reshape the features {F1,F 22 ,F 33 } Perform regularization operation; splice these features in the channel dimension, apply adaptive average pooling operation to the spliced feature F, and obtain a one-dimensional feature vector M∈R 3c , where c represents the spliced channel; For the one-dimensional feature vector M, 1×1 convolution blocks are continuously used to reduce the dimension, and the output weight is activated by the Softmax function before output; the output weight vector is expressed as {μ1,μ2,μ3}, which represents the weight size when the features at different levels are fused.
4. The vegetable ridge visual navigation method combining feature weighting and semantic alignment according to claim 3 is characterized in that: The adaptive average pooling operation is applied to the concatenated feature F, including: Where M(k) is the value of the corresponding feature layer after adaptive average pooling; W is the width dimension of the input feature layer, H is the height dimension of the input feature layer; F k (i, j) is the value of position (i, j) on the kth channel.
5. The vegetable ridge visual navigation method combining feature weighting and semantic alignment according to claim 2 is characterized in that: The stream alignment module is configured to perform the following steps: When the low-level features F of the adjacent level l and high-level features F l-1 After the input stream alignment module, 1×1 convolution is used to align the low-level features F l With high-level features F l-1 The processed features are concatenated and the semantic flow field Δ is generated through 3×3 convolution. l-1 ; The high-level features F l-1 Upsampled to the low-level feature F through Warp operation l Same resolution; in the Warp operation, the high-resolution feature map is obtained by bilinear interpolation of adjacent pixels in the low-resolution feature map, and the neighborhood of the pixel is defined according to the learned semantic flow field; The low-level features F with the same resolution l With high-level features F l-1 The final output is obtained by summing the results.
6. The vegetable ridge visual navigation method combining feature weighting and semantic alignment according to claim 1 is characterized in that: The backbone network based on the coordinate attention mechanism is configured to perform the following steps: Given an input, each channel is encoded along the horizontal and vertical axes respectively; Aggregate the encoded features in a concatenated manner, and use batch normalization and nonlinear activation to obtain the aggregated features f; The feature f is divided into two independent tensors along the spatial dimension, and the original image feature is multiplied by the weight to achieve weighting.
7. The vegetable ridge visual navigation method combining feature weighting and semantic alignment according to claim 1 is characterized in that: The backbone network based on the coordinate attention mechanism adopts the Mobilenet-v2 network as the backbone feature extraction network. The Mobilenet-v2 network adopts a depth-separable convolution and inverted residual structure to reduce the number of model parameters and computational complexity.
8. The vegetable ridge visual navigation method combining feature weighting and semantic alignment according to claim 1 is characterized in that: The step of fitting a navigation line according to the output segmentation result includes: Scan the mask line by line to obtain the non-zero image set of each line; Traverse the pixel set, search for the leftmost and rightmost pixels in each row, and calculate the midpoint; Select several points in the mask height direction at even intervals to form a point set; The intercept and slope of the fitting line are calculated using the least squares method, and the fitting navigation line is output.
9. The vegetable ridge visual navigation method combining feature weighting and semantic alignment according to claim 1, characterized in that: The vegetable images are labeled to create a training set and a validation set, including: The collected vegetable images are semantically annotated using a visual image annotation tool to obtain a JSON file, which is then converted into a grayscale image to create labels. The annotated images are then made into a VOC format dataset, which is divided into a training set and a validation set according to a preset ratio for training and testing.
10. The vegetable ridge visual navigation method combining feature weighting and semantic alignment according to claim 1, characterized in that: The segmentation indicators of the vegetable image segmentation model include average intersection-over-union ratio, average pixel accuracy and floating-point operations; the vegetable image segmentation model uses model inference speed, model memory occupancy and parameter quantity to evaluate the real-time performance of the model.
Citation Information
Cited By
An intelligent identification navigation method fusing three-dimensional image information above ridge-ploughed crops
CN122793157A
