A crop row navigation line recognition method and system based on deep learning

Abstract

The application provides a crop row navigation line recognition method and system based on deep learning, and relates to the technical field of computer vision, which comprises the following steps: obtaining crop image data; constructing a crop detection model based on an improved YOLOX-Tiny; inputting the crop image data into the crop detection model for training; obtaining real-time crop image data; inputting the real-time crop image data into the trained crop detection model to output detection frame data of the crop; calculating the center point coordinates of the detection frame according to the detection frame data to determine feature points of the crop row; fitting the feature points by using a third-order polynomial curve to determine the curve equation of the crop row; determining the navigation line slope and the navigation line intercept for tracking the crop row based on the curve equation; and determining the position and direction of the navigation line according to the navigation line slope and the navigation line intercept. The application can accurately position the navigation line of the crop row, and improves the navigation accuracy of the crop row.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a crop row navigation line recognition method and system based on deep learning. BACKGROUND

[0002] The crop row navigation line recognition method based on deep learning is a method that uses deep learning algorithms, especially convolutional neural networks (CNN) and other technologies, to automatically identify and track the paths of crop rows in farmland. This method is commonly used in agricultural automation equipment, such as unmanned agricultural machinery or robots, to improve the efficiency of agricultural production.

[0003] With the development of technology, artificial intelligence technology has made rapid progress, promoting the development of unmanned and intelligent in various industries. To achieve intelligent agriculture, the accuracy and real-time performance of field automatic navigation technology are extremely important. Crop rows, as a landmark reference in the farmland environment, can guide the movement and operation of agricultural robots. An effective seedling strip recognition algorithm can greatly improve the precision of agricultural robot navigation and operation, and can provide spatial constraints for subsequent precision fertilization, precision weeding, and intelligent agricultural machine precision control, thereby achieving the beneficial effects of saving fertilizers and pesticides, improving yield per mu, effectively reducing labor intensity, reducing environmental pollution, reducing production costs, improving resource utilization, and improving agricultural product yield and quality.

[0004] However, traditional crop row recognition methods are usually based on image processing and geometric features, which are easily affected by changes in light and uneven growth of crops, resulting in low accuracy of crop row recognition. Under different light conditions and seasonal changes, the detection effect will significantly decrease. Traditional navigation line fitting methods mostly use linear regression or Hough transform, which can only fit straight lines or approximately straight crop rows, and cannot effectively handle curved or nonlinear crop rows, making it difficult to meet the actual production and operation needs. SUMMARY

[0005] To solve the technical problem that traditional crop row recognition methods are usually based on image processing and geometric features, which are easily affected by changes in light and uneven growth of crops, resulting in low accuracy of crop row recognition, under different light conditions and seasonal changes, the detection effect will significantly decrease, traditional navigation line fitting methods mostly use linear regression or Hough transform, which can only fit straight lines or approximately straight crop rows, and cannot effectively handle curved or nonlinear crop rows, making it difficult to meet the actual production and operation needs, the present application provides a crop row navigation line recognition method and system based on deep learning.

[0006] The technical solutions provided by the embodiments of the present application are as follows:

[0007] First aspect:

[0008] The embodiment of the present application provides a crop row navigation line recognition method based on deep learning, which comprises the following steps:

[0009] S1: acquiring crop image data; S2: constructing a crop detection model based on an improved YOLOX-Tiny, wherein the crop detection model comprises a backbone network module, a NAS-FPN module and a Head module connected in sequence; S3: inputting the crop image data into the crop detection model for training; S4: acquiring real-time crop image data; S5: inputting the real-time crop image data into the trained crop detection model to output detection frame data of crops, wherein the detection frame data comprises a left upper corner coordinate of a detection frame and a right lower corner coordinate of the detection frame; S6: calculating a center point coordinate of the detection frame according to the detection frame data, and determining feature points of a crop row; S7: fitting the feature points by using a third-order polynomial curve to determine a curve equation of the crop row; S8: determining a navigation line slope and a navigation line intercept for tracking the crop row based on the curve equation; and S9: determining a position and a direction of the navigation line according to the navigation line slope and the navigation line intercept.

[0010] The second aspect is:

[0011] The embodiment of the present application provides a crop row navigation line recognition system based on deep learning, which comprises:

[0012] A processor and a memory, wherein the memory stores computer readable instructions, and the computer readable instructions are executed by the processor to realize the crop row navigation line recognition method based on deep learning.

[0013] The third aspect is:

[0014] The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the crop row navigation line recognition method based on deep learning.

[0015] The technical scheme provided by the embodiment of the present application has at least the following beneficial effects:

[0016] In the embodiment of the present application, by constructing a crop detection model based on improved YOLOX-Tiny, the global feature capturing ability of the crop row recognition model in a complex background is enhanced, so that the model can better adapt to different environmental conditions, effectively improve the detection accuracy of the crop row, and through targeted training of the model, the high accuracy of the model in actual detection after training is ensured, the detection frame of the crop row can be accurately output, the accurate recognition of the crop position and range is ensured, the feature points of the crop row are fitted through a three-order polynomial, the nonlinear structure of the crop row can be more accurately described, the problem of curved crop row can be effectively handled, the fitting accuracy and actual adaptability of the navigation line are improved, the slope and intercept of the navigation line are determined through the curve equation, the navigation line of the crop row can be accurately positioned, and the navigation accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 A flowchart of a crop row navigation line recognition method based on deep learning provided by the embodiment of the present application is shown in the figure.

[0019] Figure 2 A crop detection model structure diagram provided by the embodiment of the present application is shown in the figure.

[0020] Figure 3 A crop row navigation line fitting process diagram provided by the embodiment of the present application is shown in the figure.

[0021] Figure 4 A structure diagram of a crop row navigation line recognition system based on deep learning provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0022] The technical solutions in the present application will be described below with reference to the drawings.

[0023] In the embodiment of the present application, the words such as "example", "for example" and the like are used to represent as an example, illustration or description. Any embodiment or design scheme described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is intended to present the concept in a specific way. In addition, in the embodiment of the present application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0024] In the embodiments of the present application, "image" and "picture" can be used interchangeably, and it should be pointed out that their meanings are consistent when their differences are not emphasized.

[0025] In the embodiments of the present application, sometimes the subscript such as W1 can be written in the form of non-subscript such as W1, and their meanings are consistent when their differences are not emphasized.

[0026] To make the technical problems, technical solutions and advantages to be solved by the present application clearer, the following will be described in detail with reference to the drawings and specific embodiments.

[0027] Reference is made to the accompanying drawings and specific embodiments of the present application. Figure 1 , a flowchart of a crop row navigation line recognition method based on deep learning provided by an embodiment of the present application is shown.

[0028] The present application provides a crop row navigation line recognition method based on deep learning, which can be implemented by a crop row navigation line recognition device based on deep learning. The crop row navigation line recognition device based on deep learning can be a terminal or a server. The processing flow of the crop row navigation line recognition method based on deep learning can include the following steps:

[0029] S1: Obtain crop image data.

[0030] In one possible implementation, S1 is specifically:

[0031] Obtain crop image data using a Senyun SG2-IMX390C camera.

[0032] The Senyun SG2-IMX390C is a high-performance industrial camera that uses Sony's IMX390C sensor. It has high resolution, high sensitivity and excellent performance in low light environments, and is widely used in machine vision, automated detection, agriculture and other fields. It can capture high-quality image data.

[0033] It should be noted that using the Senyun SG2-IMX390C camera to obtain crop image data has high resolution and low light performance, and can provide clear images under different lighting conditions. The excellent imaging capability of the Senyun SG2-IMX390C camera ensures the accuracy and richness of the data.

[0034] In the present application, taking corn as an example, at the 3-5 leaf stage of corn, different weather conditions such as sunny and cloudy are selected, and videos are shot at different times of the day (such as 8 am, 12 noon, 6 pm and 8 pm) using the Senyun SG2-IMX390C camera. The camera resolution is 1080P, the frame rate is 30fps, and GMSL2 is used as the interface. The camera is installed at a height of 1.5 meters above the agricultural machinery, and the shooting angle is between 30° and 60°. During shooting, sufficient consideration is given to factors such as leaf shading, straw residue coverage, plant gap, weed interference, corn close planting, and night light supplementation to obtain a wide variety of image data. The shot video is converted into pictures frame by frame, and the original images are processed for color, brightness, contrast, noise, rotation and mirroring, etc. Then, the pre-processed images are labeled using the professional image labeling software Labelme, with corn seedlings as the labeling object, and their positions and ranges are marked. After labeling is completed, the dataset is divided into training set, validation set and test set according to the ratio of 8:1:1 for subsequent model training and evaluation.

[0035] In a possible implementation, after S1, further comprising:

[0036] Preprocessing the crop image data.

[0037] The preprocessing specifically includes color processing, brightness adjustment, contrast adjustment, noise processing, rotation processing, and mirroring processing.

[0038] The color processing refers to adjusting the color information of the image, including changing the color saturation, hue or color temperature, in order to better present the appearance of the crop. The brightness adjustment refers to increasing or decreasing the brightness of the image to improve the display effect of the image under different lighting conditions and ensure that the details are visible. The contrast adjustment refers to adjusting the difference between light and dark parts in the image to make the details of the image more prominent and increase the visual impact of the image. The noise processing refers to removing noise or unnecessary interference in the image to improve the image quality and ensure the accuracy of subsequent processing and analysis. The rotation processing refers to rotating the image to adjust the angle or direction of the image to meet the analysis requirements. The mirroring processing refers to horizontally or vertically flipping the image to help generate data at different angles and enhance the diversity of the training data.

[0039] It should be noted that preprocessing the crop image data can effectively improve the data quality. Through the adjustment of color, brightness and contrast, the image quality problems under different lighting conditions can be addressed, and the noise processing can remove unnecessary interference. The rotation and mirroring processing increase the diversity of the image, thereby improving the generalization ability of the deep learning model.

[0040] Reference is made to the attached drawings Figure 2Fig. 1 shows a structure diagram of a crop detection model according to an embodiment of the present application.

[0041] As shown in Fig. 1, the structure of the crop detection model mainly includes three parts: a backbone network, a NAS-FPN network, and a head. The backbone network is composed of Fous units, CSP1_1 units, multiple CBS units, multiple CSP1_3 units, MobileViT units, CSP2_1 units, SPPF units, and Attention Block units, and is responsible for extracting multi-scale features from input images. The NAS-FPN network is used to generate feature maps of different scales and pass them to the classification and bounding box sub-network to predict the category and location of crops. The head processes the extracted features and outputs the detection results of crops, such as category and bounding box. Through this structure, the model can accurately detect crops in complex agricultural scenes. Figure 2

[0042] S2: Construct a crop detection model based on the improved YOLOX-Tiny, wherein the crop detection model includes a backbone network module, a NAS-FPN module, and a Head module connected in sequence.

[0043] YOLOX-Tiny is a lightweight version of the YOLO series designed for real-time object detection tasks, with small computational complexity and fast inference speed. By improving the structure of the YOLO series, it can achieve high detection accuracy under low computational resources. The backbone network module (Backbone) is the core part of the deep learning model, responsible for extracting the basic features of the input image. The NAS-FPN module (Neural Architecture Search Feature Pyramid Network) is a feature pyramid network optimized by neural architecture search technology, used to effectively extract and fuse image features at multiple scales, improving the accuracy of object detection. The Head module is the last part of the detection network, responsible for classification, regression (bounding box prediction, etc.) based on the extracted features, and outputting the final results of object detection.

[0044] It should be noted that by constructing a crop detection model based on the improved YOLOX-Tiny, combined with the efficient backbone network, NAS-FPN, and Head module, efficient crop detection can be achieved under limited computational resources. The backbone network module extracts image features, the NAS-FPN module enhances the fusion ability of multi-scale features, and the Head module accurately outputs the category and location of crops, enabling the model to quickly infer with high precision.

[0045] ​In one possible implementation, the backbone network module includes, in sequence, a Fous unit, a first CBS unit, a CSP1_1 unit, a second CBS unit, a first CSP1_3 unit, a MobileViT unit, a third CBS unit, a second CSP1_3 unit, a fourth CBS unit, a CSP2_1 unit, an SPPF unit, and an Attention Block unit.

[0046] The SPPF unit includes a 5x5 max pooling operation subunit, a 9x9 max pooling operation subunit, a 13x13 max pooling operation subunit, and a basic convolution operation subunit.

[0047] Specifically, the Fous unit refers to a specific designed convolution unit, which may be responsible for feature extraction in the network. The CBS (Convolution, Batch Normalization, and Activation) unit refers to a common convolution layer structure, which includes convolution operation, batch normalization (Batch Normalization), and activation function (such as ReLU). The CSP (Cross-Stage Partial) unit is a module for reducing computational complexity, which can improve computational efficiency while maintaining performance by dividing the feature map into multiple parts, processing and merging them respectively. The MobileViT unit is a lightweight module based on VisionTransformer (ViT) design, which combines the advantages of convolutional neural network (CNN) and transformer model, and can process more complex image features at low computational cost. The SPPF (Spatial Pyramid Pooling and Fusion) unit is used to extract information of different scales, which can process input images of different sizes and fuse them, improving the model's ability to recognize multi-scale targets. The Attention Block unit uses attention mechanism to help the model automatically focus on important parts of the image. Through this mechanism, the model can more effectively identify and process valuable feature information. The max pooling operation subunit refers to the max pooling operation applied to the image, which extracts feature information of different scales from the image by using windows of different sizes (5x5, 9x9, 13x13). Pooling operation helps to reduce dimensionality, reduce computational complexity, and retain important features. The basic convolution operation subunit refers to the basic operation in convolutional neural network, which is used to extract local features of input image. Convolution operation uses filters (convolution kernels) to slide on the input image and calculates feature responses.

[0048] In the present application, after the first CSP1_3 module and the second CSP1_3 module, part of the convolutional layers is replaced by the lightweight Transformer of MobileViT. The MobileViT unit first divides the input feature map into multiple small patches, performs convolution operation in each patch to extract local features. Then, the features of these patches are sent to the Transformer unit to capture global dependencies using the self-attention mechanism. Finally, the feature map size is restored through convolution operation to match the channel number of the subsequent part of the original network. In this way, without significantly increasing the computational amount, the model's ability to capture global features of crop rows is enhanced, and the detection accuracy in complex backgrounds is improved.

[0049] For example, assuming that the input channel number of the original convolutional layer is C in , and the output channel number is C out , when it is replaced by the MobileViT unit, first ensure that the input channel number of the MobileViT unit is consistent with C in . Inside the MobileViT unit, the input feature map is divided into blocks, with a block size of P x P, and convolution operation is performed in each patch, with a convolution kernel size of K x K and a step size of . After the intra-patch convolution, the features are sent to the Transformer, with the number of heads in the Transformer set to H and the dimension of the attention head set to D. After the Transformer processing, the feature map size is restored through convolution operation to make the output channel number consistent with C out , ensuring smooth connection with the subsequent part of the original network.

[0050] In the application, a fast spatial pyramid pooling (SPPF) unit is added at the end of the backbone network to expand the receptive field and improve the target positioning accuracy. The fast spatial pyramid pooling (SPPF) unit is added at the end of the backbone network, which is composed of maximum pooling kernel units with kernel sizes of 5x5, 9x9 and 13x13, and basic convolution, which can expand the receptive field without significantly increasing the model size, enhance the expression ability of the feature map, and improve the target positioning accuracy. In order to improve the multi-scale feature fusion effect, the related structure of generating P2 feature map based on FPN concept in the original model is abandoned, and NAS-FPN module is adopted. Different scale feature maps output by the backbone network are directly input into the NAS-FPN module, and the NAS-FPN module automatically fuses and processes different scale features through the optimal structure obtained by neural architecture search. Before inputting into the NAS-FPN module, the size and channel number of the feature map are adjusted to meet the input requirements of the NAS-FPN module. After processing by the NAS-FPN module, the output multi-scale feature map is directly connected to the subsequent prediction head for crop row detection. This improved structure can more effectively fuse features of different scales and improve the detection ability of crop rows of different sizes.

[0051] For example, the P3, P4 and P5 feature maps output by the backbone network usually do not match the input requirements of the subsequent NAS-FPN module in terms of size and channel number, and therefore need to be adjusted accordingly. For the P3 feature map, size adjustment is usually achieved through convolution operation, such as using a convolution layer with a 3x3 convolution kernel. If the size needs to be reduced, the step size is set to be greater than 1, such as 2, so that the height and width of the feature map are reduced to half of the original size. If the size needs to be kept unchanged, the step size is set to 1 and the padding is set to 1. For channel number adjustment, 1x1 convolution is used. According to the input requirements of the NAS-FPN module, if the channel number is expected to be less than the original channel number of the P3 feature map, channel compression is performed by setting a smaller number of 1x1 convolution kernels. Otherwise, the number of convolution kernels is increased to expand the channel number. Similar processing is performed on the P4 and P5 feature maps. According to their respective original sizes and channel numbers, as well as the input requirements of the NAS-FPN, appropriate convolution operations and parameters are selected for adjustment. The P4 feature map is first reduced in size using a convolution layer with a step size of 2 and a 3x3 convolution kernel, and then the channel number is adjusted using a 1x1 convolution layer. The P5 feature map itself has a small size, and when adjusting, more emphasis is placed on optimizing the channel number. The size can be kept stable by padding and convolution operation, and then the channel number is accurately adjusted using a 1x1 convolution. After the adjusted P3, P4 and P5 feature maps are input into the NAS-FPN module, the NAS-FPN will use the optimal structure obtained through neural architecture search to efficiently fuse and process these feature maps, intelligently integrate information in feature maps of different scales, and enhance the feature expression of crop rows of different sizes. Finally, the NAS-FPN module outputs optimized multi-scale feature maps, which are directly connected to the subsequent prediction head. Based on these rich and well-integrated features, the prediction head detects crop rows, accurately identifies the position, size and other information of crop rows, and provides accurate data support for subsequent navigation line extraction and fitting, effectively improving the performance of the entire crop row detection system.

[0052] S3: inputting the crop image data into the crop detection model for training.

[0053] It should be noted that inputting the crop image data into the crop detection model for training can enable the model to gradually learn how to identify the features of crops through a large amount of labeled data. During the training process, the model will continuously optimize parameters and improve the ability to identify different types of crops and crops in different environments. This process can enable the model to more accurately and effectively detect crops in actual applications, and has strong adaptability and generalization ability.

[0054] In one possible implementation, S3 specifically includes:

[0055] The preprocessed crop image data is input into the crop detection model for training until the value of the loss function is less than the preset loss function value.

[0056] It should be noted that by inputting the processed crop image data into the crop detection model for training and continuously adjusting the model parameters until the loss function value reaches the preset target, the accuracy of the model can be continuously improved. The loss function is used to measure the difference between the model prediction value and the actual value, and by continuously optimizing the model, the error can be effectively reduced, so that the model can accurately identify crops in actual application.

[0057] The loss function specifically includes: an EIOU loss function, an IoU-aware classification loss function, an IoU-aware category loss function, a cross-entropy loss function, and a regularized bounding box loss function.

[0058] It should be noted that the size of the preset loss function value can be set by a person skilled in the art according to the actual situation, and the present application does not limit it.

[0059] S4: Obtain real-time crop image data.

[0060] The EIOU (Enhanced Intersection over Union) loss function is a loss function used for target detection, aiming to improve the overlap between the predicted frame and the real frame. The IoU-aware classification loss function (IoU-aware Classification Loss Function) is a classification loss function that combines IoU (Intersection over Union) information, aiming to optimize the classification task of the detection model. The IoU-aware category loss function (IoU-aware Category Loss Function) considers the influence of IoU information on category prediction, by incorporating IoU into the calculation of category prediction, to improve the accuracy of category prediction, especially in small object detection and precise positioning tasks. The cross-entropy loss function (Cross-Entropy Loss Function) is commonly used in classification problems, measuring the difference between the predicted category and the actual category. For each sample, the cross-entropy calculates the difference between the predicted probability distribution and the true distribution, the goal is to minimize the loss value, thereby optimizing the classification performance of the model. The regularized bounding box loss function (Regularized Bounding Box Loss Function) is used for the bounding box regression task in target detection, aiming to reduce the difference between the model's predicted bounding box and the real bounding box. The purpose of regularization is to constrain the output of the model to avoid overfitting and ensure the accuracy of the bounding box.

[0061] In the present application, the EIOU loss function is used instead of the original loss function, and at the same time, the IoU-aware classification loss and the IoU-aware category loss are combined, and the cross-entropy loss function is used for calculation, and the regularization bounding box loss function is added in the later training stage.

[0062] S5: inputting the real-time crop image data into the trained crop detection model, and outputting the detection frame data of the crop, wherein the detection frame data comprises the upper left corner coordinate of the detection frame and the lower right corner coordinate of the detection frame.

[0063] It should be noted that inputting the real-time crop image data into the trained crop detection model can obtain the position and size information of the crop in the image in real time, accurately output the detection frame data, timely identify the changes of the crop and position, and greatly improve the efficiency and accuracy of agricultural operation.

[0064] S6: calculating the center point coordinates of the detection frame according to the detection frame data, and determining the feature points of the crop row.

[0065] The crop row refers to the row and column structure formed by crops arranged according to certain rules in agricultural planting, which usually has consistent spacing and arrangement, which can improve planting efficiency and facilitate management and mechanized operation, such as irrigation, fertilization, weeding and harvesting, etc. In modern agriculture, the identification and management of crop rows become particularly important, especially when using automated equipment for precision agriculture, accurately identifying the position of the crop row is the key to ensuring crop growth and improving production efficiency.

[0066] Specifically, by calculating the center point coordinates of the detection frame, the center position of each crop can be accurately determined, and the feature points of the crop row can be extracted from multiple detection frames, improving the accuracy of crop row positioning and enabling more stable capture of the position of the crop row.

[0067] Reference is made to the accompanying drawings Figure 3 , which shows a crop row navigation line fitting process diagram provided by an embodiment of the present application.

[0068] As shown in Figure 3 , the fitting process of the crop row navigation line is shown. First, the crop detection model identifies the crop area in the image and labels the detection frame, then calculates the center point coordinates of each detection frame to obtain the feature points of the crop row, and then fits these feature points by a cubic polynomial curve to generate a fitting curve equation of the crop row, and further calculates the slope and intercept of the navigation line based on the curve equation to determine the specific position and direction of the crop row, and finally draws the navigation line of the crop row. This process helps agricultural machinery accurately track the crop row and perform automated operation.

[0069] S7: Adopting a third-order polynomial curve, fitting the feature points, and determining the curve equation of the crop row.

[0070] wherein the third-order polynomial curve is a mathematical expression used to fit a set of data points, capable of flexibly describing the nonlinear relationship of the data.

[0071] It should be noted that adopting a third-order polynomial curve to fit the feature points can accurately capture the trend of the crop row and generate an accurate curve equation, effectively reducing the fitting error, providing a stable navigation path, and ensuring that the agricultural machinery accurately tracks the crop row, improving the efficiency of crop management and harvesting.

[0072] In one possible implementation, the curve equation is specifically:

[0073]

[0074] wherein l i represents the fitting curve equation of the ith crop row, x represents the feature point coordinates, w 0,i , w 1,i , w 2,i , and w 3,i all represent polynomial coefficients.

[0075] S8: Based on the curve equation, determine the slope and intercept of the navigation line for tracking the crop row.

[0076] wherein the slope of the navigation line is the inclination of the navigation line relative to the horizontal line, usually represented as the ratio of the vertical change to the horizontal change of the line segment, and the slope determines the direction of the crop row. The intercept of the navigation line refers to the coordinates of the intersection of the line and the longitudinal axis (usually the y-axis), indicating the starting position of the crop row.

[0077] It should be noted that based on the curve equation, the slope and intercept of the navigation line can accurately describe the path direction and position of the crop row, providing key parameters of the navigation line, helping the agricultural equipment accurately track the crop row, thereby improving the precision and efficiency of the operation, ensuring that the management and harvesting of crops are more efficient and accurate.

[0078] In one possible implementation, S8 specifically includes:

[0079] S801: Simultaneously considering the curve equations of adjacent crop rows, determine the intersection coordinates of the curve equations.

[0080] S802: According to the intersection coordinates, determine the slope and intercept of the navigation line.

[0081] In one possible implementation, the calculation method of the slope of the navigation line is specifically:

[0082]

[0083] wherein, m c represents the slope of the navigation line, b c represents the slope of the navigation line, x i represents the intersection horizontal coordinate of the curve equation, y i represents the intersection vertical coordinate of the curve equation.

[0084] The calculation method of the navigation line intercept is specifically:

[0085]

[0086] wherein, tan represents the tangent function, arctan represents the inverse tangent function, m1 represents the slope of the first crop row curve, and m2 represents the slope of the second crop row curve.

[0087] S9: Determine the position and direction of the navigation line according to the slope and intercept of the navigation line.

[0088] It should be noted that the position and direction of the navigation line are determined according to the slope and intercept of the navigation line, which can accurately calculate the accurate path of the crop row, so that the agricultural machinery can accurately locate the trajectory of the crop row and efficiently run along the predetermined path, avoid deviating from the target, ensure the accuracy and efficiency of the crop management and harvesting process, and greatly improve the accuracy and stability of the automatic agricultural operation.

[0089] The technical scheme provided by the embodiment of the application has at least the following beneficial effects:

[0090] In the embodiment of the application, by constructing a crop detection model based on the improved YOLOX-Tiny, the global feature capturing ability of the crop row recognition model in a complex background is enhanced, so that the model can better adapt to different environmental conditions, effectively improve the detection accuracy of the crop row, and through targeted training of the model, the high accuracy of the model in actual detection after training is ensured, the detection frame of the crop row can be accurately output, the accurate identification of the crop position and range is ensured, the feature points of the crop row are fitted through a three-order polynomial, the nonlinear structure of the crop row can be more accurately described, the problem of curved crop row can be effectively handled, the fitting accuracy and actual adaptability of the navigation line are improved, the slope and intercept of the navigation line are determined through the curve equation, the navigation line of the crop row can be accurately positioned, and the navigation accuracy is improved.

[0091] Reference is made to the accompanying drawings in the description Figure 4 , which shows a structure schematic diagram of a crop row navigation line recognition system based on deep learning provided by the application.

[0092] The application further provides a crop row navigation line recognition system 20 based on deep learning, which is applied to the crop row navigation line recognition method based on deep learning and comprises:

[0093] The processor 201.

[0094] The memory 202 stores computer readable instructions, which, when executed by the processor 201, implement the deep learning-based crop row navigation line identification method according to the method embodiment.

[0095] The deep learning-based crop row navigation line identification system 20 provided by the present application can execute the deep learning-based crop row navigation line identification method described above and achieve the same or similar technical effects. To avoid repetition, the present application will not be described again.

[0096] The technical scheme provided by the embodiment of the present application has at least the following beneficial effects:

[0097] In the embodiment of the present application, by constructing the crop detection model based on the improved YOLOX-Tiny, the global feature capturing ability of the crop row identification model in a complex background is enhanced, so that the model can better adapt to different environmental conditions, effectively improve the detection accuracy of the crop row, and through targeted training of the model, the high accuracy of the model in actual detection after training is ensured, the detection frame of the crop row can be accurately output, the accurate identification of the crop position and range is ensured, the feature points of the crop row are fitted through a three-order polynomial, which can more accurately describe the nonlinear structure of the crop row, and can effectively solve the problem of curved crop row, improve the fitting accuracy and actual adaptability of the navigation line, and determine the slope and intercept of the navigation line through the curve equation, so that the navigation line of the crop row can be accurately positioned, and the navigation accuracy is improved.

[0098] It should be understood that the processor in the embodiment of the present application can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), ready-to-program gate arrays (FPGAs), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0099] It should also be understood that the memory in the embodiments of the present application can be volatile or nonvolatile memory, or can include both volatile and nonvolatile memory. Nonvolatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as external cache. By way of example, and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0100] The above-described embodiments can be implemented in whole or in part by software, hardware (such as a circuit), firmware, or any combination thereof. When implemented in software, the above-described embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions according to the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through a wired (for example, infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0101] It should be understood that the term "and / or" herein merely describes an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone, where A and B can be singular or plural. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after it, but it can also represent an "and / or" relationship, which can be understood according to the context before and after it.

[0102] In the present application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0103] It should be understood that in various embodiments of the present application, the size of the sequence number of the above-described processes does not mean the order of execution, and the execution order of the processes should be determined according to their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0104] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0105] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the devices, apparatuses and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0106] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0107] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0108] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0109] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0110] The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the deep learning based crop row navigation line identification method of the method embodiment.

[0111] The computer readable storage medium provided by the present application can realize the steps and effects of the deep learning based crop row navigation line identification method of the above method embodiment. To avoid repetition, the present application will not be described again.

[0112] The technical solutions provided by the embodiments of the present application have at least the following beneficial effects:

[0113] In the embodiments of the present application, by constructing the crop detection model based on the improved YOLOX-Tiny, the global feature capturing ability of the crop row identification model in complex background is enhanced, so that the model can better adapt to different environmental conditions, effectively improve the detection accuracy of the crop row, and through targeted training of the model, the high accuracy of the model in actual detection after training is ensured, the detection frame of the crop row can be accurately output, the accurate identification of the crop position and range is ensured, the feature points of the crop row are fitted through the third order polynomial, the nonlinear structure of the crop row can be more accurately described, the problem of curved crop row can be effectively handled, the fitting accuracy and actual adaptability of the navigation line are improved, the slope and intercept of the navigation line are determined through the curve equation, the navigation line of the crop row can be accurately positioned, and the navigation accuracy is improved.

[0114] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0115] The following points need to be explained:

[0116] (1) The drawings of the embodiments of the present application only relate to the structures involved in the embodiments of the present application, and other structures can be referred to the general design.

[0117] (2) In the drawings used to describe the embodiments of the present application, the thickness of a layer or region is exaggerated or reduced for clarity, i.e., the drawings are not drawn according to the actual scale. It can be understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, it can be "directly" on or under the other element or there can be an intermediate element.

[0118] (3) In the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other to obtain new embodiments.

[0119] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for crop row navigation line recognition based on deep learning, characterized in that, The method comprises the following steps: S1: acquiring crop image data; S2: constructing a crop detection model based on an improved YOLOX-Tiny, wherein the crop detection model comprises a backbone network module, a NAS-FPN module and a Head module connected in sequence; S3: inputting the crop image data into the crop detection model for training; S4: acquiring real-time crop image data; S5: inputting the real-time crop image data into the trained crop detection model to output detection frame data of crops, wherein the detection frame data comprises a left upper corner coordinate of a detection frame and a right lower corner coordinate of the detection frame; S6: calculating a center point coordinate of a detection frame according to the detection frame data to determine feature points of a crop row; S7: fitting the feature points by using a third-order polynomial curve to determine a curve equation of the crop row; S8: determining a navigation line slope and a navigation line intercept for tracking the crop row based on the curve equation; S9: determining a position and a direction of a navigation line according to the navigation line slope and the navigation line intercept. The backbone network module comprises Fous units, a first CBS unit, a CSP1_1 unit, a second CBS unit, a first CSP1_3 unit, a MobileViT unit, a third CBS unit, a second CSP1_3 unit, a fourth CBS unit, a CSP2_1 unit, an SPPF unit and an Attention Block unit connected in sequence. The SPPF unit comprises a 5*5 max pooling operation subunit, a 9*9 max pooling operation subunit, a 13*13 max pooling operation subunit and a basic convolution operation subunit. The S8 specifically comprises: S801: determining an intersection coordinate of the curve equation by simultaneously solving the curve equations of adjacent crop rows; S802: determining the navigation line slope and the navigation line intercept according to the intersection coordinate. 2.The crop row navigation line recognition method based on deep learning according to claim 1, characterized in that, The S1 specifically comprises: The crop image data is acquired by using a Sencloud SG2-IMX390C camera. 3.The crop row navigation line recognition method based on deep learning according to claim 1, characterized in that, After the S1, the following steps are further included: preprocessing the crop image data; The preprocessing specifically comprises color processing, brightness adjustment, contrast adjustment, noise processing, rotation processing and mirror image processing. 4.The crop row navigation line identification method based on deep learning according to claim 1, wherein, The S3 specifically comprises: The crop image data is input into the crop detection model for training until a value of a loss function is less than a preset loss function value; The loss function specifically comprises an EIOU loss function, an IoU-aware classification loss function, an IoU-aware category loss function, a cross-entropy loss function and a regularized bounding box loss function. 5.The crop row navigation line identification method based on deep learning according to claim 1, wherein, The curve equation specifically comprises: ； wherein, l i represents a curve equation of the ith crop row, x represents a feature point coordinate, w 0,i , w 1,i , w 2,i , and w 3,i all represent polynomial coefficients. 6.The crop row navigation line identification method based on deep learning according to claim 1, wherein, The calculation method of the navigation line slope specifically comprises: ； where m c represents the slope of the navigation line c, b c represents the slope of the navigation line c, x i represents the intersection horizontal coordinate of the curve equation, y i represents the intersection vertical coordinate of the curve equation; The calculation method of the navigation line intercept specifically comprises: ； wherein tan represents a tangent function, arctan represents an inverse tangent function, m1 represents a first crop row curve slope, and m2 represents a second crop row curve slope. 7.A crop row navigation line recognition system based on deep learning, characterized by The method comprises the following steps: a processor; A memory, the memory has computer readable instructions stored thereon, the computer readable instructions, when executed by the processor, implement the deep learning-based crop row navigation line identification method according to any one of claims 1 to 6.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the deep learning-based crop row navigation line identification method according to any one of claims 1 to 6.

Images