Navigation method and device of working vehicle, electronic equipment and medium

Abstract

The invention provides a navigation method and device of an operation vehicle, electronic equipment and a medium, and the method comprises the steps: capturing a current frame of seedling strip image in the driving direction of the operation vehicle through multi-mode sensing equipment deployed at the front center position of the operation vehicle; inputting the current frame of the seedling strip image into the identification model to obtain anchor point coordinates of a plurality of anchor points for representing the position of the seedling strip; according to the anchor point coordinates of the multiple anchor points, fitting classification is conducted on the multiple anchor points to obtain multiple point sets, each point set comprises two point sets, each point set is composed of points on a straight line obtained through fitting in the driving direction in the seedling belt, and the two point sets in each point set are in bilateral symmetry with the multi-modal sensing device as the center; and generating a navigation line according to the anchor points in the target point set in the plurality of point sets, so that the operation vehicle operates along the navigation line. According to the invention, high-precision prediction of the seedling strip navigation line is realized.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a navigation method, device, electronic equipment and medium for a working vehicle. Background Art

[0002] Population growth and climate change have prompted agriculture to improve efficiency and reduce resource consumption. Advances in deep learning have greatly improved crop detection capabilities. As the core of precision agriculture, agricultural machinery automatic navigation technology is crucial in tillage and other links. It uses visual sensors to obtain crop information and process it, calculate navigation lines, and achieve autonomous navigation, which significantly improves operating efficiency, reduces resource waste and costs, and promotes the development of agriculture towards intelligence and efficiency.

[0003] Although the method of calculating navigation lines through deep learning has potential, it faces the problem of large data requirements. High-quality labeled data in the agricultural field is scarce, and semantic segmentation requires processing a large amount of image information, and the amount of calculation increases dramatically at high resolution. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a navigation method, device, electronic equipment and medium for a work vehicle, aiming to achieve high-precision prediction of the navigation line of the seedling belt.

[0005] In a first aspect, the present application provides a navigation method for a work vehicle, the method comprising: capturing a current frame seedling belt image in the driving direction of the work vehicle by a multimodal sensing device deployed at a front center position of the work vehicle; inputting the current frame seedling belt image into a recognition model to obtain anchor point coordinates of multiple anchor points used to characterize the seedling belt position; fitting and classifying the multiple anchor points according to the anchor point coordinates of the multiple anchor points to obtain multiple point sets, each point set including two point sets, each point set consisting of points on a straight line fitted in the seedling belt along the driving direction, the two point sets in each point set being symmetrical with the multimodal sensing device as the center; generating a navigation line according to the anchor points in a target point set in the multiple point sets, so that the work vehicle operates along the navigation line.

[0006] In a possible implementation, the target point set is determined in the following manner: for each point set, determine whether the number of anchor point coordinates in the point set is greater than a preset number; if there is no point set with a number of anchor point coordinates greater than the preset number, determine the target point set based on the sequence of the seedling strip image of the current frame; if there is a point set with a number of anchor point coordinates greater than the preset number, determine the point set as the target point set; if there are at least two point sets with a number of anchor point coordinates greater than the preset number, combine the at least two point sets, and determine the combined point set as the target point set.

[0007] In a possible implementation, the target point set corresponding to the current frame seedling band image is determined according to the sequence of the current frame seedling band image in the following manner: determining whether the sequence of the current frame seedling band image is the first frame; if not, determining the target point set of the previous frame seedling band image as the target point set of the current frame seedling band image; if so, determining that the target point set of the current frame seedling band image is empty.

[0008] In one possible implementation, a navigation line is generated by fitting the anchor point coordinates of multiple anchor points in a target point set to obtain a center line; calculating the slope of the center line, and constructing a center line equation based on the slope to generate the navigation line based on the center line equation.

[0009] In a possible implementation, the recognition model is constructed in the following manner: obtaining a seedling belt sample image; annotating the seedling belt sample image to obtain a real anchor point for characterizing the seedling belt position; and representing the relationship between the seedling belt sample image and the real anchor point using the recognition model.

[0010] In a possible implementation, the recognition model is trained in the following manner: the seedling strip sample image is input into the recognition model to obtain a predicted anchor point; the difference between the real anchor point of the seedling strip sample image and the predicted anchor point is calculated to obtain a loss function of the seedling strip sample image; and the parameters of the recognition model and the weight of the loss function are updated by iterating the recognition model until the result of the loss function is less than a preset standard.

[0011] In a possible implementation, the loss function is obtained according to classification loss, continuity loss, position loss and segmentation loss. The classification loss is used to characterize the difference in category between the predicted anchor point and the true anchor point. The continuity loss is used to characterize the degree of stable association between the predicted anchor point between consecutive frames. The position loss is used to characterize the prediction accuracy of the recognition model for the position of the seedling band. The segmentation loss is used to characterize the classification accuracy of the recognition model for the seedling band and the background in the seedling band image of the current frame.

[0012] In a second aspect, the present application provides a navigation device for a work vehicle, the device comprising: a capture module, which captures a current frame seedling belt image in the driving direction of the work vehicle through a multimodal sensing device deployed at the front center position of the work vehicle; an input module, which is used to input the current frame seedling belt image into a recognition model to obtain anchor point coordinates of multiple anchor points used to characterize the seedling belt position; a classification module, which is used to fit and classify the multiple anchor points according to the anchor point coordinates of the multiple anchor points to obtain multiple point sets, each point set includes two point sets, each point set represents a straight line point group fitted in the seedling belt along the driving direction, and the two point sets in each point set are symmetrical with the multimodal sensing device as the center; a generation module, which is used to generate a navigation line according to the anchor points in the target point set in the multiple point sets, so that the work vehicle can operate along the navigation line.

[0013] In a third aspect, the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the above method are performed.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and the computer program executes the steps of the above method when executed by a processor.

[0015] The present application provides a navigation method, device, electronic device and medium for a working vehicle, wherein the method comprises: capturing a current frame seedling belt image in the driving direction of the working vehicle by a multimodal sensing device deployed at the front center position of the working vehicle; inputting the current frame seedling belt image into a recognition model to obtain the anchor point coordinates of multiple anchor points used to characterize the position of the seedling belt; fitting and classifying the multiple anchor points according to the anchor point coordinates of the multiple anchor points to obtain multiple point sets, each of which includes two point sets, each of which characterizes a straight line point group fitted in the driving direction of the seedling belt, and the two point sets in each point set are symmetrical with the multimodal sensing device as the center; generating a navigation line according to the anchor points in the target point set in the multiple point sets, so that the working vehicle can operate along the navigation line. Through the present application, a high-precision prediction of the seedling belt navigation line is achieved.

[0016] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flowchart of a navigation method for a working vehicle provided in an embodiment of the present application;

[0019] Figure 2 A flowchart of a training recognition model provided in an embodiment of the present application;

[0020] Figure 3 A schematic structural diagram of a navigation device for a working vehicle provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0023] First, the application scenarios to which the present application is applicable are introduced. The present application can be applied to image processing technology.

[0024] Research has found that existing seedling strip detection technologies, whether traditional image processing or deep learning methods, face challenges in agricultural visual navigation. Traditional methods are sensitive to environmental changes, rely on variable features such as color and texture, and are easily affected by light, season, and crop type, resulting in unstable detection results. In complex scenes such as shadows and weed cover, traditional image recognition methods may fail completely, and consistency is difficult to ensure when applied across regions and seasons, requiring frequent parameter adjustments, which limits the generalization ability of the algorithm. In addition, traditional algorithms require large computing resources, which is not conducive to embedded device applications. Although deep learning methods have potential, they face the problem of large data volume requirements. High-quality labeled data is scarce in the agricultural field, and semantic segmentation requires processing a large amount of image information, which increases the amount of calculations at high resolution. Some methods that use target detection have uneven detection accuracy due to labeling problems.

[0025] At present, crop row detection methods based on digital image processing technology have been widely studied and applied. Methods based on Hough transform, least squares method, vanishing point, stereo vision and horizontal stripe features can all be used for crop row detection. Hough transform effectively detects straight lines, but the calculation is large and sensitive to noise. The least squares method is simple and fast, suitable for when the crop rows are clear, but the accuracy decreases when they are blocked. The vanishing point method uses the far-end vanishing point to infer the direction, which is suitable for long and straight crop rows, but it is difficult to identify in complex environments. Stereo vision technology provides accurate 3D positioning, but the hardware cost is high and the calculation is complex. The horizontal stripe feature method simplifies the calculation by segmenting the image, which is suitable for parallel crop rows, but the effect is not good when the crop rows are irregular or densely planted.

[0026] Based on this, the embodiments of the present application provide a navigation method, device, electronic device and medium for a work vehicle, aiming to achieve high-precision prediction of the navigation line of the seedling belt.

[0027] See also Figure 1 , Figure 1 This is a flow chart of a navigation method for a working vehicle provided in an embodiment of the present application. Figure 1 As shown in , the navigation method of the working vehicle provided in the embodiment of the present application includes:

[0028] S101. Capture the current frame seedling belt image in the driving direction of the working vehicle by deploying a multimodal perception device at the front center position of the working vehicle.

[0029] Here, multimodal perception devices can be lidar, millimeter wave radar, ultrasonic radar, surround view and external cameras, which can collect real scene data. After preprocessing, these data generate high-precision true value data and upload them to the cloud. The cloud classifies the data according to the scene characteristics: high-similarity and high-precision data are added to the training set of the recognition model after automatic annotation combined with manual review optimization; low-similarity or low-precision data are regarded as complex or unknown scenes and enter the deep mining process, which may trigger more data collection or perception strategy optimization. For complex scenes such as extreme light and occlusion, the system enables enhanced mode to collect more details and mark edge case data in real time. The cloud performs perception task allocation and truth value optimization, combining automatic annotation with manual review to quickly process high-precision scenes and verify and optimize low-precision scenes. For uncovered boundaries and complex pattern data, the cloud triggers targeted collection tasks and generates customized perception tasks to provide diversified samples for the training of recognition models.

[0030] S102, inputting the seedling belt image of the current frame into the recognition model to obtain the anchor point coordinates of multiple anchor points for characterizing the seedling belt position.

[0031] Here, the recognition model is designed for efficient detection of seedling belt positions. First, images are selected from the cloud-based recognition model training set as training samples, and these sample images are accurately labeled using one-hot tags to identify and determine the positions of the real anchor points. The recognition model is established using these labeled seedling belt sample images and their corresponding real anchor points, and the recognition model is trained. Through this process, the model can learn the association between the seedling belt images and the real anchor points, so that it can automatically and accurately detect the corresponding anchor points in the new seedling belt images.

[0032] Below through Figure 2 Introduce the specific process of training the recognition model.

[0033] Figure 2 A flowchart of the training recognition model provided in an embodiment of the present application.

[0034] S201, input the seedling strip sample image into the recognition model to obtain the predicted anchor point.

[0035] Here, we select precisely labeled seedling belt sample images from the cloud recognition model training set. These sample images contain the real seedling belt location information, that is, the real anchor point. These sample images are input into the recognition model one by one. The main task of the recognition model is to predict the location of the seedling belt, that is, to generate the predicted anchor point. The predicted anchor point is the possible location of the seedling belt calculated by the model through the internal algorithm based on the characteristics of the input image.

[0036] Specifically, this recognition model integrates a feature extractor, a row anchor mechanism, a group classification strategy, and a loss function. Among them, the core of the feature extractor is based on the ResNet18 backbone network, which is used to extract feature information from the image. In order to cope with the challenge of slow semantic segmentation detection, the model introduces row anchors and group classification methods. The size of the row anchor is calculated based on the width and height pixel values ​​of the target ROI (Region of Interest). Specifically, the step size and the range covered by each anchor are determined by the following formula:

[0037]

[0038] Among them, Step represents the step size, that is, the size of each interval in the height direction of the seedling belt sample image. is the preset number of anchor points, H roi is the height of the target ROI area. Through this formula, the row intervals covered by each of the h anchor points can be accurately divided.

[0039] The group classification strategy is that for each row position of the anchor point, the model further performs grid division. The detected seedling belt can be represented as a selection of unit grids divided on the predefined row anchor points, and the number of grids is determined by the following formula:

[0040]

[0041] Among them, ω represents the number of grids set, N represents the step size of each small group (division in the width direction), W roi Indicates the width of the target ROI area.

[0042] The output of the seedling strip classifier can be expressed as the following formula:

[0043]

[0044] Where P i,j,: is a three-dimensional tensor A slice of , P: is a three-dimensional tensor containing the predicted probability distribution of all seedling strips and anchor points, R is a real number set, indicating that the elements in the tensor are real numbers, f ij (X) represents the predicted probability distribution of a specific seedling band and anchor point position, i is the index of the seedling band, ranging from 1 to C (C is the total number of seedling bands), j represents the number of the row anchor point, ranging from 1 to h, and “:” means taking all elements in the third dimension (i.e., the number of grids plus 1, because an additional background or boundary category is included).

[0045] During the training process, the model uses the known label file to compare with the predicted result and calculate the loss. In the label file, the target category is encoded as 1 and the rest of the background is 0 ，Based on the real anchor point T i,j,: and the predicted anchor point P i,j,: By comparing the models, we can identify the loss values ​​calculated by the models and use them to optimize the model parameters.

[0046] S202, calculating the difference between the real anchor points and the predicted anchor points of the seedling strip sample image to obtain the loss function of the seedling strip sample image.

[0047] Here, after obtaining the predicted anchor point, it is necessary to evaluate the prediction accuracy of the model. This step is achieved by calculating the difference between the real anchor point and the predicted anchor point. The calculation method of the difference involves a variety of loss functions. Since the seedling belt detection involves the accurate identification of the seedling belt position in the image, a loss function that can reflect the position error is used. By calculating the loss function, the gap between the model prediction result and the actual situation can be quantified, thereby providing guidance for model optimization.

[0048] The specific expression of the loss function may involve multiple aspects, such as classification loss (used to evaluate the model's prediction accuracy of the seedling belt category), continuity loss (used to ensure the continuity of seedling belt detection), position loss (used to improve the accuracy of seedling belt position estimation), etc. These loss functions together constitute the optimization target in the model training process.

[0049] In a preferred example of the present application, the loss function calculation involves multiple aspects, mainly including classification loss, continuity loss, position loss and segmentation loss.

[0050] Specifically, the classification loss is used to measure the accuracy of the model in classifying the seedling belt position. The cross entropy loss function is used to calculate the difference between the probability distribution of the seedling belt position predicted by the identification model and the true label. The specific expression involves the comparison of the predicted probability of each seedling belt and each row anchor point with the true label, and the sum is used to obtain the total classification loss. The calculation formula is as follows:

[0051]

[0052] Among them, C represents the total number of seedling strips, h represents the number of row anchor points, and p i,j,: represents the predicted probability distribution of the i-th strip and the j-th row anchor point, T i,j,: represents the corresponding real anchor point, L cE is the cross entropy loss function.

[0053] Continuity loss is used to ensure the continuity of seedling belt detection, especially in the case of broken seedlings. In the monitoring scenario, when seedlings are missing in the seedling belt, in order to ensure the continuity of the navigation line, the continuity of adjacent anchor points on each detected seedling belt must be enhanced. To this end, the present application adopts a strategy, which is to make the predicted probability distribution of connected anchor points consistent, so as to improve the continuity of seedling belt detection. At the same time, the method must also have a certain robustness against abnormal differences caused by missing seedlings. Given that the L1 norm tends to generate sparse solutions by calculating the sum of the absolute values ​​of the differences of each element, this is particularly beneficial for maintaining the local smoothness of the model. Therefore, an L1 norm constraint is imposed on the difference between the predicted probability distributions of adjacent anchor points to effectively ensure this continuity. The specific expression for calculating the continuity loss is as follows:

[0054]

[0055] Among them, L C is the continuity loss function, where L1||·|| 1 represents the L1 norm, which means taking the absolute value of each element and summing them up, P i,j,: and P i,j+1,: Represent the predicted probability distribution of the current anchor point and its adjacent anchor points respectively.

[0056] Position loss is used to ensure the smoothness and accuracy of the seedling belt navigation line. In the seedling belt detection, due to the influence of weeds and the fact that the seedling belt is not straight, in order to ensure the smoothness of the overall fitting line of the seedling belt navigation line, the maximum probability distribution in the classified prediction position is adopted and weighted to obtain continuous position estimation. The specific expression involves the calculation of the second-order difference of the positions of adjacent anchor points and the weighted summation combined with the cross entropy loss function, as follows.

[0057]

[0058] Among them, P i,j,1:w represents the classification probability of the i-th strip and the j-th row anchor point on the ω grid. i,j,1:w The generated ω probabilities are normalized into a probability distribution, and then the specific k value is determined for the location of the specific position. In the formula, k represents the index of the position, and the value range is from 1 to ω. Since softmax(P i,j,1:w ) is a discrete point, for softmax(P i,j,1:w ) is weighted averaged to obtain a continuous seedling belt detection line on the image, and Represents the i-th slender band, the possible positions on the j-th row anchor point are weighted averaged to obtain a continuous position loss function on the image as follows:

[0059]

[0060] in, represents the continuous position estimate of the jth row anchor point of the i-th strip obtained by weighted averaging. Here, second-order differences are used to smooth the position estimates and show weaker constraints on outliers, thereby improving the robustness of the model to curved strips.

[0061] The segmentation loss is used in the seedling band segmentation branch. It performs fine-grained feature learning on pixel-level features and is calculated by comparing the difference between the seedling band and background edge and regional global information features predicted by the model and the true label. The specific expression may involve comparing the predicted value of each pixel with the true value, and summing them up to get the total segmentation loss. Segmentation loss L seg The specific expression of is as follows, and the cross entropy loss is used.

[0062]

[0063] Among them, k represents each pixel in the segmentation, c represents the category (seedling or background), y k,c represents the true value of pixel k in category c using one-hot encoding, represents the corresponding predicted value, ∑ k ∑ c Represents the sum of pixel index k and category index c.

[0064] During the model training phase, these loss functions are weighted and summed to generate the total loss of the model, which is used to update and optimize the model. In this way, the model can learn how to identify the location of the seedling belt more accurately and maintain the continuity and smoothness of the seedling belt detection. Therefore, the expression of the loss function Loss is:

[0065] Loss = L cls +αL c +βL L +L seg

[0066] Among them, α and β are weight coefficients used to balance the impact of different loss functions on the total loss.

[0067] S203, updating the parameters of the recognition model and the weight of the loss function by iterating the recognition model until the result of the loss function is less than a preset standard.

[0068] Here, after obtaining the loss function, the iterative process of model training begins. This process involves updating the recognition model parameters and adjusting the loss function weights. The purpose of the iteration is to minimize the loss function, that is, to reduce the gap between the model prediction results and the actual situation.

[0069] In each round of iteration, the back propagation algorithm is used to calculate the gradient of the loss function to the model parameters, and the model parameters are updated according to the gradient. At the same time, if the loss function contains learnable weight parameters, they will also be updated accordingly. Through multiple rounds of iterations, the model gradually learns the characteristic representation of the seedling belt and improves the prediction accuracy of the seedling belt position. The iteration process will continue until the result of the loss function is less than the preset standard. The preset standard is a pre-set threshold used to determine whether the model has achieved sufficient prediction accuracy. When the result of the loss function is less than this threshold, the model is considered to have converged and the training process ends.

[0070] In the training stage of the seedling belt recognition model, the specific process is as follows: the seedling belt sample image is first subjected to feature extraction through the ResNet18 network to generate a global feature map. Subsequently, these feature maps are sent to the seedling belt position classification and seedling belt segmentation branches respectively. The classification branch divides the image into multiple grid units, and classifies and predicts the grid units on each row to generate a probability distribution of the seedling belt position. The approximate position of the seedling belt is determined by selecting the grid unit with the highest probability in each row. At the same time, the segmentation branch performs fine-grained learning on pixel-level features to extract the edge and regional global information of the seedling belt and background to improve the recognition accuracy. On this basis, the continuity loss L is calculated. C and position loss L L To constrain the continuity of the seedling belt position and the smoothness of the entire seedling belt, the classification loss L is finally calculated. cls , Continuity loss L C , position loss L L and segmentation loss L seg The total loss of the model generated by weighted summation is used to update the model. The seedling segmentation branch is deleted in the inference stage of the model, and the model is only classified by seedling position and continuity loss L C , position loss L L ,This can quickly identify the location of the seedling belt and maintain coherence, while reducing the computational overhead and improving the speed of reasoning.

[0071] return Figure 1 , S103, fitting and classifying the multiple anchor points according to the anchor point coordinates of the multiple anchor points to obtain multiple point sets.

[0072] In the embodiment of the present application, each point set includes two point sets, each point set represents a straight line point group fitted in the driving direction of the seedling belt, and the two point sets in each point set are symmetrical with the multimodal sensing device as the center.

[0073] The fitting classification process here is mainly divided into four categories, namely L1, L2, R2, and R1. These four types of point sets correspond to anchor point sets at different positions in the seedling belt image. Specifically, the row coordinates of the anchor point are used as the y value, and each anchor point corresponds to four possible x values ​​(according to its projection position on different fitting lines), and these x values ​​are arranged in ascending order and classified into the four point sets of L1, L2, R2, and R1. Therefore, for 56 anchor points, 4 point set categories will be obtained, each category contains 56 points, a total of 56*4 point sets, and the index numbers of L1, L2, R2, and R1 are assigned respectively. Each point set contains two point sets, which respectively represent the straight line point groups on the left and right sides fitted along the driving direction in the seedling belt, and are symmetrical with the multimodal sensing device as the center.

[0074] S104: Generate a navigation line according to anchor points in a target point set in a plurality of point sets, so that a working vehicle can work along the navigation line.

[0075] Here, in the process of generating the navigation line, it is first determined whether the number of points in each point set (L1, L2, R2, R1) meets the preset number, which is more than 2 / 3 of the total number.

[0076] Among them, if the sequence of the current frame seedling belt image is the first frame and the number of points does not meet the preset number, no subsequent calculation is performed; if the sequence of the current frame seedling belt image is not the first frame and the number of points does not meet the preset number, the calculation result of the previous frame is retained.

[0077] When the number of point sets meets the conditions, fit L1 and R1 (two outer seedling belts) and L2 and R2 (two inner seedling belts) respectively. For L1 and R1, calculate the center coordinates under the same anchor point and put these center coordinates into point set A; similarly, for L2 and R2, calculate the center coordinates and put them into point set B.

[0078] Next, the number of points in point set A and point set B is determined respectively. If the number of points in point set A is greater than 45, point set A is determined as the target point set, and the center line in point set A is fitted, and the slope of the fitted line and the center line equation are calculated; if the number of points in point set B is greater than 32, point set B is determined as the target point set, and the fitting and equation calculation are performed in the same way. If the conditions are not met, whether to perform the calculation is determined based on whether it is the first frame or the calculation result of the previous frame is kept.

[0079] When the number of points in point set A and point set B meets the requirements, the intersection range is obtained, and the corresponding common point set is used as the central point range. For the common point set in point set A, the x-coordinate is assigned a scale factor of 0.3; for the common point set in point set B, the x-coordinate is assigned a scale factor of 0.7. The two parts of the x-coordinate are added to form a new point set C, and point set C is determined as the target point set. The straight line equation in point set C is fitted, and its slope is calculated. According to the slope K of the fitted straight line, the heading angle can be calculated. When the slope is greater than 0, the heading angle is calculated as -|arctan(K)|; when the slope is less than 0, the heading angle is calculated as |arctan(K)|. The heading deviation is obtained by subtracting the horizontal coordinate of the preview point calculated in the fitting equation of point set C from the vertical coordinate of the preview point. If the calculated value is less than 0, it is the left deviation; otherwise, it is the right deviation.

[0080] Finally, print the center point of each frame, the vertical coordinate of the preview point, and the horizontal coordinate calculated in the fitting equation of the point set C. These values ​​together constitute the predicted trajectory, that is, the navigation line. The operating vehicle will operate along this predicted navigation line.

[0081] Compared with other technologies, this application adopts a new model architecture, focusing on the precise identification of seedling belt pixels, effectively reducing pixel-level fitting errors and significantly improving identification accuracy. The optimal path is automatically determined by an intelligent algorithm, and high-precision prediction of the seedling belt navigation line is achieved, which makes the prediction process standardized and automated, avoiding the limitations and human errors of manually set areas, thereby improving overall work efficiency and quality. The core lies in the methodology of combining deep learning detection technology with agricultural scenarios, and the efficient and accurate seedling belt identification algorithm developed based on this. This application aims to ensure the uniqueness and leading nature of the technology, promote its widespread application in agricultural scenarios such as intertillage plant protection and precision seeding, and promote the continuous innovation and development of agricultural technology.

[0082] By accurately identifying the location of the seedling belt, the operation accuracy can be significantly improved, which can guide agricultural machinery such as seeders and harvesters to operate more accurately, effectively reduce damage to crops, and improve operation efficiency and quality. This technology not only helps to reduce the excessive use of agricultural materials such as fertilizers and pesticides, and reduce agricultural production costs, but also reduces pollution to soil and water bodies and promotes ecological environmental protection. At the same time, the application of advanced image processing technology and artificial intelligence algorithms to agriculture has promoted the development of agriculture towards intelligence and informatization, laying a solid foundation for the realization of smart agriculture. In addition, accurate seedling belt detection also enhances operation safety and improves the operation safety of unmanned or less-manned agricultural machinery. By adjusting the algorithm parameters, this technology can flexibly adapt to seedling belts of different widths and shapes and crop types, and can be combined with other sensor data (such as soil moisture, light intensity, etc.) to support the refined management of farmland and further optimize resource allocation.

[0083] Based on the same inventive concept, an embodiment of the present application also provides a navigation device for a working vehicle corresponding to the navigation method for the working vehicle. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the navigation method for the working vehicle in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0084] See also Figure 3 , Figure 3 A structural diagram of a navigation device for a working vehicle provided in an embodiment of the present application is shown in FIG. Figure 3 As shown in , the navigation device 300 of the working vehicle includes:

[0085] A capture module, which captures the current frame seedling belt image in the driving direction of the working vehicle through a multimodal perception device deployed at the front center position of the working vehicle;

[0086] An input module is used to input the seedling belt image of the current frame into a recognition model to obtain the anchor point coordinates of multiple anchor points used to characterize the seedling belt position.

[0087] A classification module is used to fit and classify the multiple anchor points according to the anchor point coordinates of the multiple anchor points to obtain multiple point sets, each point set includes two point sets, each point set represents a straight line point group fitted in the seedling belt along the driving direction, and the two point sets in each point set are symmetrical with the multimodal sensing device as the center.

[0088] A generating module is used to generate a navigation line according to an anchor point in a target point set in the plurality of point sets, so that the working vehicle can operate along the navigation line.

[0089] See also Figure 4 , Figure 4This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0090] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 as well as Figure 2 The specific implementation of the steps of the navigation method for the working vehicle in the method embodiment shown can be found in the method embodiment, and will not be repeated here.

[0091] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 2 The specific implementation of the steps of the navigation method for the working vehicle in the method embodiment shown can be found in the method embodiment, and will not be repeated here.

[0092] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0093] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0094] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0095] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0096] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0097] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A navigation method for a working vehicle, characterized in that: The method comprises: The current frame seedling belt image in the driving direction of the working vehicle is captured by a multimodal sensing device deployed at the front center of the working vehicle; Inputting the seedling belt image of the current frame into a recognition model to obtain the anchor point coordinates of a plurality of anchor points used to characterize the position of the seedling belt; According to the anchor point coordinates of the plurality of anchor points, the plurality of anchor points are fitted and classified to obtain a plurality of point sets, each of which includes two point sets, each of which is composed of points on a straight line fitted along the driving direction in the seedling belt, and the two point sets in each point set are bilaterally symmetrical with the multimodal sensing device as the center; A navigation line is generated according to an anchor point in a target point set in the plurality of point sets, so that the working vehicle operates along the navigation line.

2. The method according to claim 1, characterized in that The target point set is determined by: For each point set, determining whether the number of anchor point coordinates in the point set is greater than a preset number; If there is no point set with a number of anchor point coordinates greater than a preset number, determining the target point set according to the sequence of the seedling strip image of the current frame; If the number of anchor point coordinates in a point set is greater than a preset number, the point set is determined as the target point set; If the number of anchor point coordinates in at least two point sets is greater than a preset number, the at least two point sets are combined, and the combined point set is determined as the target point set.

3. The method according to claim 2, characterized in that The target point set is determined according to the sequence of the seedling strip image of the current frame in the following manner: Determine whether the sequence of the seedling band image of the current frame is the first frame; If not, the target point set of the previous seedling belt image is determined as the target point set of the current seedling belt image; If so, it is determined that the target point set of the seedling band image of the current frame is empty.

4. The method according to claim 3, characterized in that Generate guide lines in the following ways: Fitting the anchor point coordinates of multiple anchor points in the target point set to obtain a center line; The slope of the centerline is calculated, and a centerline equation is constructed according to the slope, so as to generate the navigation line according to the centerline equation.

5. The method according to claim 1, characterized in that The recognition model is constructed in the following way: Obtain sample images of seedling strips; Annotating the seedling belt sample image to obtain a real anchor point for representing the position of the seedling belt; The relationship between the seedling strip sample image and the real anchor point is represented by the recognition model.

6. The method according to claim 5, characterized in that The recognition model is trained in the following way: Inputting the seedling strip sample image into the recognition model to obtain a predicted anchor point; Calculating the difference between the true anchor point and the predicted anchor point to obtain a loss function of the seedling strip sample image; The recognition model is iterated to update the parameters of the recognition model and the weight of the loss function until the result of the loss function is less than a preset standard.

7. The method according to claim 6, characterized in that The loss function is obtained according to classification loss, continuity loss, position loss and segmentation loss. The classification loss is used to characterize the difference in category between the predicted anchor point and the true anchor point. The continuity loss is used to characterize the degree of stable association between the predicted anchor point between consecutive frames. The position loss is used to characterize the prediction accuracy of the seedling belt position by the recognition model. The segmentation loss is used to characterize the classification accuracy of the seedling belt and the background of the recognition model in the seedling belt image of the current frame.

8. A navigation device for a working vehicle, characterized in that: The device comprises: A capture module, which captures the current frame seedling belt image in the driving direction of the working vehicle through a multimodal perception device deployed at the front center position of the working vehicle; An input module, used for inputting the seedling belt image of the current frame into a recognition model to obtain the anchor point coordinates of a plurality of anchor points for characterizing the position of the seedling belt; A classification module, configured to fit and classify the plurality of anchor points according to the anchor point coordinates of the plurality of anchor points to obtain a plurality of point sets, each of which includes two point sets, each of which is composed of points on a straight line fitted along a driving direction in the seedling belt, and the two point sets in each point set are bilaterally symmetrical with the multimodal sensing device as the center; A generating module is used to generate a navigation line according to an anchor point in a target point set in the plurality of point sets, so that the working vehicle can operate along the navigation line.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are executed.

Images