Method and device for determining agricultural machine navigation line and agricultural machine

Abstract

The invention provides an agricultural machine navigation line determination method and device and an agricultural machine, and the method comprises the steps: carrying out the image segmentation of a crop row image based on a preset image segmentation detection model and a preset region of interest, and obtaining a background and a plurality of crop row contours; determining a target crop row corresponding to each crop row contour based on the number of crop row contours and the abscissa of the central point of the circumscribed rectangle of each crop row contour; if the number of the crop row contours is less than four, determining that there is a missing target crop row, and based on the crop row detection line information of each crop row contour, obtaining crop row detection line information corresponding to the missing target crop row from the temporary crop row data storage list; and based on the crop row detection line information corresponding to each target crop row and a preset reference weight, determining an agricultural machine navigation line corresponding to the crop row image. Therefore, in a crop row missing scene, the missing crop row is complemented by analyzing the detected crop row, and the navigation line is generated according to the complemented crop row, so that the accuracy of determining the agricultural machine navigation line can be improved.

Description

Technical Field

[0001] The present application relates to the field of agricultural machinery navigation technology, and in particular to a method and device for determining an agricultural machinery navigation line, and agricultural machinery. Background Art

[0002] With the continuous breakthroughs in science and technology, agricultural production is developing towards intelligence and precision. Precision and intelligence in agriculture can significantly reduce operating costs and improve operating efficiency. As an important branch of precision agriculture, automatic navigation of agricultural machinery has a wide range of applications in many aspects such as crop sowing, spraying, fertilization and harvesting. Among them, the use of machine vision and deep learning technology to achieve automatic navigation of agricultural machinery has gradually become one of the trends in the development of navigation technology in recent years, and the determination of agricultural machinery navigation lines is the basis for realizing automatic navigation technology of agricultural machinery.

[0003] The core of agricultural machinery navigation line determination lies in the detection of crop rows. Existing crop row detection methods use crop row detection models built based on machine vision and deep learning technology to identify crop rows. However, in actual farmland operations, crop rows may have missing ridges and broken rows. In this case, the crop row detection model cannot accurately identify the crop rows, which leads to large deviations in the agricultural machinery navigation line determined based on the crop rows, and low accuracy in determining the agricultural machinery navigation line. Summary of the invention

[0004] In view of this, the embodiments of the present application at least provide a method, device and agricultural machinery for determining the navigation line of agricultural machinery. In the scenario where crop rows are missing, the missing crop rows are completed by analyzing the detected crop rows, and the navigation lines are generated according to the completed crop rows, thereby improving the accuracy of determining the navigation line of the agricultural machinery.

[0005] This application mainly includes the following aspects:

[0006] In a first aspect, an embodiment of the present application provides a method for determining an agricultural machinery navigation line, the method comprising:

[0007] Based on a preset image segmentation detection model and a preset region of interest, the crop row image acquired from the image acquisition device is segmented to obtain a background and multiple crop row contours;

[0008] Based on the number of crop row contours and the horizontal coordinates of the center points of the circumscribed rectangles of the crop row contours, the target crop rows corresponding to the crop row contours are determined; the target crop rows are reference crop rows for the agricultural machinery navigation line, and in order from left to right, they include the first crop row on the left, the second crop row on the left, the second crop row on the right, and the first crop row on the right;

[0009] If the number of crop row contours is less than four, it is determined that there are missing target crop rows in the crop row image, and based on the crop row detection line information of each of the crop row contours, the crop row detection line information corresponding to the missing target crop row is obtained from a temporary crop row data storage list; the crop row detection line information includes the center point coordinates of a group of points with the largest ordinates and the center point coordinates of a group of points with the smallest ordinates in the crop row contour; the temporary crop row data storage list stores the crop row detection line information corresponding to each of the target crop rows in the historical crop row image with four crop row contours;

[0010] Based on the crop row detection line information corresponding to each of the target crop rows and the preset reference weight, the agricultural machinery navigation line corresponding to the crop row image is determined.

[0011] In a second aspect, an embodiment of the present application further provides a device for determining an agricultural machinery navigation line, the device for determining an agricultural machinery navigation line comprising:

[0012] An image segmentation module is used to perform image segmentation on the crop row image acquired from the image acquisition device based on a preset image segmentation detection model and a preset region of interest to obtain a background and multiple crop row contours;

[0013] A first determination module is used to determine the target crop rows corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinates of the center points of the circumscribed rectangles of each crop row outline; the target crop rows are reference crop rows for the agricultural machinery navigation line, and include, in order from left to right, the first crop row on the left, the second crop row on the left, the second crop row on the right, and the first crop row on the right;

[0014] an information acquisition module, for determining, if the number of crop row contours is less than four, whether there are missing target crop rows in the crop row image, and acquiring, from a temporary crop row data storage list, crop row detection line information corresponding to the missing target crop rows based on the crop row detection line information of each of the crop row contours; the crop row detection line information includes the center point coordinates of a group of points with the largest ordinates and the center point coordinates of a group of points with the smallest ordinates in the crop row contour; the temporary crop row data storage list stores the crop row detection line information corresponding to each of the target crop rows in the historical crop row image with four crop row contours;

[0015] The second determination module is used to determine the agricultural machinery navigation line corresponding to the crop row image based on the crop row detection line information corresponding to each target crop row and a preset reference weight.

[0016] In a third aspect, an embodiment of the present application further provides an agricultural machine, including an image acquisition device and a device for determining a navigation line of the agricultural machine as described above;

[0017] The image acquisition device is used to send the acquired crop row image to the agricultural machinery navigation line determination device.

[0018] The method, device and agricultural machine for determining the agricultural machinery navigation line provided in the embodiment of the present application perform image segmentation on the crop row image acquired from the image acquisition device based on a preset image segmentation detection model and a preset region of interest to obtain a background and multiple crop row contours; based on the number of crop row contours and the horizontal coordinates of the center points of the circumscribed rectangles of the crop row contours, determine the target crop rows corresponding to the respective crop row contours; the target crop rows are the reference crop rows for the agricultural machinery navigation line, and in order from left to right, they include the first crop row on the left, the second crop row on the left, the second crop row on the right and the first crop row on the right; if the number of crop row contours is less than four, determine the crop row image; There are missing target crop rows in the image, and based on the crop row detection line information of each crop row contour, the crop row detection line information corresponding to the missing target crop row is obtained from the temporary crop row data storage list; the crop row detection line information includes the center point coordinates of a group of points with the largest ordinate in the crop row contour and the center point coordinates of a group of points with the smallest ordinate; the temporary crop row data storage list stores the crop row detection line information corresponding to each target crop row in the historical crop row image with four crop row contours; based on the crop row detection line information corresponding to each target crop row and the preset reference weight, the agricultural machinery navigation line corresponding to the crop row image is determined. In this way, in the scenario of missing crop rows, the missing crop rows are supplemented by analyzing the detected crop rows, and the navigation line is generated according to the supplemented crop rows, so as to improve the accuracy of determining the agricultural machinery navigation line.

[0019] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flow chart showing a method for determining an agricultural machinery navigation line provided in an embodiment of the present application is shown;

[0022] Figure 2 A schematic diagram of the process of training a preset image segmentation detection model in an embodiment of the present application is shown;

[0023] Figure 3 A specific flow chart of image segmentation detection in an embodiment of the present application is shown;

[0024] Figure 4 A schematic diagram showing a crop row image in an embodiment of the present application is shown;

[0025] Figure 5 A schematic diagram of crop row segmentation detection in an embodiment of the present application is shown;

[0026] Figure 6 A schematic diagram showing multiple crop row outlines in an embodiment of the present application is shown;

[0027] Figure 7 A schematic diagram showing a circumscribed rectangle of a crop row outline in an embodiment of the present application is shown;

[0028] Figure 8 A schematic diagram of a crop row detection line in an embodiment of the present application is shown;

[0029] Fig. 9 A schematic diagram showing a crop row detection line, an agricultural machinery navigation line, and an agricultural machinery driving line in an embodiment of the present application is shown;

[0030] Fig.10 One of the functional module diagrams of a device for determining a navigation line of agricultural machinery provided in an embodiment of the present application is shown;

[0031] Fig.11 A second functional module diagram of a device for determining a navigation line of an agricultural machinery provided in an embodiment of the present application is shown;

[0032] Fig.12 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0034] To facilitate the understanding of the present application, the technical solution provided by the present application is described in detail below in conjunction with specific embodiments.

[0035] The embodiments of the present application provide a method, device and agricultural machinery for determining the navigation line of agricultural machinery. In a scenario where crop rows are missing, the missing crop rows are completed by analyzing the detected crop rows, and a navigation line is generated based on the completed crop rows, thereby improving the accuracy of determining the navigation line of the agricultural machinery.

[0036] See also Figure 1 , Figure 1 This is a flow chart of a method for determining an agricultural machinery navigation line provided in an embodiment of the present application. Figure 1 As shown, the method for determining the agricultural machinery navigation line provided in the embodiment of the present application includes the following steps:

[0037] S101, based on a preset image segmentation detection model and a preset region of interest, performing image segmentation on a crop row image acquired from an image acquisition device to obtain a background and a plurality of crop row contours.

[0038] In the embodiment of the present application, it should be noted that before performing image segmentation on the crop row image obtained from the image acquisition device, it is necessary to train the preset image segmentation detection model. Figure 2 , Figure 2 Schematic diagram of the process of training the preset image segmentation detection model in the embodiment of the present application. Figure 2 As shown in the figure, the training process mainly includes data set preparation, image segmentation detection model debugging and training. The original crop row image is the basis of crop row detection. First, the original crop row image data is collected by data acquisition equipment, and then the original crop row image is labeled with crop rows through LabelMe annotation software to obtain the crop row training data set required by the detection model. Before training the image segmentation model, the parameters such as the image segmentation model training batch, batch-size, learning rate and the values ​​of hyperparameters are adjusted to obtain the current optimal detection model parameters. After the preparation work is completed, the image segmentation detection model is trained. After the training is completed, the obtained image segmentation model is verified. If the detection accuracy of the model is not high during the verification, return to the parameter adjustment step to readjust the parameters.

[0039] Specifically, see Figure 3 , Figure 3 FIG. 1 is a specific flow chart of image segmentation detection in the embodiment of the present application. Figure 3As shown in the figure, the original crop row RGB image is first labeled using LabelMe software before training through the deep learning network, and the texture, color and other features of the crop row are obtained through manual labeling. After the image is sent to the network, the image enters the backbone, neck and detection head of the network in turn for convolution, pooling, feature fusion, up and down sampling and other operations to extract the high-dimensional features of the image. Finally, the detection head segments the high-dimensional feature map, detects the crop row and labels the crop row area, and finally presents the detection results on the original image.

[0040] After the preset image segmentation detection model is trained and verified, the crop row image obtained from the image acquisition device is segmented based on the preset image segmentation detection model and the preset region of interest to obtain the background and multiple crop row contours. Figures 4 to 6 , Figure 4 is a schematic diagram of a crop row image in an embodiment of the present application, Figure 5 This is a schematic diagram of crop row segmentation detection in an embodiment of the present application. Figure 6 Schematic diagram of multiple crop row outlines in the embodiment of the present application. Figure 4 , 5 As shown in Figure 2, after segmenting multiple crop row contours with obvious features in the center of the image, Figure 6 As shown, the segmented image is subjected to image binarization processing (the crop row part is set to 255, and the background part is set to 0) and opening operation (first corrosion, then expansion) processing. After the above processing, an area with a preset range of crop row image width and crop row image height is set in the center of the crop row image as a preset region of interest (ROI). In the embodiment of the present application, the position of the ROI area is located at a position of 0.25 to 0.75 times the crop row image width and 0.25 to 0.75 times the crop row image height. All images outside the preset region of interest are set as background, with a pixel value of 0, the pixel value of the crop row area within the preset region of interest is set to 255, and the pixel values ​​of other areas are set to 0. A list of several closed image outer contours is obtained by extracting the connected domain of white pixels, and all crop row contours are extracted as crop row candidates by area.

[0041] S102, based on the number of crop row contours and the horizontal coordinates of the center points of the circumscribed rectangles of each crop row contour, determine the target crop rows corresponding to each crop row contour; the target crop rows are reference crop rows for the agricultural machinery navigation line, and include, in order from left to right, the first crop row on the left, the second crop row on the left, the second crop row on the right and the first crop row on the right.

[0042] In the present application, please refer to Figure 7 , Figure 7Schematic diagram of the circumscribed rectangle of the crop row outline in the embodiment of the present application. Figure 7 As shown, after obtaining the crop row contour, since there may be missing ridges, broken rows, etc. in the crop row image, it is necessary to determine which crop row in the target crop row each crop row contour corresponds to based on the number of crop row contours and the horizontal coordinate of the center point of the circumscribed rectangle of each crop row contour, so as to generate the subsequent agricultural machinery navigation line. In the embodiment of the present application, in order to avoid navigation line planning errors caused by the offset of crop rows in the image due to the left and right shaking of the vehicle, the present application sets four target crop rows as reference crop rows for generating agricultural machinery navigation lines, and names them as the first crop row on the left, the second crop row on the left, the second crop row on the right, and the first crop row on the right in order from left to right.

[0043] S103, if the number of crop row contours is less than four, determine that there are missing target crop rows in the crop row image, and based on the crop row detection line information of each of the crop row contours, obtain the crop row detection line information corresponding to the missing target crop rows from a temporary crop row data storage list; the crop row detection line information includes the center point coordinates of a group of points with the largest ordinate and the center point coordinates of a group of points with the smallest ordinate in the crop row contour; the temporary crop row data storage list stores the crop row detection line information corresponding to each of the target crop rows in the historical crop row image with four crop row contours.

[0044] In an embodiment of the present application, if the number of crop row contours detected based on the crop row image is less than four, it means that there are missing target crop rows in the crop row image, that is, there are missing ridges, broken rows, etc. in the farmland. At this time, data completion is performed by calling the corresponding missing data in the temporary crop row data storage list to prevent the generation of agricultural machinery navigation lines from being affected by factors such as missing ridges and broken rows. Specifically, based on the crop row detection line information of each crop row contour, search the temporary crop row data storage list for historical crop row images that are identical or similar to the crop row detection line information of each crop row contour of the current crop row image, and obtain the crop row detection line information corresponding to each of the target crop rows in the historical crop row images that are missing from the current crop row image; wherein, refer to Figure 8 , Figure 8 Schematic diagram of the crop row detection line in the embodiment of the present application. Figure 8As shown, the crop row detection line information includes the center point coordinates of a group of points with the largest ordinates and the center point coordinates of a group of points with the smallest ordinates in the crop row contour; according to the coordinates of these two points and OPENCV, the crop row detection line of the corresponding crop row can be generated. The temporary crop row data storage list stores the crop row detection line information corresponding to each of the target crop rows in the historical crop row image with four crop row contours, that is, when the number of crop row contours is detected to be four, the crop row detection line information corresponding to each of the target crop rows in the crop row image is stored in the temporary crop row data storage list.

[0045] S104: Determine the agricultural machinery navigation line corresponding to the crop row image based on the crop row detection line information corresponding to each of the target crop rows and a preset reference weight.

[0046] In this step, based on the crop row detection line information corresponding to each target crop row in the crop row image and the preset reference weight corresponding to each crop row, the agricultural machinery navigation line corresponding to the crop row image is generated through the center line obtaining algorithm.

[0047] Furthermore, S102 specifically includes:

[0048] If there are four crop row contours, the target crop rows corresponding to the respective crop row contours are respectively determined as the first crop row on the left, the second crop row on the left, the second crop row on the right and the first crop row on the right in ascending order of the horizontal coordinates of the center points of the circumscribed rectangles of the respective crop row contours.

[0049] In this step, if the number of crop row contours is four, it means that there is no crop row missing in the crop row image, and the target crop rows corresponding to each crop row contour can be directly determined as the first crop row on the left, the second crop row on the left, the second crop row on the right, and the first crop row on the right according to the horizontal coordinates of the center points of the circumscribed rectangles of each crop row contour from small to large. Further, when the number of crop row contours detected is greater than or equal to four, it means that part of the crop row in the distance may have entered the area of ​​interest due to the shaking of the vehicle, or large rocks in the farmland may have interfered with the image. At this time, four crop row contours that meet the crop row planting rules are first selected as the crop row contour recognition results, and then the above steps are performed.

[0050] Furthermore, S102 also includes:

[0051] Step a1, if there are three crop row contours, determine the first center point, the second center point and the third center point in the ascending order of the horizontal coordinates of the center points of the circumscribed rectangles of the crop row contours, and determine whether there is a multiple relationship between the first distance between the horizontal coordinates of the first center point and the second center point and the second distance between the horizontal coordinates of the second center point and the third center point.

[0052] In this step, if the number of crop row contours is four, it means that there is a crop row missing in the crop row image, so it is necessary to determine which target crop row the three crop row contours correspond to to locate the missing target crop row. Determine the first center point, the second center point, and the third center point in the order of the horizontal coordinates of the center points of the circumscribed rectangles of the crop row contours from small to large, and determine whether there is a multiple relationship between the first distance between the horizontal coordinates of the first center point and the second center point and the second distance between the horizontal coordinates of the second center point and the third center point. Specifically, it can be determined whether there is a multiple relationship between the distances of the horizontal coordinates of the center points of the circumscribed rectangles of two adjacent crop row contours to determine whether the three crop row contours are adjacent target crop rows, and then the missing target crop row can be located.

[0053] Step a2: If there is a multiple relationship, and the absolute value of the difference between the second distance and twice the first distance is less than or equal to a preset distance threshold, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point and the third center point are respectively determined as the first crop row on the left, the second crop row on the left and the first crop row on the right.

[0054] In this step, if there is a multiple relationship, and the absolute value of the difference between the second distance and twice the first distance is less than or equal to the preset distance threshold, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point, and the third center point are respectively determined as the first crop row on the left, the second crop row on the left, and the first crop row on the right. In this way, it can be determined that the missing crop row is the second crop row on the right. In addition, since the distances between the center points of the outer contours of the crop rows are not completely equal, a preset distance threshold is set as a judgment standard when judging the multiple relationship to improve the robustness of the judgment.

[0055] Step a3: If there is a multiple relationship, and the absolute value of the difference between the first distance and twice the second distance is less than or equal to the preset distance threshold, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point and the third center point are respectively determined as the first crop row on the left, the second crop row on the right and the first crop row on the right.

[0056] In this step, if there is a multiple relationship, and the absolute value of the difference between the first distance and twice the second distance is less than or equal to the preset distance threshold, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point, and the third center point are respectively determined as the first crop row on the left, the second crop row on the right, and the first crop row on the right. In this way, it can be determined that the missing crop row is the second crop row on the right. In addition, this method of steps a1 to a3 can effectively avoid the situation where the crop row is not in the center of the corresponding position due to the left and right shaking of the vehicle, resulting in the misjudgment of the crop row.

[0057] Furthermore, S102 also includes:

[0058] Step b1, if there is no multiple relationship, and the horizontal coordinate of the second center point is smaller than the horizontal coordinate of the center point of the preset area of ​​interest, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point and the third center point are respectively determined as the first crop row from the left, the second crop row from the left and the second crop row from the right.

[0059] In this step, if there is no multiple relationship, it is determined that the three crop row contours correspond to three adjacent target crop rows, so the size of the horizontal coordinate of the second center point located in the middle and the horizontal coordinate of the center point of the preset area of ​​interest can be determined, and the position of the middle crop row contour can be determined to determine the corresponding relationship between each crop row contour and the target crop row. If the horizontal coordinate of the second center point is smaller than the horizontal coordinate of the center point of the preset area of ​​interest, it means that the navigation line contour corresponding to the second center point is located on the left side, then the target crop row corresponding to the crop row contours corresponding to the first center point, the second center point and the third center point are respectively determined as the first crop row on the left, the second crop row on the left and the second crop row on the right, so that the missing crop row can be determined to be the first crop row on the right.

[0060] Step b2: If there is no multiple relationship and the horizontal coordinate of the second center point is greater than the horizontal coordinate of the center point of the preset area of ​​interest, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point and the third center point are respectively determined as the second crop row from the left, the second crop row from the right and the first crop row from the right.

[0061] In this step, according to the same principle as described in step b1, if the horizontal coordinate of the second center point is greater than the horizontal coordinate of the center point of the preset area of ​​interest, it means that the navigation line contour corresponding to the second center point is located on the right side, then the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point and the third center point are respectively determined as the second crop row from the left, the second crop row from the right and the first crop row from the right. In this way, the missing crop row can be determined to be the first crop row from the left.

[0062] Furthermore, S102 also includes:

[0063] If there are two crop row contours, the target crop row corresponding to each crop row contour is determined based on the horizontal coordinate of the center point of the circumscribed rectangle of each crop row contour in the interval where the preset area of ​​interest is located; the preset area of ​​interest is divided into four intervals according to a preset ratio, and each interval corresponds to the first crop row on the left, the second crop row on the left, the second crop row on the right and the first crop row on the right in the order of the horizontal coordinate from small to large.

[0064] In this step, since the crop row contours cannot be compared and referenced with each other, the crop row contours are marked in the interval where the preset region of interest is located according to the horizontal coordinates of the center of the circumscribed rectangle of the crop row contour. Specifically, the preset region of interest is divided into four intervals according to a preset ratio, and each interval corresponds to the first crop row on the left, the second crop row on the left, the second crop row on the right, and the first crop row on the right in the order of the horizontal coordinates from small to large. Specifically, for example, the width of the preset region of interest is divided into four intervals according to the ratio of 0.39:0.11:0.11:0.39, and the interval with the horizontal coordinate of the center point at 0-0.39 times the width corresponds to the first crop row on the left, the interval at 0.39-0.50 times the width corresponds to the second crop row on the left, the interval at 0.50-0.61 times the width corresponds to the second crop row on the right, and the interval at 0.61-1 times the width corresponds to the first crop row on the right.

[0065] Furthermore, S102 also includes:

[0066] If the number of crop row outlines is less than two, a prompt message indicating that no crop row was detected is output.

[0067] In this step, if the number of crop row contours is less than two, the missing crop row data cannot be supplemented due to lack of reference, and a prompt message indicating that the crop row was not detected is output to remind the user to take over manually. Furthermore, the agricultural machinery navigation line of the previous crop row image can be used to continue navigation for a period of time to try to detect the crop row again. If the crop row is not detected after the crop row image is acquired for a preset number of times, the agricultural machinery navigation is stopped and a prompt message is output.

[0068] Furthermore, S104 specifically includes:

[0069] Step c1, divide the target crop rows into two groups according to the first crop row on the left and the first crop row on the right, and the second crop row on the left and the second crop row on the right, and calculate the midpoint coordinates of the points with the same vertical coordinates in the crop row detection line information corresponding to the target crop rows in each group, and determine the starting point coordinates and the end point coordinates of the reference navigation line of each group according to the midpoint coordinates.

[0070] In this step, after determining the crop row detection lines of each target crop row, the target crop rows are divided into two groups according to the first crop row on the left and the first crop row on the right, and the second crop row on the left and the second crop row on the right, and the midpoint coordinates of the points with the same vertical coordinates in the crop row detection line information corresponding to the target crop rows in each group are calculated, and the starting point coordinates and the end point coordinates of the reference navigation line of each group are determined according to the midpoint coordinates.

[0071] Step c2, weighted summing the starting point coordinates and the ending point coordinates of the reference navigation line of each group according to the preset reference weights of the target crop rows of each group, to obtain the starting point coordinates and the ending point coordinates of the agricultural machinery navigation line corresponding to the crop row image.

[0072] In this step, the starting point coordinates and the end point coordinates of the two generated reference navigation lines are weighted and summed according to the preset reference weights of the target crop rows of each group, and the starting point coordinates and the end point coordinates of the agricultural machinery navigation line corresponding to the crop row image are obtained. Among them, the weight of the group of the second crop row on the left and the second crop row on the right accounts for 0.8, which is the main weight, and the weight of the group of the first crop row on the left and the first crop row on the right accounts for 0.2, which is the secondary weight.

[0073] Step c3, determining the agricultural machinery navigation line corresponding to the crop row image according to the starting point coordinates and the end point coordinates of the agricultural machinery navigation line corresponding to the crop row image.

[0074] In this step, the starting point coordinates and the end point coordinates of the final agricultural machinery navigation line calculated in the above manner are used to generate the agricultural machinery navigation line in the crop row image through OPENCV.

[0075] Furthermore, after S104, the method further includes:

[0076] The agricultural machinery driving line corresponding to the crop row image is obtained, and based on the position coordinates of the agricultural machinery navigation line and the agricultural machinery driving line in the first image preview area and the second image preview area, the lateral deviation and the heading deviation of the agricultural machinery driving line and the agricultural machinery navigation line are determined, so that the agricultural machinery adjusts the driving direction according to the lateral deviation and the heading deviation and drives according to the agricultural machinery navigation line; the first image preview area and the second image preview area are respectively the upper boundary line and the lower boundary line of the preset area of ​​interest.

[0077] Specifically, see Fig. 9 , Fig. 9 Schematic diagram of crop row detection line, agricultural machinery navigation line and agricultural machinery driving line in the embodiment of the present application. Fig. 9 As shown, in this step, after the agricultural machinery navigation line is determined, two points of the agricultural machinery navigation line and the agricultural machinery driving line on the first image preview area and the second image preview area on the upper and lower boundaries of the preset area of ​​interest are selected, and the slope of the center navigation line and the driving direction driving line slope can be obtained by using the slope calculation formula, and then the driving heading deviation can be obtained by the inverse cosine function; by calculating the image coordinates of the agricultural machinery navigation line in the first preview area and the image coordinates of the agricultural machinery driving line in the first preview area, the horizontal coordinate difference of the two coordinate positions is determined as the lateral deviation, so that the agricultural machinery adjusts the driving direction according to the lateral deviation and the heading deviation, and drives according to the agricultural machinery navigation line. Among them, the agricultural machinery driving line is the center line of the crop row image in the vertical direction, representing the actual driving direction of the agricultural machinery.

[0078] The embodiment of the present application provides a method for determining an agricultural machinery navigation line, comprising: performing image segmentation on a crop row image acquired from an image acquisition device based on a preset image segmentation detection model and a preset region of interest to obtain a background and a plurality of crop row contours; determining a target crop row corresponding to each crop row contour based on the number of crop row contours and the horizontal coordinates of the center points of the circumscribed rectangles of each crop row contour; the target crop row is a reference crop row for the agricultural machinery navigation line, and includes, in order from left to right, the first crop row on the left, the second crop row on the left, the second crop row on the right, and the first crop row on the right; if the number of crop row contours is less than four, determining the crop row in the crop row image; If there are missing target crop rows, the crop row detection line information corresponding to the missing target crop rows is obtained from the temporary crop row data storage list based on the crop row detection line information of each crop row contour; the crop row detection line information includes the center point coordinates of a group of points with the largest ordinate in the crop row contour and the center point coordinates of a group of points with the smallest ordinate; the temporary crop row data storage list stores the crop row detection line information corresponding to each target crop row in the historical crop row image with four crop row contours; based on the crop row detection line information corresponding to each target crop row and the preset reference weight, the agricultural machinery navigation line corresponding to the crop row image is determined. In this way, in the scenario of missing crop rows, the missing crop rows are completed by analyzing the detected crop rows, and the navigation line is generated according to the completed crop rows, so as to improve the accuracy of determining the agricultural machinery navigation line.

[0079] Based on the same application concept, the embodiment of the present application also provides a device for determining the agricultural machinery navigation line corresponding to the method for determining the agricultural machinery navigation line provided in the above embodiment. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the method for determining the agricultural machinery navigation line in the above embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0080] See also Fig.10 , Fig.10 This is one of the functional module diagrams of a device for determining an agricultural machinery navigation line provided in an embodiment of the present application. Fig.10 As shown, the device 1000 for determining the agricultural machinery navigation line includes:

[0081] The image segmentation module 1010 is used to perform image segmentation on the crop row image acquired from the image acquisition device based on a preset image segmentation detection model and a preset region of interest to obtain a background and multiple crop row contours.

[0082] The first determination module 1020 is used to determine the target crop rows corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinates of the center points of the circumscribed rectangles of each crop row outline; the target crop rows are reference crop rows of the agricultural machinery navigation line, and include, in order from left to right, the first crop row on the left, the second crop row on the left, the second crop row on the right and the first crop row on the right.

[0083] The information acquisition module 1030 is used to determine whether there are missing target crop rows in the crop row image if the number of crop row contours is less than four, and obtain the crop row detection line information corresponding to the missing target crop rows from a temporary crop row data storage list based on the crop row detection line information of each of the crop row contours; the crop row detection line information includes the center point coordinates of a group of points with the largest ordinate and the center point coordinates of a group of points with the smallest ordinate in the crop row contour; the temporary crop row data storage list stores the crop row detection line information corresponding to each of the target crop rows in the historical crop row image with four crop row contours.

[0084] The second determination module 1040 is used to determine the agricultural machinery navigation line corresponding to the crop row image based on the crop row detection line information corresponding to each of the target crop rows and a preset reference weight.

[0085] Further, when the first determination module 1020 is used to determine the target crop row corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinate of the center point of the circumscribed rectangle of each crop row outline, the first determination module 1020 is specifically used to:

[0086] If there are four crop row contours, the target crop rows corresponding to the respective crop row contours are respectively determined as the first crop row on the left, the second crop row on the left, the second crop row on the right and the first crop row on the right in ascending order of the horizontal coordinates of the center points of the circumscribed rectangles of the respective crop row contours.

[0087] Furthermore, when the first determining module 1020 is used to determine the target crop row corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinate of the center point of the circumscribed rectangle of each crop row outline, the first determining module 1020 is also used to:

[0088] If there are three crop row contours, determine the first center point, the second center point, and the third center point according to the ascending order of the horizontal coordinates of the center points of the circumscribed rectangles of the crop row contours, and determine whether there is a multiple relationship between a first distance between the first center point and the horizontal coordinate of the second center point and a second distance between the second center point and the horizontal coordinate of the third center point;

[0089] If there is a multiple relationship, and the absolute value of the difference between the second distance and twice the first distance is less than or equal to a preset distance threshold, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point, and the third center point are respectively determined as the first crop row on the left, the second crop row on the left, and the first crop row on the right;

[0090] If there is a multiple relationship, and the absolute value of the difference between the first distance and twice the second distance is less than or equal to the preset distance threshold, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point and the third center point are respectively determined as the left crop row, the right second crop row and the right crop row.

[0091] Furthermore, when the first determining module 1020 is used to determine the target crop row corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinate of the center point of the circumscribed rectangle of each crop row outline, the first determining module 1020 is also used to:

[0092] If there is no multiple relationship, and the abscissa of the second center point is smaller than the abscissa of the center point of the preset region of interest, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point, and the third center point are respectively determined as the first crop row from the left, the second crop row from the left, and the second crop row from the right;

[0093] If there is no multiple relationship, and the horizontal coordinate of the second center point is greater than the horizontal coordinate of the center point of the preset area of ​​interest, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point and the third center point are respectively determined as the second crop row from the left, the second crop row from the right and the first crop row from the right.

[0094] Furthermore, when the first determining module 1020 is used to determine the target crop row corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinate of the center point of the circumscribed rectangle of each crop row outline, the first determining module 1020 is also used to:

[0095] If there are two crop row contours, the target crop row corresponding to each crop row contour is determined based on the horizontal coordinate of the center point of the circumscribed rectangle of each crop row contour in the interval where the preset area of ​​interest is located; the preset area of ​​interest is divided into four intervals according to a preset ratio, and each interval corresponds to the first crop row on the left, the second crop row on the left, the second crop row on the right and the first crop row on the right in the order of the horizontal coordinate from small to large.

[0096] Furthermore, when the first determining module 1020 is used to determine the target crop row corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinate of the center point of the circumscribed rectangle of each crop row outline, the first determining module 1020 is also used to:

[0097] If the number of crop row outlines is less than two, a prompt message indicating that no crop row was detected is output.

[0098] Further, when the second determination module 1040 is used to determine the agricultural machinery navigation line corresponding to the crop row image based on the crop row detection line information corresponding to each of the target crop rows and the preset reference weight, the second determination module 1040 is specifically used to:

[0099] Divide the target crop rows into two groups according to the first crop row on the left and the first crop row on the right, and the second crop row on the left and the second crop row on the right, and calculate the midpoint coordinates of the points with the same ordinate in the crop row detection line information corresponding to the target crop rows of each group, and determine the starting point coordinates and the end point coordinates of the reference navigation line of each group according to the midpoint coordinates;

[0100] The starting point coordinates and the end point coordinates of the reference navigation line of each group are weighted and summed according to the preset reference weights of the target crop rows of each group to obtain the starting point coordinates and the end point coordinates of the agricultural machinery navigation line corresponding to the crop row image;

[0101] The agricultural machinery navigation line corresponding to the crop row image is determined according to the starting point coordinates and the ending point coordinates of the agricultural machinery navigation line corresponding to the crop row image.

[0102] For further information, see Fig.11 , Fig.11 This is a second functional module diagram of a device for determining an agricultural machinery navigation line provided in an embodiment of the present application. Fig.11 As shown, the agricultural machinery navigation line determination device 1000 also includes:

[0103] The third determination module 1050 is used to obtain the agricultural machinery driving line corresponding to the crop row image, and determine the lateral deviation and heading deviation of the agricultural machinery driving line and the agricultural machinery navigation line based on the position coordinates of the agricultural machinery navigation line and the agricultural machinery driving line in the first image preview area and the second image preview area, so that the agricultural machinery adjusts the driving direction according to the lateral deviation and the heading deviation and drives according to the agricultural machinery navigation line; the first image preview area and the second image preview area are respectively the upper boundary line and the lower boundary line of the preset area of ​​interest.

[0104] The embodiment of the present application provides a device for determining an agricultural machinery navigation line, including: an image segmentation module, which is used to perform image segmentation on a crop row image acquired from an image acquisition device based on a preset image segmentation detection model and a preset region of interest, so as to obtain a background and a plurality of crop row contours; a first determination module, which is used to determine a target crop row corresponding to each crop row contour based on the number of crop row contours and the horizontal coordinates of the center points of the circumscribed rectangles of each crop row contour; the target crop rows are reference crop rows for the agricultural machinery navigation line, which respectively include the first crop row on the left, the second crop row on the left, the second crop row on the right and the first crop row on the right in order from left to right; an information acquisition module, which is used to obtain a reference crop row on the agricultural machinery navigation line if the number of crop row contours is less than four. , determine if there are missing target crop rows in the crop row image, and based on the crop row detection line information of each crop row contour, obtain the crop row detection line information corresponding to the missing target crop row from the temporary crop row data storage list; the crop row detection line information includes the center point coordinates of a group of points with the largest ordinate in the crop row contour and the center point coordinates of a group of points with the smallest ordinate; the temporary crop row data storage list stores the crop row detection line information corresponding to each target crop row in the historical crop row image with four crop row contours; the second determination module is used to determine the agricultural machinery navigation line corresponding to the crop row image based on the crop row detection line information corresponding to each target crop row and the preset reference weight. In this way, in the scenario of missing crop rows, the missing crop rows can be supplemented by analyzing the detected crop rows, and the navigation line is generated according to the supplemented crop rows, so as to improve the accuracy of determining the agricultural machinery navigation line.

[0105] Based on the same application concept, an embodiment of the present application also provides an agricultural machinery, which includes an image acquisition device and a device for determining the agricultural machinery navigation line provided in the above embodiment; wherein the image acquisition device is used to send the acquired crop row image to the device for determining the agricultural machinery navigation line; the device for determining the agricultural machinery navigation line adopts the method for determining the agricultural machinery navigation line provided in the above embodiment to determine the agricultural machinery navigation line, and the repeated parts will not be repeated.

[0106] Based on the same application idea, please refer to Fig.12 , Fig.12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.12 As shown, the electronic device 1200 includes a processor 1210 , a memory 1220 and a bus 1230 .

[0107] The memory 1220 stores machine-readable instructions executable by the processor 1210. When the electronic device 1200 is running, the processor 1210 communicates with the memory 1220 through the bus 1230. When the processor 1210 is running, the machine-readable instructions execute the steps of the method for determining the agricultural machinery navigation line as described above. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0108] Based on the same application concept, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for determining the agricultural machinery navigation line as described above are executed. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0109] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0112] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0113] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0114] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for determining an agricultural machinery navigation line, characterized in that: The method comprises: Based on a preset image segmentation detection model and a preset region of interest, the crop row image acquired from the image acquisition device is segmented to obtain a background and multiple crop row contours; Based on the number of crop row contours and the horizontal coordinates of the center points of the circumscribed rectangles of the crop row contours, the target crop rows corresponding to the crop row contours are determined; the target crop rows are reference crop rows for the agricultural machinery navigation line, and in order from left to right, they include the first crop row on the left, the second crop row on the left, the second crop row on the right, and the first crop row on the right; If the number of crop row contours is less than four, it is determined that there are missing target crop rows in the crop row image, and based on the crop row detection line information of each of the crop row contours, the crop row detection line information corresponding to the missing target crop row is obtained from a temporary crop row data storage list; the crop row detection line information includes the center point coordinates of a group of points with the largest ordinates and the center point coordinates of a group of points with the smallest ordinates in the crop row contour; the temporary crop row data storage list stores the crop row detection line information corresponding to each of the target crop rows in the historical crop row image with four crop row contours; Based on the crop row detection line information corresponding to each of the target crop rows and the preset reference weight, the agricultural machinery navigation line corresponding to the crop row image is determined.

2. The method for determining the agricultural machinery navigation line according to claim 1, characterized in that: The step of determining the target crop row corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinates of the center points of the circumscribed rectangles of each crop row outline comprises: If there are four crop row contours, the target crop rows corresponding to the respective crop row contours are respectively determined as the first crop row on the left, the second crop row on the left, the second crop row on the right and the first crop row on the right in ascending order of the horizontal coordinates of the center points of the circumscribed rectangles of the respective crop row contours.

3. The method for determining the agricultural machinery navigation line according to claim 1, characterized in that: The step of determining the target crop row corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinates of the center points of the circumscribed rectangles of each crop row outline further includes: If there are three crop row contours, determine the first center point, the second center point, and the third center point according to the ascending order of the horizontal coordinates of the center points of the circumscribed rectangles of the crop row contours, and determine whether there is a multiple relationship between a first distance between the first center point and the horizontal coordinate of the second center point and a second distance between the second center point and the horizontal coordinate of the third center point; If there is a multiple relationship, and the absolute value of the difference between the second distance and twice the first distance is less than or equal to a preset distance threshold, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point, and the third center point are respectively determined as the first crop row on the left, the second crop row on the left, and the first crop row on the right; If there is a multiple relationship, and the absolute value of the difference between the first distance and twice the second distance is less than or equal to the preset distance threshold, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point and the third center point are respectively determined as the left crop row, the right second crop row and the right crop row.

4. The method for determining the agricultural machinery navigation line according to claim 3, characterized in that: The method further comprises: If there is no multiple relationship, and the abscissa of the second center point is smaller than the abscissa of the center point of the preset region of interest, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point, and the third center point are respectively determined as the first crop row from the left, the second crop row from the left, and the second crop row from the right; If there is no multiple relationship, and the horizontal coordinate of the second center point is greater than the horizontal coordinate of the center point of the preset area of ​​interest, the target crop rows corresponding to the crop row contours corresponding to the first center point, the second center point and the third center point are respectively determined as the second crop row from the left, the second crop row from the right and the first crop row from the right.

5. The method for determining the agricultural machinery navigation line according to claim 1, characterized in that: The step of determining the target crop row corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinates of the center points of the circumscribed rectangles of each crop row outline further includes: If there are two crop row contours, the target crop row corresponding to each crop row contour is determined based on the horizontal coordinate of the center point of the circumscribed rectangle of each crop row contour in the interval where the preset area of ​​interest is located; the preset area of ​​interest is divided into four intervals according to a preset ratio, and each interval corresponds to the first crop row on the left, the second crop row on the left, the second crop row on the right and the first crop row on the right in the order of the horizontal coordinate from small to large.

6. The method for determining the agricultural machinery navigation line according to claim 1, characterized in that: The step of determining the target crop row corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinates of the center points of the circumscribed rectangles of each crop row outline further includes: If the number of crop row outlines is less than two, a prompt message indicating that no crop row was detected is output.

7. The method for determining the agricultural machinery navigation line according to claim 1, characterized in that: The determining the agricultural machinery navigation line corresponding to the crop row image based on the crop row detection line information corresponding to each target crop row and the preset reference weight includes: Divide the target crop rows into two groups according to the first crop row on the left and the first crop row on the right, and the second crop row on the left and the second crop row on the right, and calculate the midpoint coordinates of the points with the same ordinate in the crop row detection line information corresponding to the target crop rows of each group, and determine the starting point coordinates and the end point coordinates of the reference navigation line of each group according to the midpoint coordinates; The starting point coordinates and the end point coordinates of the reference navigation line of each group are weighted and summed according to the preset reference weights of the target crop rows of each group to obtain the starting point coordinates and the end point coordinates of the agricultural machinery navigation line corresponding to the crop row image; The agricultural machinery navigation line corresponding to the crop row image is determined according to the starting point coordinates and the ending point coordinates of the agricultural machinery navigation line corresponding to the crop row image.

8. The method for determining the agricultural machinery navigation line according to claim 1, characterized in that: After determining the agricultural machinery navigation line corresponding to the crop row image based on the crop row detection line information corresponding to each of the target crop rows and the preset reference weight, the method further includes: The agricultural machinery driving line corresponding to the crop row image is obtained, and based on the position coordinates of the agricultural machinery navigation line and the agricultural machinery driving line in the first image preview area and the second image preview area, the lateral deviation and the heading deviation of the agricultural machinery driving line and the agricultural machinery navigation line are determined, so that the agricultural machinery adjusts the driving direction according to the lateral deviation and the heading deviation and drives according to the agricultural machinery navigation line; the first image preview area and the second image preview area are respectively the upper boundary line and the lower boundary line of the preset area of ​​interest.

9. A device for determining a navigation line of an agricultural machinery, characterized in that: The determination of the agricultural machinery navigation line includes: An image segmentation module is used to perform image segmentation on the crop row image acquired from the image acquisition device based on a preset image segmentation detection model and a preset region of interest to obtain a background and multiple crop row contours; A first determination module is used to determine the target crop rows corresponding to each crop row outline based on the number of crop row outlines and the horizontal coordinates of the center points of the circumscribed rectangles of each crop row outline; the target crop rows are reference crop rows for the agricultural machinery navigation line, and include, in order from left to right, the first crop row on the left, the second crop row on the left, the second crop row on the right, and the first crop row on the right; an information acquisition module, for determining, if the number of crop row contours is less than four, whether there are missing target crop rows in the crop row image, and acquiring, based on the crop row detection line information of each of the crop row contours, crop row detection line information corresponding to the missing target crop rows from a temporary crop row data storage list; the crop row detection line information includes the center point coordinates of a group of points with the largest ordinates and the center point coordinates of a group of points with the smallest ordinates in the crop row contour; the temporary crop row data storage list stores the crop row detection line information corresponding to each of the target crop rows in the historical crop row image with four crop row contours; The second determination module is used to determine the agricultural machinery navigation line corresponding to the crop row image based on the crop row detection line information corresponding to each target crop row and a preset reference weight.

10. An agricultural machine, characterized in that: It comprises an image acquisition device and a device for determining the agricultural machinery navigation line according to claim 9; The image acquisition device is used to send the acquired crop row image to the agricultural machinery navigation line determination device.

Images