A method and system for agricultural machinery navigation based on computer vision

Abstract

The present invention discloses a method and system for agricultural machinery navigation based on computer vision, and the method includes: image acquisition, farmland image processing, agricultural machinery path planning and agricultural machinery navigation. The present invention belongs to the field of navigation technology, and specifically refers to a method and system for agricultural machinery navigation based on computer vision. The scheme introduces enhancement coefficient and gamma correction to calculate detail enhanced image, calculates reflection component based on brightness adjustment amount and brightness enhancement coefficient, completes low-frequency fusion based on saliency, completes high-frequency fusion based on PA-PCNN, obtains farmland enhanced image, and constructs farmland map; calculates fitness value based on path length and path smoothness, designs path planning entropy difference and adaptive threshold, introduces expansion coefficient and acceleration factor to update individual position in segments, performs position detection based on obstacles and farmland boundaries, determines optimal path planning scheme, and improves the accuracy and work efficiency of agricultural machinery navigation.

Description

Technical Field

[0001] The present invention belongs to the field of navigation technology, and specifically refers to a method and system for agricultural machinery navigation based on computer vision. Background Art

[0002] The agricultural machinery navigation method uses advanced computer vision algorithms for analysis and processing to provide navigation guidance for agricultural machinery, so that agricultural machinery can operate efficiently and accurately according to the preset route, and improve the automation and accuracy of agricultural machinery operations. However, the existing agricultural machinery navigation methods have the problem that the quality of farmland images varies greatly under different lighting conditions, the details of farmland images are unclear and the edges are blurred, resulting in inaccurate identification of farmland boundaries and obstacles; the existing agricultural machinery navigation methods have the problem that the farmland environment is complex and diverse, and obstacles in the field cannot be effectively avoided. The path planning is not adaptable enough and the speed is slow, resulting in inaccurate agricultural machinery navigation and low work efficiency. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a method and system for agricultural machinery navigation based on computer vision. In view of the problem that the quality of farmland images under different lighting conditions is greatly different, the details of farmland images are unclear and the edges are blurred, resulting in inaccurate recognition of farmland boundaries and obstacles in the existing agricultural machinery navigation methods, this solution introduces enhancement coefficients and gamma correction to calculate detail enhanced images, which helps to more accurately identify subtle features in the farmland; edge enhancement is performed based on Gaussian filtering, and the reflection component is calculated based on the brightness adjustment amount and the brightness enhancement coefficient, and the edge enhanced image is calculated, so that the image can present a more appropriate state under different lighting conditions, and the edge information is better highlighted; NSST is used for image decomposition, low-frequency fusion is completed based on saliency, high-frequency fusion is completed based on PA-PCNN, and then inverse NSST processing is performed to obtain a farmland enhanced image, and a farmland map is constructed. The map can provide an accurate and clear image basis for the identification of farmland boundaries and obstacles, thereby ensuring the reliability and accuracy of agricultural machinery navigation. In view of the complex and diverse farmland environment in existing agricultural machinery navigation methods, the obstacles in the field cannot be effectively avoided, the path planning is not adaptable enough and the speed is slow, resulting in inaccurate agricultural machinery navigation and low work efficiency. This scheme uses the population to represent the path planning scheme, calculates the fitness value based on the path length and path smoothness, designs the path planning entropy and adaptive threshold, which is conducive to balancing exploration and utilization and enhancing the adaptability in complex environments. The expansion coefficient and acceleration factor are introduced to update the individual position in segments, which improves the convergence speed and planning efficiency. Position detection is performed based on obstacles and farmland boundaries to determine the optimal path planning scheme, and the navigation path deviated due to environmental interference is discovered and corrected in time, thereby improving the accuracy and work efficiency of agricultural machinery navigation.

[0004] The technical solution adopted by the present invention is as follows: The present invention provides a method for agricultural machinery navigation based on computer vision, the method comprising the following steps:

[0005] Step S1: image acquisition;

[0006] Step S2: farmland image processing;

[0007] Step S3: agricultural machinery path planning;

[0008] Step S4: Agricultural machinery navigation.

[0009] Furthermore, in step S1, the image acquisition is to acquire historical farmland images and real-time farmland images in RGB mode, and to annotate the historical farmland images, with the annotation types being farmland boundaries and obstacles.

[0010] Furthermore, in step S2, the farmland image processing specifically includes the following steps:

[0011] Step S21: Enhance image details. Decompose the detail image from the farmland image, introduce the enhancement coefficient for preliminary enhancement, and then perform gamma correction to obtain a detail enhanced image. The formula used is as follows:

[0012] ;

[0013] Where A is the detail enhanced image, I is the farmland image, is the enhancement coefficient, is a convolution operation, P is an average filter of size 50×50, γ is the gamma value, and c is a constant;

[0014] Step S22: enhancing the image edge, including the following steps:

[0015] Step S221: Perform edge enhancement based on Gaussian filtering, and the formula used is as follows:

[0016] ;

[0017] In the formula, is the farmland image after edge enhancement, ψ{·} is the normalization operator, and G is the Gaussian filter;

[0018] Step S222: Calculate the brightness value and normalize it. The formula used is as follows:

[0019] ;

[0020] ;

[0021] In the formula, is the brightness value at the coordinate point (x, y), , and are the pixel values ​​of the red channel, green channel, and blue channel at the coordinate point (x, y), respectively. x and y are the horizontal and vertical coordinates of the pixel in the farmland image, respectively. is the normalized brightness value at the coordinate point (x, y), is the maximum value of brightness, is the logarithmic mean brightness value, , N is the total number of pixels, is a smoothing term;

[0022] Step S223: filtering, the formula used is as follows:

[0023] ;

[0024] In the formula, yes The filtered value, Guide is the guided filter, Yes all The maximum value in ;

[0025] Step S224: Design the brightness adjustment amount and brightness enhancement coefficient, the formula used is as follows:

[0026] ;

[0027] ;

[0028] In the formula, δ is the brightness adjustment amount, λ is the nonlinear control parameter, is the normalized logarithmic mean brightness value, , is the brightness enhancement coefficient at the coordinate point (x, y), is the maximum value of the normalized brightness value, and β is the adjustment parameter;

[0029] Step S225: Calculate the reflection component based on the brightness adjustment amount and the brightness enhancement coefficient. The formula used is as follows:

[0030] ;

[0031] In the formula, is the reflected component at the coordinate point (x, y);

[0032] Step S226: Calculate the edge enhanced image B using the following formula:

[0033] ;

[0034] In the formula, is the pixel value of the edge enhanced image at the coordinate point (x, y);

[0035] Step S23: image decomposition, decomposing the detail enhanced image A and the edge enhanced image B based on non-subsampled shearlet transform NSST to obtain a low frequency component A1 and a high frequency component A2 of the detail enhanced image, and a low frequency component B1 and a high frequency component B2 of the edge enhanced image, respectively;

[0036] Step S24: low-frequency fusion, weighted fusion of the low-frequency component A1 of the detail-enhanced image and the low-frequency component B1 of the edge-enhanced image based on saliency, the formula used is as follows:

[0037] ;

[0038] Where C1 is the low-frequency fusion image, Saliency (·) is the saliency function, and are the significance of A1 and B1 respectively;

[0039] Step S25: high frequency fusion, based on the parameter adaptive pulse coupled neural network PA-PCNN, the high frequency component A2 of the detail enhanced image is fused with the high frequency component B2 of the edge enhanced image to obtain a high frequency fused image C2;

[0040] Step S26: Calculate the farmland enhanced image, perform inverse NSST processing on the low-frequency fusion image C1 and the high-frequency fusion image C2, and obtain the farmland enhanced image C;

[0041] Step S27: Construct a farmland map. The farmland enhancement image includes a historical farmland enhancement image and a real-time farmland enhancement image. A U-Net model is trained based on the historical farmland enhancement image. The trained U-Net model is then used to process the real-time farmland enhancement image to identify the farmland boundaries and obstacles, and digitally represent them to obtain a farmland map in a raster format.

[0042] Furthermore, in step S3, the agricultural machinery path planning specifically includes the following steps:

[0043] Step S31: Initialization, setting the starting point and the end point of the path based on the farmland map, and performing path planning between the starting point and the end point. Each path planning scheme includes D navigation points, and the individual position is used as the representative of the navigation point. The D individual positions form a population, and the population is used as the representative of the path planning scheme. H populations are randomly generated within the farmland boundary to avoid obstacles;

[0044] Step S32: Calculate the fitness value. The fitness value of the population is calculated based on the path length and path smoothness. The formula used is as follows:

[0045] ;

[0046] ;

[0047] ;

[0048] Where, L h is the fitness value of the hth population, h is the population index, ω1 and ω2 are the path length weight and path smoothness weight respectively, and are the path length and path smoothness corresponding to the hth population, Q star and Q end The starting point and the end point respectively. , , , , , and are the 1st, 2nd, d-1th, dth, d+1th, D-1th, and Dth individual positions in the hth population, d is the individual position index, |·| is the vector length, and arccos(·) is the inverse cosine function;

[0049] Step S33: Design path planning entropy, the formula used is as follows:

[0050] ;

[0051] ;

[0052] Where s(t) and s(t-1) are the path planning entropies at the t-th and t-1-th iterations, respectively, t is the iteration index, is the path planning entropy difference at the tth iteration, and are the fitness values ​​of the hth population at the tth and t-1th iterations, respectively. and are the fitness values ​​of the mth population at the tth and t-1th iterations, respectively, where m is the population index and α is the adjustment factor;

[0053] Step S34: Calculate the adaptive threshold using the following formula:

[0054] ;

[0055] Where μ(t) is the adaptive threshold at the tth iteration, is the average fitness value of all populations at the tth iteration;

[0056] Step S35: Population update, introducing expansion coefficient and acceleration factor to update the individual positions in the population in sections, the formula used is as follows:

[0057] ;

[0058] wherein, and are the positions of the d-th individual in the h-th population at the (t + 1)-th and t-th iterations respectively, d is the individual position index, p max and p min are the maximum and minimum values of the expansion coefficient respectively, T is the maximum number of iterations, is the position of the d-th individual in the optimal population at the t-th iteration, S0 is the initial value of the acceleration factor, t c is a constant value, ρ is the attenuation rate, r1 and r2 are the first random number and the second random number respectively;

[0059] Step S36: Position detection. Predetermine the first detection threshold J1 and the second detection threshold J2, and calculate the minimum Euclidean distance J 1min between the position of each individual in the H populations and the nearest obstacle. If there exists a J 1min < J1 in population h, then go to step S35 to update population h again; calculate the minimum Euclidean distance J 2min between the position of each individual in the H populations and the farmland boundary. If there exists a J 2min < J2 in population h, then go to step S35 to update population h again; otherwise, go to step S37;

[0060] Step S37: Determine the optimal path planning scheme. Predetermine the fitness threshold, update the fitness values of the populations. When the fitness value of the optimal population is lower than the fitness threshold, determine the path planning scheme represented by the optimal population as the optimal path planning scheme; otherwise, if the maximum number of iterations is reached, return to step S31 to re-initialize; otherwise, increment the number of iterations by 1 and return to step S33 to continue the iteration.

[0061] Furthermore, in step S4, the agricultural machinery navigation is based on the D navigation points in the optimal path planning scheme and the starting point and the ending point to obtain the complete optimal path, and send the optimal path to the agricultural machinery control system to realize the automatic navigation of the agricultural machinery.

[0062] An agricultural machinery navigation system based on computer vision provided by the present invention includes an image acquisition module, a farmland image processing module, an agricultural machinery path planning module, and an agricultural machinery navigation module;

[0063] The image acquisition module acquires historical farmland images and real-time farmland images, and sends the data to the farmland image processing module;

[0064] The farmland image processing module introduces enhancement coefficients and gamma correction to calculate detail enhanced images, performs edge enhancement based on Gaussian filtering, calculates reflection components based on brightness adjustment and brightness enhancement coefficients, calculates edge enhanced images, uses NSST to decompose images, completes low-frequency fusion based on saliency, completes high-frequency fusion based on PA-PCNN, and then performs inverse NSST processing to obtain farmland enhanced images, construct farmland maps, and send data to the agricultural machinery path planning module;

[0065] The agricultural machinery path planning module uses populations to represent path planning solutions, calculates fitness values ​​based on path length and path smoothness, designs path planning entropy and adaptive thresholds, introduces expansion coefficients and acceleration factors to update individual positions in segments, performs position detection based on obstacles and farmland boundaries, determines the optimal path planning solution, and sends data to the agricultural machinery navigation module;

[0066] The agricultural machinery navigation module sends the optimal path to the agricultural machinery control system to realize automatic navigation of the agricultural machinery.

[0067] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0068] (1) In view of the problem that the quality of farmland images under different lighting conditions varies greatly in the existing agricultural machinery navigation methods, the details of farmland images are unclear and the edges are blurred, which leads to inaccurate recognition of farmland boundaries and obstacles. This scheme introduces enhancement coefficients and gamma correction to calculate detail enhanced images, making the image details clearer, which helps to more accurately identify subtle features in the farmland; edge enhancement is performed based on Gaussian filtering, and the reflection component is calculated based on the brightness adjustment amount and the brightness enhancement coefficient. The edge enhanced image is calculated so that the image can present a more appropriate state under different lighting conditions, while better highlighting the edge information and enhancing the image recognition; NSST is used for image decomposition, low-frequency fusion is completed based on saliency, high-frequency fusion is completed based on PA-PCNN, and then inverse NSST processing is performed to obtain a farmland enhanced image, and a farmland map is constructed to effectively extract and integrate different features, improve image quality and fusion effect, and better provide an accurate and clear image basis for farmland boundary and obstacle recognition, thereby ensuring the reliability and accuracy of agricultural machinery navigation.

[0069] (2) In view of the problems in existing agricultural machinery navigation methods, such as the complex and diverse farmland environment, the inability to effectively avoid obstacles in the field, the lack of adaptability and slow speed of path planning, which lead to inaccurate agricultural machinery navigation and low work efficiency, this scheme uses a population to represent the path planning scheme, calculates the fitness value based on the path length and path smoothness, designs the path planning entropy and adaptive threshold, which is conducive to balancing exploration and utilization and enhancing the adaptability in complex environments; and introduces expansion coefficients and acceleration factors to update individual positions in segments, which improves the convergence speed and planning efficiency; performs position detection based on obstacles and farmland boundaries, determines the optimal path planning scheme, and promptly discovers and corrects the navigation path deviated due to environmental interference, thereby improving the accuracy and work efficiency of agricultural machinery navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A flowchart of a method for agricultural machinery navigation based on computer vision provided by the present invention;

[0071] Figure 2 A schematic diagram of an agricultural machinery navigation system based on computer vision provided by the present invention;

[0072] Figure 3 is a schematic flow chart of step S2;

[0073] Figure 4 It is a schematic diagram of the process of step S3.

[0074] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0076] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0077] Example 1, see Figure 1The present invention provides a method for agricultural machinery navigation based on computer vision, the method comprising the following steps:

[0078] Step S1: image acquisition, collecting historical farmland images and real-time farmland images;

[0079] Step S2: farmland image processing, introducing enhancement coefficient and gamma correction to calculate detail enhanced image, edge enhancement based on Gaussian filtering, and calculating reflection component based on brightness adjustment amount and brightness enhancement coefficient, edge enhanced image, using NSST to decompose image, completing low-frequency fusion based on saliency, completing high-frequency fusion based on PA-PCNN, and then performing inverse NSST processing to obtain farmland enhanced image and construct farmland map;

[0080] Step S3: Agricultural machinery path planning, using populations to represent path planning schemes, calculating fitness values ​​based on path length and path smoothness, designing path planning entropy and adaptive thresholds, and introducing expansion coefficients and acceleration factors to update individual positions in segments, performing position detection based on obstacles and farmland boundaries, and determining the optimal path planning scheme;

[0081] Step S4: Agricultural machinery navigation, sending the optimal path to the agricultural machinery control system to achieve automatic navigation of the agricultural machinery.

[0082] Example 2, see Figure 1 This embodiment is based on the above embodiment. In step S1, the image acquisition is to acquire historical farmland images and real-time farmland images in RGB mode, and to annotate the historical farmland images, and the annotation types are farmland boundaries and obstacles.

[0083] Example 3, see Figure 1 and Figure 3 This embodiment is based on the above embodiment. In step S2, the farmland image processing specifically includes the following steps:

[0084] Step S21: Enhance image details. Decompose the detail image from the farmland image, introduce the enhancement coefficient for preliminary enhancement, and then perform gamma correction to obtain a detail enhanced image. The formula used is as follows:

[0085] ;

[0086] Where A is the detail enhanced image, I is the farmland image, is the enhancement coefficient in the range (0, 1), is a convolution operation, P is an average filter of size 50×50, γ is the gamma value, and c is a constant in the range of (0, 1);

[0087] Step S22: enhancing the image edge, including the following steps:

[0088] Step S221: Perform edge enhancement based on Gaussian filtering, and the formula used is as follows:

[0089] ;

[0090] In the formula, is the farmland image after edge enhancement, ψ{·} is the normalization operator, and G is the Gaussian filter;

[0091] Step S222: Calculate the brightness value and normalize it. The formula used is as follows:

[0092] ;

[0093] ;

[0094] In the formula, is the brightness value at the coordinate point (x, y), , and are the pixel values ​​of the red channel, green channel, and blue channel at the coordinate point (x, y), respectively. x and y are the horizontal and vertical coordinates of the pixel in the farmland image, respectively. is the normalized brightness value at the coordinate point (x, y), is the maximum value of brightness, is the logarithmic mean brightness value, , N is the total number of pixels, is a smoothing term;

[0095] Step S223: filtering, the formula used is as follows:

[0096] ;

[0097] In the formula, yes The filtered value, Guide is the guided filter, Yes all The maximum value in ;

[0098] Step S224: Design the brightness adjustment amount and brightness enhancement coefficient, the formula used is as follows:

[0099] ;

[0100] ;

[0101] In the formula, δ is the brightness adjustment amount, λ is the nonlinear control parameter, , z is a correction factor in the range of (0.1, 1), is the normalized logarithmic mean brightness value, , is the brightness enhancement coefficient at the coordinate point (x, y), and are the maximum and minimum values ​​of the normalized brightness value, and β is an adjustment parameter in the range of (0.5, 1);

[0102] Step S225: Calculate the reflection component based on the brightness adjustment amount and the brightness enhancement coefficient. The formula used is as follows:

[0103] ;

[0104] In the formula, is the reflected component at the coordinate point (x, y);

[0105] Step S226: Calculate the edge enhanced image B using the following formula:

[0106] ;

[0107] In the formula, is the pixel value of the edge enhanced image at the coordinate point (x, y);

[0108] Step S23: image decomposition, decomposing the detail enhanced image A and the edge enhanced image B based on non-subsampled shearlet transform NSST to obtain a low frequency component A1 and a high frequency component A2 of the detail enhanced image, and a low frequency component B1 and a high frequency component B2 of the edge enhanced image, respectively;

[0109] Step S24: low-frequency fusion, weighted fusion of the low-frequency component A1 of the detail-enhanced image and the low-frequency component B1 of the edge-enhanced image based on saliency, the formula used is as follows:

[0110] ;

[0111] Where C1 is the low-frequency fusion image, Saliency (·) is the saliency function, and are the significance of A1 and B1 respectively;

[0112] Step S25: high frequency fusion, based on the parameter adaptive pulse coupled neural network PA-PCNN, the high frequency component A2 of the detail enhanced image is fused with the high frequency component B2 of the edge enhanced image to obtain a high frequency fused image C2;

[0113] Step S26: Calculate the farmland enhanced image, perform inverse NSST processing on the low-frequency fusion image C1 and the high-frequency fusion image C2, and obtain the farmland enhanced image C;

[0114] Step S27: Construct a farmland map. The farmland enhancement image includes a historical farmland enhancement image and a real-time farmland enhancement image. A U-Net model is trained based on the historical farmland enhancement image. The trained U-Net model is then used to process the real-time farmland enhancement image to identify the farmland boundaries and obstacles, and digitally represent them to obtain a farmland map in a raster format.

[0115] By performing the above operations, in view of the problem that the quality of farmland images under different lighting conditions is very different in the existing agricultural machinery navigation methods, the details of farmland images are unclear and the edges are blurred, resulting in inaccurate recognition of farmland boundaries and obstacles, this scheme introduces enhancement coefficients and gamma correction to calculate detail enhanced images, so that the image details are clearer, which helps to more accurately identify subtle features in the farmland; edge enhancement is performed based on Gaussian filtering, and the reflection component is calculated based on the brightness adjustment amount and the brightness enhancement coefficient, and the edge enhanced image is calculated, so that the image can present a more appropriate state under different lighting conditions, while better highlighting the edge information and enhancing the image recognition; NSST is used for image decomposition, low-frequency fusion is completed based on saliency, high-frequency fusion is completed based on PA-PCNN, and then inverse NSST processing is performed to obtain a farmland enhanced image, and a farmland map is constructed to effectively extract and integrate different features, improve image quality and fusion effect, and better provide an accurate and clear image basis for farmland boundary and obstacle recognition, thereby ensuring the reliability and accuracy of agricultural machinery navigation.

[0116] Example 4, see Figure 1 and Figure 4 This embodiment is based on the above embodiment. In step S3, the agricultural machinery path planning specifically includes the following steps:

[0117] Step S31: Initialization, setting the starting point and the end point of the path based on the farmland map, and performing path planning between the starting point and the end point. Each path planning scheme includes D navigation points, and the individual position is used as the representative of the navigation point. The D individual positions form a population, and the population is used as the representative of the path planning scheme. H populations are randomly generated within the farmland boundary to avoid obstacles;

[0118] Step S32: Calculate the fitness value. The fitness value of the population is calculated based on the path length and path smoothness. The formula used is as follows:

[0119] ;

[0120] ;

[0121] ;

[0122] Where, L his the fitness value of the hth population, h is the population index, ω1 and ω2 are the path length weight and path smoothness weight in the range of (0, 1), and ω1+ω2=1, and are the path length and path smoothness corresponding to the hth population, Q star and Q end The starting point and the end point respectively. , , , , , and are the 1st, 2nd, d-1th, dth, d+1th, D-1th, and Dth individual positions in the hth population, d is the individual position index, |·| is the vector length, and arccos(·) is the inverse cosine function;

[0123] Step S33: Design path planning entropy, the formula used is as follows:

[0124] ;

[0125] ;

[0126] Where s(t) and s(t-1) are the path planning entropies at the t-th and t-1-th iterations, respectively, t is the iteration index, is the path planning entropy difference at the tth iteration, and are the fitness values ​​of the hth population at the tth and t-1th iterations, respectively. and are the fitness values ​​of the mth population at the tth and t-1th iterations, respectively, where m is the population index and α is an adjustment factor in the range of (0, 0.3);

[0127] Step S34: Calculate the adaptive threshold using the following formula:

[0128] ;

[0129] Where μ(t) is the adaptive threshold at the tth iteration, is the average fitness value of all populations at the tth iteration;

[0130] Step S35: Population update, introducing expansion coefficient and acceleration factor to update the individual positions in the population in sections, the formula used is as follows:

[0131] ;

[0132] In the formula, and are the positions of the d-th individual in the h-th population at the (t + 1)-th and t-th iterations respectively. d is the individual position index, p max and p min are the maximum and minimum values of the expansion coefficient respectively. T is the maximum number of iterations, is the position of the d-th individual in the optimal population at the t-th iteration. The optimal population is the population with the minimum fitness value. S0 is the initial value of the acceleration factor, t c is a constant value in [0, T), ρ is the decay rate in the range (0.1, 0.5), and r1 and r2 are the first random number and the second random number in the range (0, 1) respectively;

[0133] Step S36: Position detection. Preset the first detection threshold J1 and the second detection threshold J2, and calculate the minimum Euclidean distance J between each individual position in the H populations and the nearest obstacle 1min , if there exists a J 1min < J1 in population h, then go to step S35 to update population h again; calculate the minimum Euclidean distance J between each individual position in the H populations and the farmland boundary 2min , if there exists a J 2min < J2 in population h, then go to step S35 to update population h again; otherwise, go to step S37;

[0134] Step S37: Determine the optimal path planning scheme. Preset the fitness threshold, update the fitness values of the populations. When the fitness value of the optimal population is lower than the fitness threshold, determine the path planning scheme represented by the optimal population as the optimal path planning scheme; otherwise, if the maximum number of iterations is reached, return to step S31 to re-initialize; otherwise, increment the number of iterations by 1 and return to step S33 to continue the iteration.

[0135] By performing the above operations, for the problems in the existing agricultural machinery navigation methods, such as the complex and diverse farmland environment, inability to effectively avoid obstacles in the field, insufficient adaptability and slow planning speed of the existing path planning methods, resulting in inaccurate agricultural machinery navigation and low work efficiency, this solution uses populations to represent path planning schemes, calculates fitness values based on path length and path smoothness, designs path planning entropy and adaptive thresholds, which is beneficial to balance exploration and exploitation and enhances the adaptability in complex environments; and introduces expansion coefficients and acceleration factors to update individual positions in segments, improving the convergence speed and planning efficiency; performs position detection based on obstacles and farmland boundaries to determine the optimal path planning scheme, timely discovers and corrects the navigation path deviated due to environmental interference, and improves the accuracy and work efficiency of agricultural machinery navigation.

[0136] Example Five, refer to Figure 1This embodiment is based on the above embodiment. In step S4, agricultural machinery navigation is based on the D navigation points and the starting point and the end point in the optimal path planning scheme to obtain a complete optimal path, and the optimal path is sent to the agricultural machinery control system to realize automatic navigation of the agricultural machinery.

[0137] Example 6, see Figure 2 , this embodiment is based on the above embodiment, and the present invention provides an agricultural machinery navigation system based on computer vision, including an image acquisition module, a farmland image processing module, an agricultural machinery path planning module and an agricultural machinery navigation module;

[0138] The image acquisition module acquires historical farmland images and real-time farmland images, and sends the data to the farmland image processing module;

[0139] The farmland image processing module introduces enhancement coefficients and gamma correction to calculate detail enhanced images, performs edge enhancement based on Gaussian filtering, calculates reflection components based on brightness adjustment and brightness enhancement coefficients, calculates edge enhanced images, uses NSST to decompose images, completes low-frequency fusion based on saliency, completes high-frequency fusion based on PA-PCNN, and then performs inverse NSST processing to obtain farmland enhanced images, construct farmland maps, and send data to the agricultural machinery path planning module;

[0140] The agricultural machinery path planning module uses populations to represent path planning solutions, calculates fitness values ​​based on path length and path smoothness, designs path planning entropy and adaptive thresholds, introduces expansion coefficients and acceleration factors to update individual positions in segments, performs position detection based on obstacles and farmland boundaries, determines the optimal path planning solution, and sends data to the agricultural machinery navigation module;

[0141] The agricultural machinery navigation module sends the optimal path to the agricultural machinery control system to realize automatic navigation of the agricultural machinery.

[0142] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0143] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

[0144] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A method for agricultural machinery navigation based on computer vision, characterized in that: The method comprises the following steps: Step S1: image acquisition, collecting historical farmland images and real-time farmland images; Step S2: farmland image processing, introducing enhancement coefficient and gamma correction to calculate detail enhanced image, edge enhancement based on Gaussian filtering, and calculating reflection component based on brightness adjustment amount and brightness enhancement coefficient, edge enhanced image, using NSST to decompose image, completing low-frequency fusion based on saliency, completing high-frequency fusion based on PA-PCNN, and then performing inverse NSST processing to obtain farmland enhanced image and construct farmland map; Step S3: Agricultural machinery path planning, calculating the fitness value based on the path length and path smoothness, designing the path planning entropy and adaptive threshold, introducing the expansion coefficient and acceleration factor to update the individual position in segments, performing position detection based on obstacles and farmland boundaries, and determining the optimal path planning scheme; Step S4: agricultural machinery navigation, sending the optimal path to the agricultural machinery control system to achieve automatic navigation of the agricultural machinery; Step S3 includes step S33: designing path planning entropy, and the formula used is as follows: ； ； In the formula, and are the path planning entropies at the t-th and t-1-th iterations, respectively, t is the iteration index, is the path planning entropy difference at the tth iteration, and are the fitness values ​​of the hth population at the tth and t-1th iterations, respectively. and are the fitness values ​​of the mth population at the tth and t-1th iterations, respectively, α is the adjustment factor, H is the population size, h and m are the population indexes; In step S3, the agricultural machinery path planning specifically includes the following steps: Step S31: Initialization, setting the starting point and the end point of the path based on the farmland map, and performing path planning between the starting point and the end point. Each path planning scheme includes D navigation points, and the individual position is used as the representative of the navigation point. The D individual positions form a population, and the population is used as the representative of the path planning scheme. H populations are randomly generated within the farmland boundary to avoid obstacles; Step S32: Calculate the fitness value. The fitness value of the population is calculated based on the path length and path smoothness. The formula used is as follows: ； Where, L h is the fitness value of the hth population, h is the population index, ω1 and ω2 are the path length weight and path smoothness weight respectively, and are the path length and path smoothness corresponding to the h-th population respectively; Step S33: design path planning entropy; Step S34: Calculate the adaptive threshold using the following formula: ； Where μ(t) is the adaptive threshold at the tth iteration, is the average fitness value of all populations at the tth iteration; Step S35: population update; Step S36: Location detection. Predetermine a first detection threshold J1 and a second detection threshold J2, and calculate the minimum Euclidean distance J between the position of each individual in H populations and the nearest obstacle 1min , if there exists a J 1min < J1 in population h, then go to step S35 to update population h again; calculate the minimum Euclidean distance J between the position of each individual in H populations and the farmland boundary 2min , if there exists a J 2min < J2 in population h, then go to step S35 to update population h again; otherwise, go to step S37; Step S37: The optimal path planning scheme is determined, the fitness threshold is pre-set, and the fitness value of the population is updated. When the fitness value of the optimal population is lower than the fitness threshold, the path planning scheme represented by the optimal population is determined as the optimal path planning scheme; otherwise, if the maximum number of iterations is reached, return to step S31 to reinitialize; otherwise, increase the number of iterations by 1 and return to step S33 to continue iterating.

2. The agricultural machinery navigation method based on computer vision according to claim 1, characterized in that: In step S2, the farmland image processing specifically includes the following steps: Step S21: Enhance image details. Decompose the detail image from the farmland image, introduce the enhancement coefficient for preliminary enhancement, and then perform gamma correction to obtain a detail enhanced image. The formula used is as follows: ； Where A is the detail enhanced image, I is the farmland image, is the enhancement coefficient, is a convolution operation, P is an average filter of size 50×50, γ is the gamma value, and c is a constant; Step S22: enhancing the image edge; Step S23: image decomposition, decomposing the detail enhanced image A and the edge enhanced image B based on non-subsampled shearlet transform NSST to obtain a low frequency component A1 and a high frequency component A2 of the detail enhanced image, and a low frequency component B1 and a high frequency component B2 of the edge enhanced image, respectively; Step S24: low-frequency fusion, weighted fusion of the low-frequency component A1 of the detail-enhanced image and the low-frequency component B1 of the edge-enhanced image based on saliency, the formula used is as follows: ； Where C1 is the low-frequency fusion image, Saliency (·) is the saliency function, and are the significance of A1 and B1 respectively; Step S25: high frequency fusion, based on the parameter adaptive pulse coupled neural network PA-PCNN, the high frequency component A2 of the detail enhanced image is fused with the high frequency component B2 of the edge enhanced image to obtain a high frequency fused image C2; Step S26: Calculate the farmland enhanced image, perform inverse NSST processing on the low-frequency fusion image C1 and the high-frequency fusion image C2, and obtain the farmland enhanced image C; Step S27: Construct a farmland map. The farmland enhancement image includes a historical farmland enhancement image and a real-time farmland enhancement image. A U-Net model is trained based on the historical farmland enhancement image. The trained U-Net model is then used to process the real-time farmland enhancement image to identify the farmland boundaries and obstacles, and digitally represent them to obtain a farmland map in a raster format.

3. The agricultural machinery navigation method based on computer vision according to claim 2, characterized in that: In step S22, the image edge enhancement specifically includes the following steps: Step S221: Perform edge enhancement based on Gaussian filtering, and the formula used is as follows: ； In the formula, is the farmland image after edge enhancement, ψ{·} is the normalization operator, and G is the Gaussian filter; Step S222: Calculate the brightness value and normalize it. The formula used is as follows: ； ； In the formula, is the brightness value at the coordinate point (x, y), , and are the pixel values ​​of the red channel, green channel, and blue channel at the coordinate point (x, y), respectively. x and y are the horizontal and vertical coordinates of the pixel in the farmland image, respectively. is the normalized brightness value at the coordinate point (x, y), is the maximum value of brightness, is the logarithmic mean brightness value, , N is the total number of pixels, is a smoothing term; Step S223: filtering, the formula used is as follows: ； In the formula, yes The filtered value, Guide is the guided filter, Yes all The maximum value in ; Step S224: Design the brightness adjustment amount and brightness enhancement coefficient, the formula used is as follows: ； ； In the formula, δ is the brightness adjustment amount, λ is the nonlinear control parameter, is the normalized logarithmic mean brightness value, , is the brightness enhancement coefficient at the coordinate point (x, y), is the maximum value of the normalized brightness value, and β is the adjustment parameter; Step S225: Calculate the reflection component based on the brightness adjustment amount and the brightness enhancement coefficient. The formula used is as follows: ； In the formula, is the reflected component at the coordinate point (x, y); Step S226: Calculate the edge enhanced image B using the following formula: ； In the formula, is the pixel value of the edge enhanced image at the coordinate point (x, y).

4. The agricultural machinery navigation method based on computer vision according to claim 1, characterized in that: In step S35, population update is to introduce expansion coefficient and acceleration factor to update the individual positions in the population in sections, and the formula used is as follows: ； In the formula, and are the dth individual position in the hth population at the t+1th and tth iterations, respectively, d is the individual position index, and p max and p min are the maximum and minimum values ​​of the expansion coefficient, T is the maximum number of iterations, is the position of the dth individual in the optimal population at the tth iteration, S0 is the initial value of the acceleration factor, and t c is a constant value, ρ is the decay rate, r1 and r2 are the first random number and the second random number respectively.

5. The agricultural machinery navigation method based on computer vision according to claim 1, characterized in that: In step S4, the agricultural machinery navigation is based on the D navigation points and the starting point and the end point in the optimal path planning scheme to obtain a complete optimal path, and the optimal path is sent to the agricultural machinery control system to realize automatic navigation of the agricultural machinery.

6. The agricultural machinery navigation method based on computer vision according to claim 1, characterized in that: In step S1, the image acquisition is to acquire historical farmland images and real-time farmland images in RGB mode, and to mark the historical farmland images, with the marking types being farmland boundaries and obstacles.

7. An agricultural machinery navigation system based on computer vision, used to implement an agricultural machinery navigation method based on computer vision as claimed in any one of claims 1 to 6, characterized in that: It includes image acquisition module, farmland image processing module, agricultural machinery path planning module and agricultural machinery navigation module; The image acquisition module acquires historical farmland images and real-time farmland images, and sends the data to the farmland image processing module; The farmland image processing module introduces enhancement coefficients and gamma correction to calculate detail enhanced images, performs edge enhancement based on Gaussian filtering, calculates reflection components based on brightness adjustment and brightness enhancement coefficients, calculates edge enhanced images, uses NSST to decompose images, completes low-frequency fusion based on saliency, completes high-frequency fusion based on PA-PCNN, and then performs inverse NSST processing to obtain farmland enhanced images, construct farmland maps, and send data to the agricultural machinery path planning module; The agricultural machinery path planning module uses populations to represent path planning solutions, calculates fitness values ​​based on path length and path smoothness, designs path planning entropy and adaptive thresholds, introduces expansion coefficients and acceleration factors to update individual positions in segments, performs position detection based on obstacles and farmland boundaries, determines the optimal path planning solution, and sends data to the agricultural machinery navigation module; The agricultural machinery navigation module sends the optimal path to the agricultural machinery control system to realize automatic navigation of the agricultural machinery.

Images