Method and device for extracting paths between ridges of unmanned tractor operation for open-field vegetables

Abstract

The present application provides a method and device for extracting inter-ridge paths during unmanned tractor operation of open-field vegetables, and relates to the field of path extraction technology. The method comprises: determining a target segmentation probability map of an inter-ridge path image based on a semantic segmentation network; the target segmentation probability map indicates the probability that each pixel point contained in the inter-ridge path image belongs to the inter-ridge path; determining a value of a target parameter based on the target segmentation probability map; the target parameter indicates a weight coefficient of each pixel point contained in the inter-ridge path image; the target parameter is used for an active contour model; based on the active contour model, the target segmentation probability map and the value of the target parameter, extracting the inter-ridge path during the operation of an unmanned tractor for open-field vegetables. The method and device for extracting inter-ridge paths during unmanned tractor operation of open-field vegetables provided in the present application can improve the accuracy of extracting inter-ridge paths.

Description

Technical Field

[0001] The present application relates to the field of path extraction technology, and in particular to a method and device for extracting paths between ridges of unmanned tractor operation of open-field vegetables. Background Art

[0002] With the rapid development of precision agriculture, intelligent agricultural equipment (such as unmanned tractors) use GPS / Beidou devices to locate their positions and combine artificial intelligence technology to automatically perform operations such as land preparation, sowing and transplanting, field plant protection and crop harvesting, greatly improving operational efficiency. This is of great significance to promoting the development of intelligent agriculture.

[0003] In the field environment, ridge path extraction is mainly targeted at crops such as cotton, wheat, corn and rice. It is only effective in specific crops, specific operation links and specific environments, and rarely takes vegetables as the research object. The difficulties in ridge path extraction for open-field vegetables include that the spacing between planting rows of open-field vegetables is narrower, the ridge boundaries are similar to those of unridged soil, and the ridge boundaries become increasingly blurred as the crops grow, which makes it difficult to accurately extract the ridge paths of open-field vegetables. Summary of the invention

[0004] The present application provides a method and device for extracting the inter-ridge paths of open-field vegetables by unmanned tractor operation, so as to solve the defect that the inter-ridge paths of open-field vegetables are difficult to accurately extract in the prior art.

[0005] In a first aspect, the present application provides a method for extracting paths between ridges of unmanned tractor operation of open-field vegetables, comprising:

[0006] Based on the semantic segmentation network, a target segmentation probability map of the inter-ridge path image is determined; the target segmentation probability map represents the probability that each pixel point included in the inter-ridge path image belongs to the inter-ridge path;

[0007] Based on the target segmentation probability map, determining the value of a target parameter; the target parameter represents a weight coefficient of each pixel point included in the inter-ridge path image; the target parameter is used for an active contour model;

[0008] Based on the active contour model, the target segmentation probability map and the value of the target parameter, the inter-ridge path during the operation of the unmanned tractor for open-field vegetables is extracted.

[0009] In one embodiment, determining the value of the target parameter based on the target segmentation probability map includes:

[0010] Based on the target segmentation probability map, determining an exponential value of an exponential function;

[0011] Based on the exponential value of the exponential function, the value of the target parameter is determined; the target parameter includes a first weight coefficient and a second weight coefficient.

[0012] In one embodiment, the specific calculation method of the target parameter is as follows:

[0013]

[0014] Wherein, λ1 represents the first weight coefficient of each pixel point included in the inter-ridge path image, exp() represents an exponential function with e as the base, and Y pred represents the target segmentation probability map, and λ2 represents the second weight coefficient of each pixel point included in the inter-ridge path image.

[0015] In one embodiment, the extracting of the inter-ridge path during the operation of the unmanned tractor for open-field vegetables based on the active contour model, the target segmentation probability map and the value of the target parameter includes:

[0016] Inputting the target segmentation probability map into the active contour model; and using the value of the target parameter as the parameter value corresponding to the energy functional of the active contour model;

[0017] determining a first target value of an energy functional of the active contour model;

[0018] Based on the first target value, the inter-ridge paths during the operation of the unmanned tractor for open-field vegetables contained in the target segmentation probability map are extracted.

[0019] In one embodiment, the method further comprises:

[0020] Based on the Dice loss function and the distance loss function, the loss function of the semantic segmentation network is determined; the distance loss function is determined based on the signed distance map and the label distance map; the signed distance map and the label distance map are determined based on the semantic segmentation network.

[0021] In one embodiment, the loss function of the active contour model is a cross entropy loss function; the cross entropy loss function is determined based on the semantic segmentation network.

[0022] In one embodiment, the specific calculation formula of the cross entropy loss function is as follows:

[0023]

[0024] Among them, L ACM represents the cross entropy loss function, n represents the number of pixels contained in the inter-ridge path image, and p irepresents the predicted value of the i-th pixel in the signed distance map determined based on the semantic segmentation network, σ i represents the target value of the i-th pixel in the label distance map determined based on the semantic segmentation network.

[0025] In a second aspect, the present application also provides a device for extracting paths between ridges of unmanned tractor-assisted open-field vegetable operations, comprising:

[0026] A first determination module is used to determine a target segmentation probability map of the inter-ridge path image based on a semantic segmentation network; the target segmentation probability map represents the probability that each pixel point included in the inter-ridge path image belongs to the inter-ridge path;

[0027] A second determination module is used to determine the value of a target parameter based on the target segmentation probability map; the target parameter represents a weight coefficient of each pixel point included in the inter-ridge path image; the target parameter is used for an active contour model;

[0028] The extraction module is used to extract the inter-ridge path during the operation of the unmanned tractor for open-field vegetables based on the active contour model, the target segmentation probability map and the value of the target parameter.

[0029] In a third aspect, the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0030] In a fourth aspect, the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0031] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements the method described in the first aspect when executed by a processor.

[0032] The present application provides a method and device for extracting the inter-ridge path of an unmanned tractor operating open-field vegetables. The method determines the corresponding parameter values ​​of an active contour model through a semantic segmentation network, and then extracts the inter-ridge path during the operation of an unmanned tractor operating open-field vegetables based on the active contour model, the target segmentation probability map and the value of the target parameter. The method can obtain connected segmentation areas with clearer boundaries, improve the image segmentation accuracy, obtain high-precision inter-ridge path boundary extraction results, and provide a basis for subsequent inter-ridge path navigation line fitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 It is a flow chart of the method for extracting paths between ridges of unmanned tractor operation of open-field vegetables provided by the present application;

[0035] Figure 2 It is a schematic diagram of the principle of the method for extracting the inter-ridge path of unmanned tractor operation of open-field vegetables provided by the present application;

[0036] Figure 3 It is a structural schematic diagram of a model for using a method for extracting inter-ridge paths for unmanned tractor operation of open-field vegetables provided in the present application;

[0037] Figure 4 It is a structural schematic diagram of the inter-ridge path extraction device for unmanned tractor operation of open-field vegetables provided by the present application;

[0038] Figure 5 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0039] Visual navigation obtains real-time field dynamics by collecting operation images, and uses image processing and analysis to detect navigation paths to complete field operations. In a large field environment, satellite signals are affected by base stations, climate and obstacles, and positioning may deviate. Visual navigation can correct satellite navigation deviations and assist agricultural tractors in precise movement.

[0040] The field automatic navigation system of intelligent agricultural machinery equipment (such as unmanned tractors) is mainly composed of satellite positioning system navigation and machine vision navigation, and is mostly used in the large-scale production of crops such as cotton, wheat and rice.

[0041] The navigation technology based on the satellite navigation positioning system has limited accuracy and is suitable for large-field path planning scenarios. However, due to the complex terrain, open climate and dense crop distribution of farmland, it is difficult to ensure signal stability when the satellite navigation system is applied to the agricultural environment, and the navigation path is prone to deviations. Especially in vegetable production operations, due to the narrow spacing between field planting rows, uneven terrain, and misalignment of the central axis of agricultural machinery, it is difficult to ensure the stability of the field advance path, and agricultural machinery is prone to ridge crushing and seedling damage during operation. In addition, agricultural tractors that are originally shipped from the factory are rarely equipped with satellite navigation systems, which need to be installed later. Some remote agricultural production areas lack base stations and tractors equipped with navigation systems, making it difficult to promote intelligent operation equipment and operation modes.

[0042] The vision-based navigation method does not rely on satellite signals and base stations and has better scene adaptability. However, the paths between ridges in agricultural production are complex and diverse, including high ridge paths, low ridge paths, and flat tractor paths. Due to different tractor structural parameters, the proportion of the image occupied by the paths between ridges may be different. On the other hand, as crops grow, the paths between ridges are blocked by crops or weeds, which will cause the paths between ridges in the image to be discontinuous and unable to show the continuous path characteristics, which also affects the effect of path extraction.

[0043] Although the deep semantic segmentation method can achieve a certain accuracy in inter-ridge path extraction, it is necessary to balance the segmentation accuracy and model scale; moreover, the existing research based on deep learning semantic segmentation cannot be well explained in the process of crop inter-ridge navigation path identification, and the training model is difficult to apply to different open environments.

[0044] Based on this, the present application proposes a method and device for extracting the inter-ridge paths of open-field vegetables using an unmanned tractor, which can solve the defect in the prior art that the inter-ridge paths of open-field vegetables are difficult to accurately extract.

[0045] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] Figure 1 Schematic diagram of the process of extracting the path between ridges of unmanned tractor operation of open-field vegetables provided in the embodiment of the present application. Figure 1 The embodiment of the present application provides a method for extracting paths between ridges of unmanned tractor operation of open-field vegetables, the execution subject of which may be an electronic device, for example, a controller, and the method may include:

[0047] Step 110: determining a target segmentation probability map of the inter-ridge path image based on the semantic segmentation network; the target segmentation probability map represents the probability that each pixel point included in the inter-ridge path image belongs to the inter-ridge path;

[0048] Step 120, based on the target segmentation probability map, determine the value of the target parameter; the target parameter represents the weight coefficient of each pixel point included in the inter-ridge path image; the target parameter is used for the active contour model;

[0049] Step 130: extracting the inter-ridge paths during the operation of the unmanned tractor for open-field vegetables based on the active contour model, the target segmentation probability map and the values ​​of the target parameters.

[0050] Target segmentation is an important part of agricultural image analysis. Methods such as thresholding, region growing, and clustering have been widely studied in existing results. Convolutional neural networks, due to their powerful feature learning capabilities, can learn semantically rich high-dimensional features in data, significantly improving the performance of many segmentation tasks. However, field ridge path images contain a variety of complex backgrounds, with weeds, mulch, crops, and other interferences. Figure 2 , Figure 3 , the method for extracting the inter-ridge path of an unmanned tractor operating open-field vegetables provided in the embodiment of the present application is described in detail:

[0051] In step 110, the controller may determine a target segmentation probability map of the inter-ridge path image based on a semantic segmentation network.

[0052] Specifically, the controller can locate the inter-ridge path area through the Deeplabv3+ semantic segmentation network model and adjust the model to a 2-classification semantic segmentation model. In the 2-classification semantic segmentation model, one is classified as the inter-ridge path and the other is classified as the field background. The semantic segmentation network can map the input inter-ridge path image into two pixel-level target segmentation probability maps and background segmentation probability maps. The target segmentation probability map represents the probability that the pixel points contained in the inter-ridge path image belong to the target (i.e., the inter-ridge path), and the background segmentation probability map represents the probability that the pixel points contained in the inter-ridge path image belong to the background. Based on the target segmentation probability map and the background segmentation probability map, the energy weights inside and outside the inter-ridge path in the image can be calculated to initialize the energy function of the active contour model.

[0053] In step 120, the controller may determine the value of the target parameter based on the target segmentation probability map obtained in step 110. The target parameter represents the weight coefficient of each pixel point included in the inter-ridge path image; the target parameter is used for the active contour model. The value of the target parameter corresponds to the corresponding parameter value in the energy function of the active contour model.

[0054] In step 130, the controller can extract the path between ridges during the operation of the unmanned tractor for open-field vegetables based on the active contour model, the target segmentation probability map and the value of the target parameter. Specifically, the controller can use the fuzzy energy active contour model to fine-tune the target segmentation probability map, use the information obtained by the semantic segmentation network to set parameters (corresponding to the target parameter) for each pixel, and then combine the image's own information to achieve the segmentation of the path between ridges.

[0055] The Deeplabv3+ semantic segmentation network has a symmetrical encoder-decoder structure. Since the initialization parameters of the active contour model need to be learned, the active contour model can be introduced in the lower three layers of the decoder part of the network. The segmentation probability map obtained by training each layer is divided more finely using the active contour model, and then added to the original output feature map of each layer of the network, so as to perform fine extraction of the edges between ridges.

[0056] The method for extracting the inter-ridge paths of unmanned tractors for open-field vegetables provided in the embodiment of the present application determines the corresponding parameter values ​​of the active contour model through a semantic segmentation network, and then extracts the inter-ridge paths during the operation of the unmanned tractor for open-field vegetables based on the active contour model, the target segmentation probability map and the target parameter values. This can obtain connected segmentation areas with clearer boundaries, improve image segmentation accuracy, and obtain high-precision inter-ridge path boundary extraction results, providing a basis for subsequent inter-ridge path navigation line fitting.

[0057] In one embodiment, determining the value of the target parameter based on the target segmentation probability map includes:

[0058] Based on the target segmentation probability map, determine the exponential value of the exponential function;

[0059] Based on the exponential value of the exponential function, a value of a target parameter is determined; the target parameter includes a first weight coefficient and a second weight coefficient.

[0060] The controller can perform relevant operations based on the target segmentation probability map to obtain the exponential value of the exponential function with e as the base, and then determine the value of the target parameter based on the exponential value of the exponential function. The target parameter includes two parameters: a first weight coefficient and a second weight coefficient.

[0061] The method for extracting the inter-ridge path of an unmanned tractor for open-field vegetables provided in the embodiment of the present application combines the target segmentation probability map with the exponential function to determine the target parameters. The target parameters are used in the active contour model, which can further improve the image segmentation accuracy, obtain high-precision inter-ridge path boundary recognition results, and provide a basis for subsequent inter-ridge path navigation line fitting.

[0062] In one embodiment, the specific calculation method of the target parameter is as follows:

[0063]

[0064] Where λ1 represents the first weight coefficient of each pixel point contained in the inter-ridge path image, exp() represents the exponential function with e as the base, and Y pred represents the target segmentation probability map, and λ2 represents the second weight coefficient of each pixel point contained in the inter-ridge path image.

[0065] The method for extracting the inter-ridge path of an unmanned tractor for open-field vegetables provided in the embodiment of the present application combines the target segmentation probability map with the exponential function to determine the target parameters. The target parameters are used in the active contour model, which can further improve the image segmentation accuracy, obtain high-precision inter-ridge path boundary recognition results, and provide a basis for subsequent inter-ridge path navigation line fitting.

[0066] In one embodiment, based on the active contour model, the target segmentation probability map and the value of the target parameter, the inter-ridge path of the unmanned tractor for open-field vegetable operation is extracted, including:

[0067] The target segmentation probability map is input into the active contour model; and the value of the target parameter is used as the parameter value corresponding to the energy functional of the active contour model;

[0068] determining a first objective value of an energy functional of the active contour model;

[0069] Based on the first target value, the inter-ridge paths during the operation of the unmanned tractor for open-field vegetables contained in the target segmentation probability map are extracted.

[0070] After using the Deeplabv3+ semantic segmentation network to obtain the target segmentation probability map, the controller inputs it into the active contour model to divide the two-dimensional ridge path image into three regions: the area to be segmented, the background, and the ridge edge curve. Assume that the original image data defined on the image domain Ω is I, I∈Ω. The evolution curve C can represent the boundary of the ridge path area and divide the image into two regions, the inside of the curve is the ridge area, and the outside is the background area. The ridge path extraction can be simplified to calculating the curve C that propagates along the normal direction over time. The contour curve C can be represented by using 0.5 as the pseudo level set function for segmentation. The pseudo level set is defined as:

[0071] u(x)=0.5I(x)∈C

[0072] u(x)>0.5I(x)∈in(C)

[0073] u(x)<0.5I(x)∈out(C)

[0074] Among them, u(x) represents the membership function, and C represents the boundary of the inter-ridge path area.

[0075] The accurate segmentation result of the inter-ridge path can be obtained by minimizing the energy functional. The energy functional consists of two terms: the data term and the shape prior term. This application defines it as:

[0076] E FEAC (c1,c2,u(x))=λ1∫ Ω [u(x)] m |I(x)-c1| 2 dx+λ2∫Ω [1-u(x)] m |I(x)-c2| 2 dx+ηLength(C)

[0077] Among them, E FEAC (c1, c2, u(x)) represents the energy functional, c1 represents the average gray value of the image inside the curve, c2 represents the average gray value of the image outside the curve, u(x) represents the membership function, λ1 represents the first weight coefficient, Ω represents the inter-ridge path image domain, m represents the exponent, λ2 represents the second weight coefficient, η represents a constant, η≥0, which is a fixed parameter, and Length(C) represents the length of the boundary C of the inter-ridge path area.

[0078] λ1 and λ2 can better control the evolution of curve C, and thus better represent the position information of the image where the point is located.

[0079] Keeping u(x) unchanged, minimizing the energy functional yields:

[0080]

[0081] Update the energy functional, keep c1 and c2 unchanged, and minimize the energy functional to obtain the membership degree:

[0082]

[0083] Solving the energy functional minimization value is the image segmentation process. The controller can use the gradient descent flow method to solve the energy functional minimum value (first target value). Based on the first target value, image segmentation is achieved, and the inter-ridge path contained in the target segmentation probability map during the operation of the open-field vegetable unmanned tractor can be extracted.

[0084] The method for extracting the inter-ridge path of an unmanned tractor operating open-field vegetables provided in an embodiment of the present application inputs a target segmentation probability map into an active contour model, and uses the value of the target parameter as the parameter value corresponding to the energy functional of the active contour model (first weight coefficient λ1, second weight coefficient λ2), and obtains the inter-ridge path by minimizing the energy functional, thereby further improving the image segmentation accuracy and obtaining a high-precision inter-ridge path boundary recognition result.

[0085] In one embodiment, the method for extracting paths between ridges of unmanned tractor operation of open-field vegetables further includes:

[0086] Based on the Dice loss function and the distance loss function, the loss function of the semantic segmentation network is determined; the distance loss function is determined based on the symbol distance map and the label distance map; the symbol distance map and the label distance map are determined based on the semantic segmentation network.

[0087] This application can use the Dice loss function and the distance loss function to form the loss function of the semantic segmentation model backbone network:

[0088] L CNN =L Dice +L dist

[0089] Among them, L CNN represents the loss function of the semantic segmentation network, L Dice Represents the Dice loss function, calculated according to the Dice coefficient, L dist Represents the distance loss function, which is obtained by the L2 loss function.

[0090] In order to further improve the segmentation effect of the inter-ridge path by using the active contour model, a 1×1 convolution is added to the last layer of the semantic segmentation network. After the 1×1 convolution, a signed distance map and a label distance map can be regressed.

[0091]

[0092] Among them, L dist represents the distance loss function, n represents the number of pixels in the inter-ridge path image, and p i represents the predicted value of the i-th pixel in the signed distance map determined based on the semantic segmentation network, σ i Represents the target value of the i-th pixel in the label distance map determined based on the semantic segmentation network.

[0093] The method for extracting inter-ridge paths for unmanned tractor operations on open-field vegetables provided in the embodiment of the present application determines the loss function of the semantic segmentation network based on the Dice loss function and the distance loss function, which can further improve the image segmentation accuracy and obtain high-precision inter-ridge path boundary recognition results.

[0094] In one embodiment, the loss function of the active contour model is a cross entropy loss function; the cross entropy loss function is determined based on a semantic segmentation network.

[0095] The controller can determine the symbol distance map and the label distance map based on the semantic segmentation network, and then determine the cross entropy loss function. The cross entropy loss function can be used as the loss function of the active contour model.

[0096] The method for extracting the inter-ridge path of an unmanned tractor operating open-field vegetables provided in the embodiment of the present application uses the cross entropy loss function as the loss function of the active contour model to predict all pixels in the image, which can further improve the image segmentation accuracy and obtain high-precision inter-ridge path boundary recognition results.

[0097] In one embodiment, the specific calculation formula of the cross entropy loss function is as follows:

[0098]

[0099] Among them, L ACM represents the cross entropy loss function, n represents the number of pixels in the inter-ridge path image, and p i represents the predicted value of the i-th pixel in the signed distance map determined based on the semantic segmentation network, σ i Represents the target value of the i-th pixel in the label distance map determined based on the semantic segmentation network.

[0100] In summary, the overall loss function of this application can be written as:

[0101] L=L CNN +L ACM

[0102] Among them, L represents the overall loss function, L CNN represents the loss function of the semantic segmentation network, L ACM represents the cross entropy loss function.

[0103] The method for extracting the inter-ridge paths of unmanned tractor operations for open-field vegetables provided in the embodiment of the present application determines the symbol distance map and the label distance map based on the semantic segmentation network, and then determines the cross entropy loss function, and predicts all pixels in the image, which can further improve the image segmentation accuracy and obtain high-precision inter-ridge path boundary recognition results.

[0104] Based on the above embodiments, the present application integrates the fuzzy energy active contour model and the semantic segmentation network in view of the advantages and limitations of computer vision in the segmentation of inter-ridge paths, and proposes to establish an inter-ridge path fine segmentation network based on the active contour model. First, the semantic segmentation neural network is used to obtain the coarse segmentation result of the inter-ridge path, and the coarse segmentation result provides the initial contour for the fine segmentation model in the active contour model. The active contour model can perform curve fitting on the boundary of the image target to produce accurate boundary positioning. The present application uses a fuzzy energy active contour model based on level set initialization to optimize the segmentation boundary, and obtains a connected segmentation area with clearer boundaries. At the same time, the distance loss is introduced into the semantic segmentation to provide constraints for the extraction of inter-ridge paths, thereby improving the segmentation accuracy and providing a basis for the subsequent fitting of the inter-ridge path navigation line.

[0105] The introduction of the active contour model to perform curve fitting on the image target boundary can produce accurate boundary positioning. The present application can use the semantic segmentation model as the backbone of the convolutional neural network, and use the above-mentioned active contour model based on level set initialization to optimize the segmentation of the network model. The method proposed in the present application integrates the active contour model with the convolutional neural network backbone to form a two-stage framework. In the first stage, the inter-ridge path area is located by the convolutional neural network, and the initial segmentation probability map and the signed distance function required for calculating the initialization of the level set active contour model are generated. The initial segmentation map can be used to calculate the internal and external energy weights of the inter-ridge path for initialization of the active contour model energy function. In the second stage, an active contour model is used to fine-tune the segmentation surface. The active contour model can use the information obtained by the convolutional neural network to set parameters for each pixel, and combine the image's own information to achieve the segmentation of the inter-ridge path image.

[0106] This application applies the improved DeepLabv3+ semantic segmentation network to the automatic extraction of inter-ridge paths of open-field vegetables. In order to compensate for and refine the problem of insufficient positioning accuracy of inter-ridge areas in the deep semantic segmentation network, this application also proposes a two-stage inter-ridge path segmentation model, which uses the inter-ridge path recognition results of the semantic segmentation network as shape priors, constructs a fuzzy energy active contour model with shape priors, and combines the color information brought by the original image to obtain more accurate segmentation results. The key points of this application are as follows:

[0107] The boundaries of the paths between ridges are fuzzy and the shapes are difficult to predict. The segmentation accuracy cannot be improved by post-processing with the active contour model alone. Therefore, the active contour model is defined as the loss function of the semantic segmentation network, and a trainable image segmentation framework based on Deeplabv3+ is proposed. The framework includes a convolutional neural network and an active contour function with learnable parameters. The global optimization characteristics of the active contour function can accurately detect the object boundaries and train them through end-to-end differentiation.

[0108] This application proposes a segmentation framework based on the level set active contour model for the problem of extracting and segmenting inter-ridge paths. The semantic segmentation network encoder-decoder structure is used to provide pixel-by-pixel parameter mapping and initialization contours for the level set active contours, and then the predicted segmentation map is obtained by iterating the active contour function. This method has a certain robustness to unstructured open-field vegetable environments and can perform automated inter-ridge path segmentation. Compared with traditional semantic segmentation networks and threshold segmentation methods, the method proposed in this application can improve the accuracy of inter-ridge path boundary recognition.

[0109] The device for extracting paths between ridges of unmanned tractor operation for open-field vegetables provided in the present application is described below. The device for extracting paths between ridges of unmanned tractor operation for open-field vegetables described below and the method for extracting paths between ridges of unmanned tractor operation for open-field vegetables described above can be referenced to each other.

[0110] Figure 4 Schematic diagram of the structure of the device for extracting the inter-ridge path of an unmanned tractor operating open-field vegetables provided in the embodiment of the present application. Figure 4 The device for extracting paths between ridges of unmanned tractor operation of open-field vegetables provided in the embodiment of the present application may include:

[0111] A first determination module 410 is used to determine a target segmentation probability map of the inter-ridge path image based on a semantic segmentation network; the target segmentation probability map represents the probability that each pixel point included in the inter-ridge path image belongs to the inter-ridge path;

[0112] A second determination module 420 is used to determine the value of a target parameter based on the target segmentation probability map; the target parameter represents a weight coefficient of each pixel point included in the inter-ridge path image; the target parameter is used for an active contour model;

[0113] The extraction module 430 is used to extract the inter-ridge path during the operation of the unmanned tractor for open-field vegetables based on the active contour model, the target segmentation probability map and the value of the target parameter.

[0114] The device for extracting the inter-ridge paths of unmanned tractor operations for open-field vegetables provided in the embodiment of the present application determines the corresponding parameter values ​​of the active contour model through a semantic segmentation network, and then extracts the inter-ridge paths during the operation of the unmanned tractor for open-field vegetables based on the active contour model, the target segmentation probability map and the target parameter values. This can obtain connected segmentation areas with clearer boundaries, improve image segmentation accuracy, and obtain high-precision inter-ridge path boundary extraction results, thereby providing a basis for subsequent inter-ridge path navigation line fitting.

[0115] Specifically, the above-mentioned open-field vegetable unmanned tractor operation ridge path extraction device provided in the embodiment of the present application can implement all the method steps implemented by the above-mentioned method embodiment in which the execution subject is the controller, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those of the method embodiment will not be described in detail here.

[0116] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the method for extracting the path between ridges of unmanned tractor operation of open-field vegetables, for example, including:

[0117] Based on the semantic segmentation network, a target segmentation probability map of the inter-ridge path image is determined; the target segmentation probability map represents the probability that each pixel point included in the inter-ridge path image belongs to the inter-ridge path;

[0118] Based on the target segmentation probability map, determining the value of a target parameter; the target parameter represents a weight coefficient of each pixel point included in the inter-ridge path image; the target parameter is used for an active contour model;

[0119] Based on the active contour model, the target segmentation probability map and the value of the target parameter, the inter-ridge path during the operation of the unmanned tractor for open-field vegetables is extracted.

[0120] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0121] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for extracting the inter-ridge path of an unmanned tractor operating open-field vegetables provided by the above methods, for example, including:

[0122] Based on the semantic segmentation network, a target segmentation probability map of the inter-ridge path image is determined; the target segmentation probability map represents the probability that each pixel point included in the inter-ridge path image belongs to the inter-ridge path;

[0123] Based on the target segmentation probability map, determining the value of a target parameter; the target parameter represents a weight coefficient of each pixel point included in the inter-ridge path image; the target parameter is used for an active contour model;

[0124] Based on the active contour model, the target segmentation probability map and the value of the target parameter, the inter-ridge path during the operation of the unmanned tractor for open-field vegetables is extracted.

[0125] In another aspect, the present application further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the steps of the method for extracting inter-ridge paths of unmanned tractor operation of open-field vegetables provided by the above methods, for example, including:

[0126] Based on the semantic segmentation network, a target segmentation probability map of the inter-ridge path image is determined; the target segmentation probability map represents the probability that each pixel point included in the inter-ridge path image belongs to the inter-ridge path;

[0127] Based on the target segmentation probability map, determining the value of a target parameter; the target parameter represents a weight coefficient of each pixel point included in the inter-ridge path image; the target parameter is used for an active contour model;

[0128] Based on the active contour model, the target segmentation probability map and the value of the target parameter, the inter-ridge path during the operation of the unmanned tractor for open-field vegetables is extracted.

[0129] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0130] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0131] It should also be noted that the terms "first", "second", etc. in the embodiments of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0132] In the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0133] "Determine B based on A" in the embodiments of the present application means that the factor A should be considered when determining B. It is not limited to "B can be determined based on A alone", but should also include: "Determine B based on A and C", "Determine B based on A, C and E", "Determine C based on A, and further determine B based on C", etc. In addition, it can also include taking A as a condition for determining B, for example, "When A meets the first condition, use the first method to determine B"; for another example, "When A meets the second condition, determine B", etc.; for another example, "When A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be a condition that takes A as a factor for determining B, for example, "When A meets the first condition, use the first method to determine C, and further determine B based on C", etc.

[0134] In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for extracting paths between ridges of unmanned tractor operation of open-field vegetables, characterized in that: include: Based on the semantic segmentation network, a target segmentation probability map of the inter-ridge path image is determined; the target segmentation probability map represents the probability that each pixel point included in the inter-ridge path image belongs to the inter-ridge path; Based on the target segmentation probability map, determining the value of a target parameter; the target parameter represents a weight coefficient of each pixel point included in the inter-ridge path image; the target parameter is used for an active contour model; Extracting the inter-ridge path during the operation of the unmanned tractor for open-field vegetables based on the active contour model, the target segmentation probability map and the value of the target parameter; The extracting of the inter-ridge path during the operation of the unmanned tractor for open-field vegetables based on the active contour model, the target segmentation probability map and the value of the target parameter includes: Inputting the target segmentation probability map into the active contour model; and using the value of the target parameter as the parameter value corresponding to the energy functional of the active contour model; determining a first target value of an energy functional of the active contour model; Based on the first target value, extracting the inter-ridge path during the operation of the unmanned tractor for open-field vegetables contained in the target segmentation probability map; The energy functional consists of two terms: the data term and the shape prior term; the energy functional is: E FEAC (c1,c2,u(x))=λ1∫ Ω [u(x)] m |I(x)-c1| 2 dx+λ2∫ Ω [1-u(x)] m |I(x)-c2| 2 dx+ηLength(C); Among them, E FEAC (c1, c2, u(x)) represents the energy functional, c1 represents the average gray value of the image inside the curve, c2 represents the average gray value of the image outside the curve, u(x) represents the membership function, λ1 represents the first weight coefficient, Ω represents the inter-ridge path image domain, m represents the index, λ2 represents the second weight coefficient, η represents a constant, η≥0, which is a fixed parameter, and Length(C) represents the length of the boundary C of the inter-ridge path area; The method further comprises: The semantic segmentation network is a Deeplabv3+ semantic segmentation network; and a 1×1 convolution is added to the last layer of the Deeplabv3+ semantic segmentation network, and a signed distance map and a label distance map are regressed after the 1×1 convolution; the Deeplabv3+ semantic segmentation network has a symmetrical encoder-decoder structure, and an active contour model is introduced into the lower three layers of the decoder of the Deeplabv3+ semantic segmentation network, and the segmentation probability map obtained by training each layer is divided more finely using the active contour model, and then added to the original output feature map of each layer of the network to perform fine extraction of the edges between ridges; Based on the Dice loss function and the distance loss function, determining the loss function of the semantic segmentation network; the distance loss function is determined based on the signed distance graph and the label distance graph; the signed distance graph and the label distance graph are determined based on the semantic segmentation network; The loss function of the active contour model is a cross entropy loss function; the cross entropy loss function is determined based on the semantic segmentation network; The loss function of the semantic segmentation model backbone network is: L CNN =L Dice +L dist ; Among them, L CNN represents the loss function of the semantic segmentation network, L Dice represents the Dice loss function, L dist represents the distance loss function; Among them, L dist represents the distance loss function, n represents the number of pixels in the inter-ridge path image, and p i represents the predicted value of the i-th pixel in the signed distance map determined based on the semantic segmentation network, σ i represents the target value of the i-th pixel in the label distance map determined based on the semantic segmentation network; The specific calculation formula of the cross entropy loss function is as follows: Among them, L ACM represents the cross entropy loss function, n represents the number of pixels contained in the inter-ridge path image, and p i represents the predicted value of the i-th pixel in the signed distance map determined based on the semantic segmentation network, σ i represents the target value of the i-th pixel in the label distance map determined based on the semantic segmentation network; The overall loss function is: L=L CNN +L ACM ; L represents the overall loss function.

2. The method for extracting paths between ridges of unmanned tractor operation of open-field vegetables according to claim 1, characterized in that: The step of determining a value of a target parameter based on the target segmentation probability map comprises: Based on the target segmentation probability map, determining an exponential value of an exponential function; Based on the exponential value of the exponential function, the value of the target parameter is determined; the target parameter includes a first weight coefficient and a second weight coefficient.

3. The method for extracting paths between ridges of unmanned tractor operation of open-field vegetables according to claim 2, characterized in that: The specific calculation method of the target parameters is as follows: Wherein, λ1 represents the first weight coefficient of each pixel point included in the inter-ridge path image, exp() represents an exponential function with e as the base, and Y pred represents the target segmentation probability map, and λ2 represents the second weight coefficient of each pixel point included in the inter-ridge path image.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for extracting the inter-ridge paths of unmanned tractor operations for open-field vegetables as described in any one of claims 1 to 3 is implemented.

5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for extracting inter-ridge paths for unmanned tractor operation of open-field vegetables as described in any one of claims 1 to 3 is implemented.

Images