Harvester control method and device, electronic equipment and storage medium
By implementing a full-frame boundary line detection model and steering control method on the harvester, the problems of operation fatigue and inefficiency caused by the existing harvester relying on the driver's subjective judgment are solved, and more efficient and automated harvesting operations are achieved.
Patent Information
- Application Number
- CN202510099327.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-22
- Publication Date
- 2025-05-27
AI Technical Summary
In the full-scale harvesting operation, existing harvesters rely on the driver's naked eye observation and subjective judgment, resulting in operation fatigue, low cutting rate and harvesting efficiency.
By obtaining the front plot image of the harvester and the full range boundary line at the previous moment, the full range boundary line detection model (including the residual module, channel attention module and statistical grouping module) is used to detect the target point, fit the polynomial curve to determine the initial full range boundary line, and obtain the target full range boundary line through correction, and finally control the steering according to the deviation value between the full range boundary line and the preset header line.
It improves the crop cutting rate and harvesting efficiency, reduces the driver's operating fatigue, and achieves more automated and efficient harvesting operations.
Smart Images
Figure CN120044945A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of agricultural harvesting machine control, and more particularly, to a control method, device, electronic device, and storage medium for a harvesting machine. Background Art
[0002] A harvesting machine is an agricultural machine used for harvesting mature cereal crops such as wheat, corn, rice, etc.
[0003] Currently, the realization of full-width harvesting by a harvesting machine mainly depends on the driver's visual observation and subjective judgment to control the forward direction of the vehicle. Long-term harvesting operations cause driver fatigue, resulting in a low cutting width rate and harvesting efficiency of crops. Summary of the Invention
[0004] In view of this, the purpose of this application is to provide a control method, device, electronic device, and storage medium for a harvesting machine, which can improve the cutting width rate and harvesting efficiency of crops.
[0005] In a first aspect, an embodiment of this application provides a control method for a harvesting machine, and the method includes:
[0006] Obtain the image of the front plot of the harvesting machine at the current moment, and the full-width boundary line of the front plot at the previous moment;
[0007] Input the front plot image into the full-width boundary line detection model to obtain the target points on the full-width boundary line in the front plot image; fit all the target points into a polynomial curve to obtain the initial full-width boundary line of the front plot at the current moment;
[0008] Correct the initial full-width boundary line according to the full-width boundary line of the front plot at the previous moment to obtain the target full-width boundary line of the front plot at the current moment;
[0009] Perform steering control on the harvesting machine according to the deviation value between the target full-width boundary line and the preset cutter bar line in the front plot image.
[0010] In a possible implementation, the full-width boundary line detection model includes a residual module, a channel attention module, and a statistical grouping module; the step of inputting the front plot image into the full-width boundary line detection model to obtain the target points on the full-width boundary line in the front plot image includes:
[0011] Input the front plot image into the residual module to obtain a feature image with a preset number of channels;
[0012] For each channel's feature image, input the feature image into the channel attention module to weight the feature image to obtain a weighted feature image;
[0013] Input all the weighted feature images into the statistical grouping module to obtain the target points on the full-width boundary line in the front plot image.
[0014] In a possible implementation manner, the step of inputting the feature image into the channel attention module to weight the feature image to obtain a weighted feature image includes:
[0015] Compress the feature image through global average pooling to obtain an image feature vector;
[0016] Calculate the weight corresponding to the feature image according to the image feature vector;
[0017] Multiply the feature image by the corresponding weight to obtain a weighted feature image.
[0018] In a possible implementation manner, the step of inputting all the weighted feature images into the statistical grouping module to obtain the target points on the full-width boundary line in the front plot image includes:
[0019] Recombine all the weighted feature images to obtain a target feature image;
[0020] Divide the target feature image into h×w cells, and add an empty column after the cells in the last column; where h is the height of the target feature image and w is the width of the target feature image;
[0021] Input the target feature image with the added empty column into a classification function to obtain the probability that there is a full-width boundary line in each cell;
[0022] Determine the coordinates of the cells with probabilities greater than a preset probability as the coordinates of the target points.
[0023] In a possible implementation manner, the step of correcting the initial full-width boundary line according to the full-width boundary line of the front plot at the previous moment to obtain the target full-width boundary line of the front plot at the current moment includes:
[0024] Substitute the full-width boundary line of the front plot at the previous moment and the initial full-width boundary line into the following formula to obtain the target full-width boundary line of the front plot at the current moment:
[0025]
[0026]
[0027]
[0028]
[0029]
[0030] Among them, is the curve coefficient of the full-width boundary line of the front plot at the previous moment, A is the unit matrix, k is the current moment, W k is the process noise at the current moment, is the prior curve coefficient of the full-width boundary line of the front plot at the current moment, H is the observation matrix, V k is the observation noise at the current moment, Zk is the observation value at the current moment, is the prior error covariance matrix at the current moment, P K-1 is the posterior error covariance matrix at the previous moment, T is the transpose, Q is the process error, K k is the gain, R is the observation error, is the curve coefficient of the target full-width boundary line of the front plot at the current moment, is the curve coefficient of the initial full-width boundary line.
[0031] In a possible implementation manner, the deviation value between the target full-width boundary line and the preset cutting table line in the front plot image is calculated through the following steps:
[0032] Determine the abscissa of the point on the target full-width boundary line with the ordinate of h / 2 as the first value; where h is the height of the target feature image;
[0033] Determine the abscissa of the point on the preset cutting table line with the ordinate of h / 2 as the second value;
[0034] Subtract the second value from the first value to obtain the deviation value between the target full-width boundary line and the preset cutting table line in the front plot image.
[0035] In a possible implementation manner, the steering control of the harvester according to the deviation value between the target full-width boundary line and the preset cutting table line in the front plot image includes:
[0036] If the deviation value is greater than or equal to a preset threshold, control the harvester to turn so that the deviation value between the target full-width boundary line in the front plot image after turning and the preset cutting table line is less than the preset threshold.
[0037] In a second aspect, the embodiments of the present application further provide a control device for a harvester, and the device includes:
[0038] An acquisition module, configured to acquire the front plot image of the harvester at the current moment and the full-width boundary line of the front plot at the previous moment;
[0039] An input fitting module, configured to input the front plot image into a full-width boundary line detection model to obtain target points on the full-width boundary line in the front plot image; fit all the target points into a polynomial curve to obtain an initial full-width boundary line of the front plot at the current moment;
[0040] A correction module, configured to correct the initial full-width boundary line according to the full-width boundary line of the front plot at the previous moment to obtain a target full-width boundary line of the front plot at the current moment;
[0041] A control module, configured to perform steering control on the harvester according to the deviation value between the target full-width boundary line and a preset cutter bar line in the front plot image.
[0042] In a possible implementation manner, the full-width boundary line detection model includes a residual module, a channel attention module, and a statistical grouping module; the input fitting module is specifically configured to input the front plot image into the residual module to obtain a feature image with a preset number of channels; for the feature image of each channel, input the feature image into the channel attention module to weight the feature image to obtain a weighted feature image; input all the weighted feature images into the statistical grouping module to obtain target points on the full-width boundary line in the front plot image.
[0043] In a possible implementation manner, the input fitting module is specifically configured to compress the feature image through global average pooling to obtain an image feature vector; calculate the weight corresponding to the feature image according to the image feature vector; multiply the feature image by the corresponding weight to obtain a weighted feature image.
[0044] In a possible implementation manner, the input fitting module is specifically configured to reorganize all the weighted feature images to obtain a target feature image; divide the target feature image into h×w cells, and add an empty column after the cells in the last column; where h is the height of the target feature image and w is the width of the target feature image; input the target feature image after adding the empty column into a classification function to obtain the probability that there is a full-width boundary line in each cell; determine the coordinates of the cells with a probability greater than a preset probability as the coordinates of the target points.
[0045] In a possible implementation manner, the correction module is specifically configured to substitute the full-width boundary line of the front plot at the previous moment and the initial full-width boundary line into the following formula to obtain the target full-width boundary line of the front plot at the current moment:
[0046]
[0047]
[0048]
[0049]
[0050]
[0051] Among them, is the curve coefficient of the full-width boundary line of the front plot at the previous moment, A is the unit matrix, k is the current moment, W k is the process noise at the current moment, is the prior curve coefficient of the full-width boundary line of the front plot at the current moment, H is the observation matrix, V k is the observation noise at the current moment, Z k is the observation value at the current moment, is the prior error covariance matrix at the current moment, P K-1 is the posterior error covariance matrix at the previous moment, T is the transpose, Q is the process error, K k is the gain, R is the observation error, is the curve coefficient of the target full-width boundary line of the front plot at the current moment, is the curve coefficient of the initial full-width boundary line.
[0052] In a possible implementation manner, the control module is further configured to:
[0053] Determine the abscissa of the point with the ordinate of 2 on the target full-width boundary line as the first value; where h is the height of the target feature image;
[0054] Determine the abscissa of the point with the ordinate of h / 2 on the preset cutter bar line as the second value;
[0055] Subtract the second value from the first value to obtain the deviation value between the target full-width boundary line and the preset cutter bar line in the front plot image.
[0056] In a possible implementation manner, the control module is specifically configured to, if the deviation value is greater than or equal to a preset threshold, control the harvester to turn so that the deviation value between the target full-width boundary line and the preset cutter bar line in the front plot image after the harvester turns is less than the preset threshold.
[0057] In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the control method of the harvester according to any one of the first aspects.
[0058] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, it performs the steps of the control method of the harvester according to any one of the first aspects.
[0059] An embodiment of the present application provides a control method, device, electronic device, and storage medium for a harvester. The method includes: obtaining an image of the front plot of the harvester at the current moment and the full-width boundary line of the front plot at the previous moment; inputting the front plot image into a full-width boundary line detection model to obtain target points on the full-width boundary line in the front plot image; fitting all the target points into a polynomial curve to obtain the initial full-width boundary line of the front plot at the current moment; correcting the initial full-width boundary line according to the full-width boundary line of the front plot at the previous moment to obtain the target full-width boundary line of the front plot at the current moment; and controlling the steering of the harvester according to the deviation value between the target full-width boundary line and a preset cutter bar line in the front plot image. Through the method of the present application, the cutting width rate and harvesting efficiency of crops can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.
[0061] Figure 1 It shows a schematic flow chart of a control method for a harvester provided by an embodiment of the present application;
[0062] Figure 2 It shows a schematic diagram of a target feature image with empty columns added provided by an embodiment of the present application;
[0063] Figure 3 It shows a schematic diagram of a control device for a harvester provided by an embodiment of the present application;
[0064] Figure 4 It shows a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0065] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will, in conjunction with the accompanying drawings in the embodiments of this application, clearly and completely describe the technical solutions in the embodiments of this application. It should be understood that the accompanying drawings in this application only serve the purposes of illustration and description, and are not used to limit the protection scope of this application. Additionally, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of this application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without a logical context relationship may be reversed in order or implemented simultaneously. Furthermore, those skilled in the art, under the guidance of the content of this application, may add one or more other operations to the flowchart or remove one or more operations from the flowchart.
[0066] In addition, the described embodiments are only some embodiments of this application, rather than all of the embodiments. The components of the embodiments of this application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application claimed, but merely represents the selected embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of this application.
[0067] To enable those skilled in the art to use the content of this application, in combination with the specific application scenario of "agricultural harvesting machine control field", the following implementation manners are given. For those skilled in the art, without departing from the spirit and scope of this application, the general principles defined here can be applied to other embodiments and application scenarios. Although this application is mainly described around the "agricultural harvesting machine control field", it should be understood that this is only an exemplary embodiment.
[0068] It should be noted that the term "including" will be used in the embodiments of this application to indicate the existence of the features stated thereafter, but does not exclude adding other features.
[0069] The following provides a detailed description of a control method for a harvester provided in the embodiments of this application.
[0070] Refer to Figure 1 As shown, it is a schematic flowchart of a control method for a harvester provided in the embodiments of this application. The following explains each step exemplary in the embodiments of this application:
[0071] S101. Obtain the image of the front plot of the harvester at the current moment and the full-frame boundary line of the front plot at the previous moment.
[0072] In the embodiment of the present application, a camera is placed directly above the harvester. The camera captures the plot within a preset range in front of the harvester in real time to obtain an image of the front plot. Set the current moment as k. The full-width boundary line of the front plot at the previous moment (i.e., the moment k - 1) is determined based on the full-width boundary line detected in the front plot image at the previous moment and the full-width boundary line of the front plot at the corresponding previous moment (i.e., the moment k - 2). The full-width boundary line refers to the boundary line between the harvested and unharvested areas of the crops that the harvester needs to identify during operation.
[0073] Here, the process of determining the full-width boundary line of the front plot at the previous moment can refer to the process of determining the target full-width boundary line of the front plot at the current moment, which will not be elaborated here.
[0074] S102. Input the front plot image into the full-width boundary line detection model to obtain the target points on the full-width boundary line in the front plot image; fit all the target points into a polynomial curve to obtain the initial full-width boundary line of the front plot at the current moment.
[0075] In the embodiment of the present application, the full-width boundary line detection model is trained based on the front plot sample image and the target points on the full-width boundary line in the front plot sample image. The full-width boundary line detection model includes a residual module, a channel attention module, and a statistical grouping module. A cubic polynomial is used to fit all the target points into a polynomial curve to obtain the initial full-width boundary line of the front plot at the current moment.
[0076] Specifically, inputting the front plot image into the full-width boundary line detection model to obtain the target points on the full-width boundary line in the front plot image includes:
[0077] Step 1. Input the front plot image into the residual module to obtain a feature image with a preset number of channels.
[0078] In the embodiment of the present application, the front plot image is input into a convolutional layer with a convolution kernel of 7×7 for 2-fold downsampling and then the number of channels is expanded to obtain the first image feature with a preset number of channels; for the first image feature of each channel, the first image feature of this channel is input into a max pooling layer with a convolution kernel of 3×3 to obtain the second image feature of this channel; for the second image feature of each channel, the second image feature of this channel is subjected to 2-fold downsampling to obtain the third image feature of this channel; for the third image feature of each channel, the third image feature of this channel is input into a convolutional layer with a convolution kernel of 3×3 to obtain the final feature image of this channel.
[0079] Among them, the feature image output by the residual module is a feature image that is four times smaller than the front plot image.
[0080] Step 2: For the feature image of each channel, input the feature image into the channel attention module to weight the feature image, and obtain the weighted feature image.
[0081] i. Compress the feature image through global average pooling to obtain an image feature vector.
[0082] In the embodiment of the present application, the formula for compressing the feature image through global average pooling is as follows:
[0083]
[0084] where sq is the image feature vector, H is the height of the feature image, W is the width of the feature image, and u(i, j) is the feature value of the feature point with coordinates (i, j) in the feature image.
[0085] ii. Calculate the weight corresponding to the feature image according to the image feature vector.
[0086] In the embodiment of the present application, substitute the image feature vector into the following formula to obtain the weight corresponding to the feature image:
[0087] ex = σ(W 2 δ(W 1 sq));
[0088] where ex is the weight corresponding to the feature image, σ is the ReLU activation function, W 2 is the weight matrix of the second fully connected layer, δ is the sigmoid activation function, and W 1 is the weight matrix of the first fully connected layer.
[0089] iii. Multiply the feature image by the corresponding weight to obtain the weighted feature image.
[0090] Step 3: Input all the weighted feature images into the statistical grouping module to obtain the target points on the full-width boundary line of the front plot image.
[0091] i. Recombine all the weighted feature images to obtain the target feature image.
[0092] In the embodiment of the present application, input each weighted feature image into the fully connected layer and then splice the obtained one-dimensional feature images; and perform a full convolution operation on the spliced one-dimensional feature images to obtain the target feature image.
[0093] ii. Divide the target feature image into h×w cells, and add an empty column after the cells in the last column; where h is the height of the target feature image and w is the width of the target feature image.
[0094] Refer to Figure 2As shown in the figure, it is a schematic diagram of the target feature image after adding empty columns provided by the embodiment of the present application. In the target feature image after adding empty columns, the number of rows of cells is h, the number of columns is w + 1, and the number of cells is h * (w + 1).
[0095] iii. Input the target feature image after adding empty columns into the classification function to obtain the probability that there is a full-width boundary line in each cell.
[0096] In the embodiment of the present application, inputting the target feature image after adding empty columns into the classification function to obtain the probability that there is a full-width boundary line in each cell can be expressed by the following formula:
[0097] P m = f m (I), m ∈ [1, h]);
[0098] Among them, P m is the probability matrix that there is a full-width boundary line in the cells of the m-th row. P m contains the probability that there is a full-width boundary line in each cell in the m-th row. f m is the classification function, and f m realizes the ability to find the position of the cell where the full-width boundary line is located in the m-th row.
[0099] iv. Determine the coordinates of the target point as the coordinates of the cell where the probability is greater than the preset probability.
[0100] In the embodiment of the present application, the coordinates of the center point of the cell, the coordinates of the upper left corner point, or the coordinates of any other point in the cell can be determined as the coordinates of the cell.
[0101] Furthermore, the loss function of the full-width boundary line detection model is as follows:
[0102]
[0103] Among them, L is the loss value of the full-width boundary line detection model, G m is the identification (such as 0 or 1) vector that there is a full-width boundary line in the cells of the m-th row (this vector is a one-hot vector of w + 1), and G m contains the identification that there is a full-width boundary line in each cell in the m-th row. L CE is the cross-entropy loss.
[0104] S103. Correct the initial full-width boundary line according to the full-width boundary line of the front plot at the previous moment to obtain the target full-width boundary line of the front plot at the current moment.
[0105] In the embodiment of the present application, the full-width boundary line of the front plot at the previous moment and the initial full-width boundary line are substituted into the following formula to obtain the target full-width boundary line of the front plot at the current moment:
[0106]
[0107]
[0108]
[0109]
[0110]
[0111] Among them, is the curve coefficient of the full-width boundary line of the front plot at the previous moment, A is the unit matrix, k is the current moment, W k is the process noise at the current moment, is the prior curve coefficient of the full-width boundary line of the front plot at the current moment, H is the observation matrix, V k is the observation noise at the current moment, Z k is the observation value at the current moment, is the prior error covariance matrix at the current moment, P K-1 is the posterior error covariance matrix at the previous moment, T is the transpose, Q is the process error, K k is the gain, R is the observation error, is the curve coefficient of the target full-width boundary line of the front plot at the current moment, is the curve coefficient of the initial full-width boundary line.
[0112] S104. Perform steering control on the harvester according to the deviation value between the target full-width boundary line and the preset cutter bar line in the front plot image.
[0113] In the embodiment of the present application, if the deviation value is greater than or equal to the preset threshold (such as 12 cm), the harvester is controlled to steer so that the deviation value between the target full-width boundary line in the front plot image after steering and the preset cutter bar line is less than the preset threshold. If the deviation value is less than the preset threshold (such as 12 cm), the current driving state of the harvester is maintained.
[0114] Further, calculating the deviation value between the target full-width boundary line and the preset cutter bar line in the image of the front plot includes: determining the abscissa of the point with the ordinate of h / 2 on the target full-width boundary line as the first value, where h is the height of the target feature image; determining the abscissa of the point with the ordinate of h / 2 on the preset cutter bar line as the second value; and subtracting the second value from the first value to obtain the deviation value between the target full-width boundary line and the preset cutter bar line in the image of the front plot.
[0115] Wherein, the cutter bar line refers to the boundary line of the cutter bar part in agricultural machinery, that is, the boundary where the cutter bar contacts the crops during field operation.
[0116] Optionally, controlling the harvester to turn includes: calculating the angle between the tangent of the point with the ordinate of h / 2 on the target full-width boundary line and the tangent of the point with the ordinate of h / 2 on the preset cutter bar line, and controlling the harvester to turn based on this angle, so that the deviation value between the target full-width boundary line and the preset cutter bar line in the image of the front plot after the harvester turns is less than the preset threshold.
[0117] The embodiment of the present application provides a control method for a harvester. The method includes: acquiring the image of the front plot of the harvester at the current moment and the full-width boundary line of the front plot at the previous moment; inputting the image of the front plot into the full-width boundary line detection model to obtain the target points on the full-width boundary line in the image of the front plot; fitting all the target points into a polynomial curve to obtain the initial full-width boundary line of the front plot at the current moment; correcting the initial full-width boundary line according to the full-width boundary line of the front plot at the previous moment to obtain the target full-width boundary line of the front plot at the current moment; and controlling the turning of the harvester according to the deviation value between the target full-width boundary line and the preset cutter bar line in the image of the front plot. Through the method of the present application, the cutting width rate and harvesting efficiency of crops can be improved.
[0118] Based on the same inventive concept, the embodiment of the present application also provides a control device for a harvester corresponding to the control method of the harvester. Since the principle of solving problems by the device in the embodiment of the present application is similar to that of the control method of the harvester in the above embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.
[0119] Refer to Figure 3 As shown, it is a schematic diagram of a control device for a harvester provided by the embodiment of the present application. The control device for the harvester includes:
[0120] An acquisition module 301, configured to acquire the image of the front plot of the harvester at the current moment and the full-width boundary line of the front plot at the previous moment;
[0121] An input fitting module 302, configured to input the front plot image into a full-frame boundary line detection model to obtain target points on the full-frame boundary line in the front plot image; fit all the target points into a polynomial curve to obtain an initial full-frame boundary line of the front plot at the current moment;
[0122] A correction module 303, configured to correct the initial full-frame boundary line according to the full-frame boundary line of the front plot at the previous moment to obtain a target full-frame boundary line of the front plot at the current moment;
[0123] A control module 304, configured to perform steering control on the harvester according to the deviation value between the target full-frame boundary line and a preset cutter bar line in the front plot image.
[0124] In a possible implementation manner, the full-frame boundary line detection model includes a residual module, a channel attention module, and a statistical grouping module; the input fitting module 302 is specifically configured to input the front plot image into the residual module to obtain a feature image with a preset number of channels; for the feature image of each channel, input the feature image into the channel attention module to weight the feature image to obtain a weighted feature image; input all the weighted feature images into the statistical grouping module to obtain target points on the full-frame boundary line in the front plot image.
[0125] In a possible implementation manner, the input fitting module 302 is specifically configured to compress the feature image through global average pooling to obtain an image feature vector; calculate the weight corresponding to the feature image according to the image feature vector; multiply the feature image by the corresponding weight to obtain a weighted feature image.
[0126] In a possible implementation manner, the input fitting module 302 is specifically configured to reorganize all the weighted feature images to obtain a target feature image; divide the target feature image into h×w cells, and add an empty column after the cells in the last column; where h is the height of the target feature image and w is the width of the target feature image; input the target feature image after adding the empty column into a classification function to obtain the probability that there is a full-frame boundary line in each cell; determine the coordinates of the cells with a probability greater than a preset probability as the coordinates of the target points.
[0127] In a possible implementation manner, the correction module 303 is specifically configured to substitute the full-frame boundary line of the front plot at the previous moment and the initial full-frame boundary line into the following formula to obtain the target full-frame boundary line of the front plot at the current moment:
[0128]
[0129]
[0130]
[0131]
[0132]
[0133] Among them, is the curve coefficient of the full-width boundary line of the front plot at the previous moment, A is the unit matrix, k is the current moment, and W k is the process noise at the current moment, is the prior curve coefficient of the full-width boundary line of the front plot at the current moment, H is the observation matrix, and V k is the observation noise at the current moment, and Z k is the observation value at the current moment, is the prior error covariance matrix at the current moment, P K-1 is the posterior error covariance matrix at the previous moment, T is the transpose, Q is the process error, and K k is the gain, R is the observation error, is the curve coefficient of the target full-width boundary line of the front plot at the current moment, is the curve coefficient of the initial full-width boundary line.
[0134] In a possible implementation manner, the control module 304 is further configured to:
[0135] Determine the abscissa of the point on the target full-width boundary line with the ordinate of h / 2 as the first value; where h is the height of the target feature image;
[0136] Determine the abscissa of the point on the preset cutter bar line with the ordinate of h / 2 as the second value;
[0137] Subtract the second value from the first value to obtain the deviation value between the target full-width boundary line and the preset cutter bar line in the front plot image.
[0138] In a possible implementation manner, the control module 304 is specifically configured to, if the deviation value is greater than or equal to a preset threshold, control the harvester to turn so that the deviation value between the target full-width boundary line and the preset cutter bar line in the front plot image after the harvester turns is less than the preset threshold.
[0139] The present application provides a control device for a harvester. The device includes: an acquisition module 301, configured to acquire an image of the front plot of the harvester at the current moment, and the full-width boundary line of the front plot at the previous moment; an input fitting module 302, configured to input the front plot image into a full-width boundary line detection model to obtain target points on the full-width boundary line in the front plot image; fit all the target points into a polynomial curve to obtain an initial full-width boundary line of the front plot at the current moment; a correction module 303, configured to correct the initial full-width boundary line according to the full-width boundary line of the front plot at the previous moment to obtain a target full-width boundary line of the front plot at the current moment; and a control module 304, configured to perform steering control on the harvester according to the deviation value between the target full-width boundary line and a preset cutter bar line in the front plot image. By the method of the present application, the cutting width rate and the harvesting efficiency of crops can be improved.
[0140] As Figure 4 shown, an electronic device 400 provided by an embodiment of the present application includes: a processor 401, a memory 402, and a bus. The memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device runs, communication is performed between the processor 401 and the memory 402 through the bus, and the processor 401 executes the machine-readable instructions to perform the steps of the control method of the harvester as described above.
[0141] Specifically, the above-mentioned memory 402 and processor 401 can be general-purpose memory and processor, and no specific limitation is made here. When the processor 401 runs the computer program stored in the memory 402, it can execute the control method of the harvester as described above.
[0142] Corresponding to the above control method of the harvester, an embodiment of the present application further provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, it executes the steps of the control method of the harvester as described above.
[0143] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the method embodiments, and will not be elaborated herein. In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed among each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or modules can be in electrical, mechanical, or other forms.
[0144] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0145] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.
[0146] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the information processing method described in each embodiment of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.
[0147] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims
1. A control method for a harvester, characterized in that: The method comprises: Obtain the image of the field in front of the harvester at the current moment, and the full boundary line of the field in front of the harvester at the previous moment; Inputting the front land block image into a full-width boundary line detection model to obtain target points on the full-width boundary line in the front land block image; fitting all target points into a polynomial curve to obtain the initial full-width boundary line of the front land block at the current moment; Correcting the initial full-width boundary line according to the full-width boundary line of the front plot at the previous moment to obtain a target full-width boundary line of the front plot at the current moment; The harvester is steered according to a deviation value between the target full-width boundary line and a preset header line in the front plot image.
2. The control method of the harvester according to claim 1, characterized in that: The full-width boundary line detection model includes a residual module, a channel attention module and a statistical grouping module; the step of inputting the front land block image into the full-width boundary line detection model to obtain a target point on the full-width boundary line in the front land block image includes: Inputting the front land block image into the residual module to obtain a feature image with a preset number of channels; For each channel's feature image, input the feature image into the channel attention module to weight the feature image to obtain a weighted feature image; All weighted feature images are input into the statistical grouping module to obtain target points on the full-width boundary line in the front land image.
3. The control method of the harvester according to claim 2, characterized in that: The step of inputting the feature image into the channel attention module to weight the feature image to obtain a weighted feature image comprises: Compressing the feature image by global average pooling to obtain an image feature vector; Calculate the weight corresponding to the feature image according to the image feature vector; The feature image is multiplied by the corresponding weight to obtain a weighted feature image.
4. The control method of the harvester according to claim 2, characterized in that: The step of inputting all weighted feature images into the statistical grouping module to obtain target points on the full-width boundary line in the front land image includes: All weighted feature images are reorganized to obtain the target feature image; The target feature image is divided into h×w cells, and a blank column is added after the last column of cells; wherein h is the height of the target feature image, and w is the width of the target feature image; Input the target feature image after adding the empty column into the classification function to obtain the probability of the existence of a full-width boundary line in each cell; The coordinates of the cells whose probabilities are greater than the preset probabilities are determined as the coordinates of the target points.
5. The control method of the harvester according to claim 1, characterized in that: The step of correcting the initial full-width boundary line according to the full-width boundary line of the front plot at the previous moment to obtain the target full-width boundary line of the front plot at the current moment includes: Substitute the full-width boundary line of the front plot at the previous moment and the initial full-width boundary line into the following formula to obtain the target full-width boundary line of the front plot at the current moment: in, is the curve coefficient of the full boundary line of the front plot at the previous moment, A is the unit matrix, k is the current moment, W k is the process noise at the current moment, is the priori curve coefficient of the full boundary line of the front block at the current moment, H is the observation matrix, V k is the observation noise at the current moment, Z k is the observed value at the current moment, is the prior error covariance matrix at the current moment, P K-1 is the posterior error covariance matrix of the previous moment, T is the transpose, Q is the process error, K k is the gain, R is the observation error, is the curve coefficient of the target full-width boundary line of the front plot at the current moment, is the curve coefficient of the initial full-scale boundary line.
6. The control method of the harvester according to claim 1, characterized in that: The deviation value between the target full-width boundary line and the preset header line in the front plot image is calculated by the following steps: The horizontal coordinate of the point whose vertical coordinate is h / 2 on the full-width boundary line of the target is determined as a first value; wherein h is the height of the target feature image; The horizontal coordinate of the point whose vertical coordinate is h / 2 on the preset header line is determined as a second value; The second value is subtracted from the first value to obtain a deviation value between the target full-width boundary line and a preset header line in the front land image.
7. The control method of the harvester according to claim 1, characterized in that: The step of controlling the harvester to steer according to the deviation between the target full-width boundary line and the preset header line in the front plot image comprises: If the deviation value is greater than or equal to a preset threshold, the harvester is controlled to turn so that the deviation value between the target full-width boundary line and the preset header line in the front plot image of the harvester after turning is less than the preset threshold.
8. A control device for a harvester, characterized in that: The device comprises: An acquisition module is used to acquire the image of the field in front of the harvester at the current moment and the full-width boundary line of the field in front of the harvester at the previous moment; An input fitting module is used to input the front land block image into the full-width boundary line detection model to obtain target points on the full-width boundary line in the front land block image; all target points are fitted into a polynomial curve to obtain the initial full-width boundary line of the front land block at the current moment; A correction module, configured to correct the initial full-width boundary line according to the full-width boundary line of the front plot at the previous moment, so as to obtain a target full-width boundary line of the front plot at the current moment; A control module is used to control the steering of the harvester according to the deviation value between the target full-width boundary line and the preset header line in the front plot image.
9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the control method of the harvester as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the control method of the harvester as described in any one of claims 1 to 7 are executed.