Image-based intelligent material scattering method and system

Abstract

The invention discloses an image-based intelligent material scattering method and system, and the method comprises the steps: S1, collecting and preprocessing material scattering data and image data at different material scattering moments, and obtaining preprocessed material scattering data, livestock image data and material scattering image data; s2, based on the preprocessed livestock image data, extracting livestock characteristics, and calculating increment characteristics and livestock weight; s3, the spreading amount is calculated according to the weight of the livestock, the spreading amount is corrected in combination with increment characteristics, and the corrected spreading amount is obtained; s4, calculating a target rotating speed of a material scattering motor according to the corrected material scattering amount, the preprocessed material scattering data and the preprocessed material scattering image data; s5, controlling the material scattering vehicle to scatter materials according to the target rotating speed of the material scattering motor; the feed scattering amount is calculated according to the livestock growth condition, image features and sensor data are fused, the target rotating speed of the feed scattering motor is intelligently controlled, and the problems that in a traditional feed scattering mode, feed is likely to be scattered outside a trough, and the feed amount is not appropriate are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent material spreading, and in particular to an image-based intelligent material spreading method and system. Background Art

[0002] Intelligent spreading is a farming method that integrates sensor technology and automated control technology. It aims to achieve accurate and efficient spreading of feed through intelligent means, thereby improving farming efficiency and reducing feed waste. Traditional spreading methods usually include manual spreading and mechanical spreading vehicles. Manual spreading mainly relies on manual hand-held or back-carrying of feed for spreading. This method is labor-intensive, inefficient, and difficult to ensure uniformity of spreading. With the development of mechanization, mechanically powered spreading vehicles have emerged. Such spreading vehicles usually control the spreading speed and direction through a mechanical transmission system, which improves efficiency and uniformity of spreading compared to manual spreading.

[0003] However, for traditional mechanical feed spreaders, the speed of the feed spreader motor is usually fixed or needs to be manually adjusted. The motor speed cannot adapt to itself. During the spreading process, if the speed of the feed spreader changes or the direction shifts, and the motor still maintains the original speed, the feed will be spread outside the predetermined area, causing waste. Therefore, the driver needs to concentrate highly on driving the feed spreader to avoid this situation, which increases the difficulty of operation and labor intensity.

[0004] On the other hand, livestock at different growth stages have different demands for feed. Traditional feed spreaders usually rely solely on manual experience to control the amount of feed spread, which can easily lead to insufficient or excess feed when farmers prepare feed, affecting the growth efficiency and breeding costs of livestock. Summary of the invention

[0005] In view of this, the present invention proposes an image-based intelligent feeding method and system, aiming to solve the problem that the traditional feeding method easily spreads the feed outside the trough and the feed amount is inappropriate.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] An image-based intelligent spreading method, comprising:

[0008] S1: collecting and preprocessing feeding data and image data at different feeding time, and obtaining preprocessed feeding data, preprocessed livestock image data and preprocessed feeding image data;

[0009] S2: based on the preprocessed livestock image data, extract livestock image features, calculate incremental features, and then calculate livestock weight according to the incremental features;

[0010] S3: Calculate the amount of feed spread according to the weight of the livestock, and correct the amount of feed spread in combination with the incremental characteristics to obtain the corrected amount of feed spread;

[0011] S4: Calculate the target speed of the material spreading motor according to the corrected material spreading amount, the preprocessed material spreading data and the preprocessed material spreading image data;

[0012] S5: Control the material spreading vehicle to spread the material according to the calculated target speed of the material spreading motor.

[0013] Furthermore, in step S1, the feeding data and the image data at different feeding times are collected and preprocessed to obtain the preprocessed feeding data, the preprocessed livestock image data and the preprocessed feeding image data, including:

[0014] S11: collecting material spreading data, wherein the material spreading data includes a material spreading distance between the material spreading vehicle and the trough and a moving speed of the material spreading vehicle; the material spreading distance between the material spreading vehicle and the trough is obtained by measuring the distance between the wheel track and the trough, and the moving speed of the material spreading vehicle is obtained by an acceleration sensor;

[0015] S12: Traverse the moving speed of the feeding vehicle, use the 3σ principle to remove outliers, and obtain the preprocessed moving speed of the feeding vehicle; then use the mean interpolation method to interpolate every two data points for the feeding distance between the feeding vehicle and the trough, and obtain the preprocessed feeding distance. The preprocessed moving speed of the feeding vehicle and the preprocessed feeding distance constitute the preprocessed feeding data;

[0016] S13: collecting image data at different material spreading moments at a frame rate of 30 frames per second through a video camera installed on the material spreading vehicle;

[0017] S14: cropping the image data at different feeding moments, taking the part containing livestock as livestock image data and the remaining part as feeding image data, and then adjusting the size of the livestock image data and feeding image data to 640 pixels * 640 pixels to obtain preprocessed livestock image data and preprocessed feeding image data.

[0018] Furthermore, in step S2, based on the preprocessed livestock image data, livestock image features are extracted, incremental features are calculated, and then livestock weight is calculated according to the incremental features, including:

[0019] Firstly, livestock image features are extracted based on the preprocessed livestock image data. Secondly, the contour features and geometric features of livestock at different feeding moments are extracted based on the livestock image features, and a gating mechanism is designed to calculate the incremental features. Finally, livestock weight is calculated based on the incremental features.

[0020] Furthermore, the step S2 includes:

[0021] S21: Extract the livestock image features at the time of spreading feed, and the calculation method is:

[0022]

[0023] in, is the original feature of the livestock image at the moment of spreading feed, CNN() is the convolutional neural network operation, is the preprocessed livestock image data at the time of spreading feed, is the livestock image feature at the time of spreading feed, W down is a dimensionality reduction matrix operation, Concat() is a concatenation operation, GAP() is a global pooling operation, CNN big () is a convolutional neural network operation with a convolution kernel size greater than or equal to 64, CNN small () is a convolutional neural network operation with a convolution kernel size less than or equal to 32;

[0024] It is difficult to accurately grasp the real-time weight and growth of livestock in traditional feeding methods, resulting in inaccurate feeding amount, which may cause feed waste or livestock malnutrition; at the same time, traditional methods mostly rely on manual experience to estimate livestock weight, which lacks scientific basis; the present invention uses convolutional neural networks and global pooling and other operations to calculate livestock image features, and uses convolutional neural networks with different convolution kernel sizes, uses large convolution kernels to capture more extensive context information, and uses small convolution kernels to capture detail information, and finally reduces the dimension of the features through a dimensionality reduction matrix, removes redundant information, and fuses the above features to obtain livestock image features; the extracted livestock image features are used to correct the geometric features such as the contour and area of ​​the livestock in subsequent steps, so as to more accurately estimate the weight of the livestock;

[0025] S22: Combined with livestock image features at the time of spreading feed The preprocessed livestock image data at the time of spreading the feed is subjected to hierarchical feature extraction to extract the first contour feature and the first geometric feature of the livestock. The calculation method is:

[0026]

[0027] G (1) =MLP(Concat(L,S))

[0028] in, is the original contour feature, Canny() is the Canny edge detection operation, is the first outline feature of livestock, L (1) ,S (1) are the first perimeter feature and the first area feature of livestock respectively. DenseNet() is the DenseNet network operation. G(1) It is the first geometric characteristic of livestock;

[0029] The traditional Canny edge detection algorithm is often unable to accurately extract the contour of the target object when processing images containing complex backgrounds or interference factors. In particular, in this scenario, the presence of the trough guardrail will seriously interfere with the extraction of the livestock contour. Directly using the Canny edge detection algorithm will cause the extracted contour features to contain a large number of errors, thereby affecting the subsequent livestock weight estimation and feed spreading amount calculation. The present invention uses the results obtained by the Canny edge detection algorithm as the original contour features, and then corrects the edge detection errors caused by the trough guardrail by combining the livestock image features calculated in step S21, thereby more accurately extracting the livestock contour features, and providing more accurate basic data for the subsequent extraction of livestock geometric features (such as perimeter, area, etc.);

[0030] The above steps of extracting the first contour feature and the first geometric feature of the livestock are uniformly expressed as follows:

[0031]

[0032] Among them, Feature() is a feature extraction operation;

[0033] S23: Preprocessed livestock image data based on feeding time 2 The method in step S21 is used to extract the livestock image features at the second feeding time. Combined with livestock image features at feeding time 2 The method in step S22 is used to process the preprocessed livestock image data at the second feeding time. Perform hierarchical feature extraction to extract the second contour features of livestock and the second geometric characteristic G of livestock (2) , calculated as:

[0034]

[0035] S24: Design a gating mechanism to calculate the incremental features. The calculation method is:

[0036]

[0037] Gate = sigmoid(W1·(E (1) ,E (2) )+b)

[0038] ΔF=Gate·(E (1) -E (2) )

[0039] Among them, E (1) For the first livestock characteristic, E(2) is the second livestock feature, Gate is the incremental gate, sigmoid() is the sigmoid function, W1 is the incremental weight matrix, b is the incremental bias vector, ΔF is the incremental feature;

[0040] In order to adapt to the characteristics of livestock weight and morphology changing over time, the present invention combines livestock image data at different times and expresses the changing characteristics of livestock by calculating incremental features; wherein, the incremental gate is designed to adjust the calculation of incremental features, when the livestock features at two times are greatly different, the Gate value is close to 1, and the calculation result of the incremental features is large, and when the livestock features at two times are less different, the Gate value is close to 0, and the calculation result of the incremental features is small; this mechanism helps to pay more attention when the feature changes greatly, and reduce unnecessary interference when the feature changes slightly, making the entire intelligent spreading system more flexible and intelligent;

[0041] S25: Calculate the livestock weight based on the incremental characteristics, the calculation method is:

[0042]

[0043] Among them, Weight is the weight of livestock.

[0044] Furthermore, in step S3, the feed spreading amount is calculated according to the livestock weight, and the feed spreading amount is corrected in combination with the incremental characteristics to obtain the corrected feed spreading amount, including:

[0045] S31: Calculate the amount of feed to be spread based on the weight of the livestock. The calculation method is:

[0046] A=SVR(Weight)

[0047] Among them, A is the spreading amount, SVR() is the support vector regression model;

[0048] S32: Correct the spreading amount according to the incremental feature to obtain the corrected spreading amount, which is calculated as follows:

[0049] A′=A+SVR(ΔF)

[0050] Among them, A ′ is the corrected spreading amount;

[0051] The present invention calculates a preliminary feeding amount A according to the weight of livestock through a support vector regression model (SVR), and then uses the SVR model again to correct the feeding amount according to an incremental feature ΔF to obtain a corrected feeding amount A'; the incremental feature ΔF reflects the characteristic changes of livestock between different feeding times, and can be converted into a correction value for the feeding amount through the SVR model, thereby realizing dynamic adjustment of the feeding amount.

[0052] Furthermore, in step S4, the target speed of the material spreading motor is calculated according to the corrected material spreading amount, the preprocessed material spreading data and the preprocessed material spreading image data, including:

[0053] Firstly, the reference speed of the spreading motor is calculated according to the corrected spreading amount and the moving speed of the spreading vehicle after preprocessing. Secondly, the original spreading features are extracted according to the preprocessed spreading image data, and the spreading deviation category distribution is calculated according to the original spreading features. Then, the spreading deviation features are calculated based on the spreading distance between the spreading vehicle and the trough and the spreading deviation category distribution. Finally, the target speed of the spreading motor is calculated based on the reference speed of the spreading motor and the spreading deviation features.

[0054] Further, the step S4 includes:

[0055] S41: Calculate the reference speed of the material spreading motor according to the corrected material spreading amount and the moving speed of the material spreading vehicle after pre-processing. The calculation method is:

[0056]

[0057] Among them, N base is the reference speed of the material spreading motor, K is the proportional factor, which is used to convert the ratio of the material spreading amount to the speed of the spreading vehicle into the motor speed, V vehicle is the moving speed of the material spreading vehicle after pretreatment, and B is the reference offset, which is used to adjust the reference value of the motor speed;

[0058] S42: extracting original spreading features according to the preprocessed spreading image data, and calculating the spreading deviation category distribution according to the original spreading features; and then calculating the spreading deviation features in combination with the spreading distance between the spreading vehicle and the trough and the spreading deviation category distribution, using the following calculation method:

[0059] F feed =(W′ down Concat(GAP(P feed ),CNN big (P feed ),CNN small (P feed ))

[0060] p = softmax(MLP(F feed ))

[0061] F′ feed =Concat(F feed ,p)

[0062] Among them, F feed is the original spreading characteristic, W′ down is the weight matrix of the scattering and downsampling, p feedis the preprocessed scattering image data, p is the scattering deviation category distribution (no deviation, deviation, severe deviation), softmax() is the softmax function, D is the preprocessed scattering distance, F′ feed It is the material spreading deviation characteristic;

[0063] Step S42 is intended to calculate the spreading deviation feature to evaluate whether the feed falls into the trough accurately and the degree of deviation during the spreading process. The present invention proposes to calculate the spreading deviation feature directly by analyzing the spreading image data, and adjust the motor speed accordingly. First, the preprocessed spreading image data is subjected to global, large and local feature extraction through a convolutional neural network. Then, these features are input into a multi-layer perceptron to calculate the spreading deviation category distribution p, where p represents the degree of deviation of the feed falling into the trough during the spreading process, and is divided into three categories: no deviation, deviation and severe deviation. Finally, the original spreading feature and the spreading deviation category distribution p are spliced ​​to obtain the spreading deviation feature, which is used in the subsequent spreading motor target speed calculation step to provide information for adjusting the motor speed.

[0064] S43: Calculate the target speed of the spreading motor by combining the reference speed of the spreading motor and the spreading deviation characteristics. The calculation method is:

[0065] N final =N base +MLP(F′ feed )

[0066] Among them, N final It is the target speed of the spreading motor.

[0067] Furthermore, in step S5, controlling the material spreading vehicle to spread material according to the calculated target speed of the material spreading motor includes:

[0068] S51: inputting the calculated target speed of the material spreading motor into the material spreading vehicle motor control system, and converting the target speed of the material spreading motor into a digital signal of the material spreading vehicle motor;

[0069] S52: Using PID control algorithm, adjusting the control signal according to the deviation between the actual speed of the spreading motor and the target speed of the spreading motor;

[0070] S53: Monitor the actual speed of the material spreading motor in real time and feed it back to the PID control algorithm to adjust the target speed of the material spreading motor in real time.

[0071] The present invention also discloses an image-based intelligent material spreading system, comprising:

[0072] Data acquisition and preprocessing module: collects and preprocesses the spreading data and image data at different spreading times to obtain preprocessed spreading data, preprocessed livestock image data and preprocessed spreading image data;

[0073] Livestock feature extraction module: based on the preprocessed livestock image data, extract livestock image features, calculate incremental features, and then calculate livestock weight based on the incremental features;

[0074] Feeding amount calculation module: calculates the feeding amount according to the weight of livestock, and corrects the feeding amount based on the incremental characteristics to obtain the corrected feeding amount;

[0075] Spreading motor target speed calculation module: calculates the spreading motor target speed according to the corrected spreading amount, preprocessed spreading data and preprocessed spreading image data;

[0076] Material spreading control module: controls the material spreading vehicle to spread materials according to the calculated target speed of the material spreading motor.

[0077] Compared with the prior art, the advantages of the present invention are:

[0078] (1) The present invention first extracts the contour features and geometric features of livestock based on image data to provide basic data for subsequent calculation of feeding amount; secondly, by comparing the livestock features at different feeding times, the incremental features are calculated, and the feeding amount is adjusted according to the incremental features and the livestock weight, ensuring that the feeding amount always matches the actual needs of the livestock; finally, during the feeding process, the present invention detects the feeding deviation in real time based on the image, and adjusts the speed of the feeding motor according to the degree of deviation, ensuring that the feed can accurately fall into the trough, reducing the situation of feeding outside the trough;

[0079] (2) The present invention addresses the problem that the trough guardrail interferes with the extraction of livestock contours. The result obtained by the Canny edge detection algorithm is used as the original contour feature. Then, by combining the calculated livestock image features, the edge detection error caused by the trough guardrail is corrected, thereby providing more accurate basic data for the subsequent extraction of livestock geometric features (such as perimeter, area, etc.);

[0080] (3) The present invention proposes a design of an incremental gate, and calculates the gate value through a sigmoid function. When the difference between livestock characteristics at two moments is large, the incremental gate value is close to 1, and the calculation result of the incremental characteristic is large; when the difference is small, the incremental gate value is close to 0, and the calculation result of the incremental characteristic is small, which helps to pay more attention when the livestock characteristics change greatly, and reduce unnecessary interference when the livestock characteristics change slightly, making the entire intelligent feeding system more flexible and intelligent;

[0081] (4) The present invention introduces the incremental characteristics and livestock weight as the basis for calculating the amount of feed to be spread, thereby realizing dynamic adjustment of the amount of feed to be spread, and can adjust the amount of feed to be spread according to the actual growth conditions of the livestock, thereby avoiding feed waste and shortage and improving feed utilization;

[0082] (5) Traditional material spreading deviation detection usually relies on expensive distance sensors, while the present invention proposes a low-cost image-based material spreading deviation detection method, which reduces the cost of the system by analyzing the material spreading image data, calculating the material spreading deviation characteristics, and adjusting the motor speed accordingly. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 A schematic diagram of the process of the image-based intelligent material spreading method provided by the present invention;

[0084] Figure 2 A schematic diagram of the algorithm flow for extracting livestock contour features and geometric features provided by the present invention;

[0085] Figure 3 A schematic diagram of the algorithm flow of incremental feature extraction provided by the present invention. DETAILED DESCRIPTION

[0086] The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

[0087] Embodiment 1: An intelligent material spreading method based on an image, such as Figure 1 As shown, the following steps are included:

[0088] S1: collecting and preprocessing feeding data and image data at different feeding time, and obtaining preprocessed feeding data, preprocessed livestock image data and preprocessed feeding image data;

[0089] S11: collecting material spreading data, wherein the material spreading data includes a material spreading distance between the material spreading vehicle and the trough and a moving speed of the material spreading vehicle; the material spreading distance between the material spreading vehicle and the trough is obtained by measuring the distance between the wheel track and the trough, and the moving speed of the material spreading vehicle is obtained by an acceleration sensor;

[0090] S12: Traverse the moving speed of the feeding vehicle, use the 3σ principle to remove outliers, and obtain the preprocessed moving speed of the feeding vehicle; then use the mean interpolation method to interpolate every two data points for the feeding distance between the feeding vehicle and the trough, and obtain the preprocessed feeding distance. The preprocessed moving speed of the feeding vehicle and the preprocessed feeding distance constitute the preprocessed feeding data;

[0091] S13: collecting image data at different material spreading moments at a frame rate of 30 frames per second through a video camera installed on the material spreading vehicle;

[0092] S14: cropping the image data at different feeding moments, taking the part containing livestock as livestock image data and the remaining part as feeding image data, and then adjusting the size of the livestock image data and feeding image data to 640 pixels * 640 pixels to obtain preprocessed livestock image data and preprocessed feeding image data.

[0093] S2: based on the preprocessed livestock image data, extract livestock image features, calculate incremental features, and then calculate livestock weight according to the incremental features;

[0094] S21: Extract the livestock image features at the time of spreading feed, and the calculation method is:

[0095]

[0096] in, is the original feature of the livestock image at the moment of spreading feed, CNN() is the convolutional neural network operation, is the preprocessed livestock image data at the time of spreading feed, is the livestock image feature at the time of spreading feed, W down is a dimensionality reduction matrix operation, Concat() is a concatenation operation, GAP() is a global pooling operation, CNN big () is a convolutional neural network operation with a convolution kernel size greater than or equal to 64, CNN small () is a convolutional neural network operation with a convolution kernel size less than or equal to 32;

[0097] It is difficult to accurately grasp the real-time weight and growth of livestock in traditional feeding methods, resulting in inaccurate feeding amount, which may cause feed waste or livestock malnutrition; at the same time, traditional methods mostly rely on manual experience to estimate livestock weight, which lacks scientific basis; the present invention uses convolutional neural networks and global pooling and other operations to calculate livestock image features, and uses convolutional neural networks with different convolution kernel sizes, uses large convolution kernels to capture more extensive context information, and uses small convolution kernels to capture detail information, and finally reduces the dimension of the features through a dimensionality reduction matrix, removes redundant information, and fuses the above features to obtain livestock image features; the extracted livestock image features are used to correct the geometric features such as the contour and area of ​​the livestock in subsequent steps, so as to more accurately estimate the weight of the livestock;

[0098] S22: Combined with livestock image features at the time of spreading feed The preprocessed livestock image data at the time of spreading the feed is subjected to hierarchical feature extraction to extract the first contour feature and the first geometric feature of the livestock, such as Figure 2 As shown, the calculation method is:

[0099]

[0100] G (1) =MLP(Concat(L,S))

[0101] in, is the original contour feature, Canny() is the Canny edge detection operation, is the first outline feature of livestock, L (1) ,S (1) are the first perimeter feature and the first area feature of livestock respectively. DenseNet() is the DenseNet network operation. G (1) It is the first geometric characteristic of livestock;

[0102] The traditional Canny edge detection algorithm is often unable to accurately extract the contour of the target object when processing images containing complex backgrounds or interference factors. In particular, in this scenario, the presence of the trough guardrail will seriously interfere with the extraction of the livestock contour. Directly using the Canny edge detection algorithm will cause the extracted contour features to contain a large number of errors, thereby affecting the subsequent livestock weight estimation and feed spreading amount calculation. The present invention uses the results obtained by the Canny edge detection algorithm as the original contour features, and then corrects the edge detection errors caused by the trough guardrail by combining the livestock image features calculated in step S21, thereby more accurately extracting the livestock contour features, and providing more accurate basic data for the subsequent extraction of livestock geometric features (such as perimeter, area, etc.);

[0103] The above steps of extracting the first contour feature and the first geometric feature of the livestock are uniformly expressed as follows:

[0104]

[0105] Among them, Feature() is a feature extraction operation;

[0106] S23: Preprocessed livestock image data based on feeding time 2 The method in step S21 is used to extract the livestock image features at the second feeding time. Combined with livestock image features at feeding time 2 The method in step S22 is used to process the preprocessed livestock image data at the second feeding time. Perform hierarchical feature extraction to extract the second contour features of livestock and the second geometric characteristic G of livestock (2) , calculated as:

[0107]

[0108] S24: Design a gating mechanism to calculate incremental features, such as Figure 3 As shown, the calculation method is:

[0109]

[0110] Gate = sigmoid(W1·(E (1) ,E (2) )+b)

[0111] ΔF=Gate·(E (1) -E (2) )

[0112] Among them, E (1) For the first livestock characteristic, E (2) is the second livestock feature, Gate is the incremental gate, sigmoid() is the sigmoid function, W1 is the incremental weight matrix, b is the incremental bias vector, ΔF is the incremental feature;

[0113] In order to adapt to the characteristics of livestock weight and morphology changing over time, the present invention combines livestock image data at different times and expresses the changing characteristics of livestock by calculating incremental features; wherein, the incremental gate is designed to adjust the calculation of incremental features, when the livestock features at two times are greatly different, the Gate value is close to 1, and the calculation result of the incremental features is large, and when the livestock features at two times are less different, the Gate value is close to 0, and the calculation result of the incremental features is small; this mechanism helps to pay more attention when the feature changes greatly, and reduce unnecessary interference when the feature changes slightly, making the entire intelligent spreading system more flexible and intelligent;

[0114] S25: Calculate the livestock weight based on the incremental characteristics, the calculation method is:

[0115]

[0116] Among them, Weight is the weight of livestock.

[0117] S3: Calculate the amount of feed spread according to the weight of the livestock, and correct the amount of feed spread in combination with the incremental characteristics to obtain the corrected amount of feed spread;

[0118] S31: Calculate the amount of feed to be spread based on the weight of the livestock. The calculation method is:

[0119] A=SVR(Weight)

[0120] Among them, A is the spreading amount, SVR() is the support vector regression model;

[0121] S32: Correct the spreading amount according to the incremental feature to obtain the corrected spreading amount, which is calculated as follows:

[0122] A′=A+SVR(ΔF)

[0123] Among them, A′ is the corrected spreading amount;

[0124] For example, suppose that in a farm, the weight calculated at time 1 is Weight1 = 150 kg, the weight calculated at time 2 is Weight2 = 165 kg, and the calculated incremental feature ΔF = 0.1;

[0125] Using the weight Weight2 at time 2 as input, the amount of material spread calculated by the SVR model is A=5kg. Using the incremental feature ΔF as input, the correction amount calculated by the SVR model is SVR(ΔF)=0.5kg. Then the corrected amount of material spread is A′=A+SVR(ΔF)=5kg+0.5kg=5.5kg.

[0126] The present invention calculates a preliminary feeding amount A according to the weight of livestock through a support vector regression model (SVR), and then uses the SVR model again to correct the feeding amount according to an incremental feature ΔF to obtain a corrected feeding amount A'; the incremental feature ΔF reflects the characteristic changes of livestock between different feeding times, and can be converted into a correction value for the feeding amount through the SVR model, thereby realizing dynamic adjustment of the feeding amount.

[0127] S4: Calculate the target speed of the material spreading motor according to the corrected material spreading amount, the preprocessed material spreading data and the preprocessed material spreading image data;

[0128] S41: Calculate the reference speed of the material spreading motor according to the corrected material spreading amount and the moving speed of the material spreading vehicle after pre-processing. The calculation method is:

[0129]

[0130] Among them, N base is the reference speed of the material spreading motor, K is the proportional factor, which is used to convert the ratio of the material spreading amount to the speed of the spreading vehicle into the motor speed, V vehicle is the moving speed of the material spreading vehicle after pretreatment, and B is the reference offset, which is used to adjust the reference value of the motor speed;

[0131] S42: extracting original spreading features according to the preprocessed spreading image data, and calculating the spreading deviation category distribution according to the original spreading features; and then calculating the spreading deviation features in combination with the spreading distance between the spreading vehicle and the trough and the spreading deviation category distribution, using the following calculation method:

[0132] F feed =(W′ down Concat(GAP(P feed ),CNN big (P feed ),CNN small (P feed ))

[0133] p = softmax(MLP(F feed ))

[0134] F′ feed =Concat(F feed ,p)

[0135] Among them, F feed is the original spreading characteristic, W′ down is the weight matrix of the scattering and downsampling, P feed is the preprocessed scattering image data, p is the scattering deviation category distribution, softmax() is the softmax function, D is the preprocessed scattering distance, F′ feed It is the material spreading deviation characteristic;

[0136] In addition, when the spreader is getting farther and farther away from the trough, resulting in multiple consecutive serious deviations or cross-occurrence of "deviation" and "serious deviation", the system can automatically detect such situations and issue an alarm to the driver. The calculation method is:

[0137]

[0138] pro=sigmoid(W2·U+b2)

[0139] Among them, U is the deviation alarm feature, GRU5() is a gated recurrent unit operation with a length of 5, is the original spreading characteristic at time t, p t is the distribution of material spreading deviation categories at time t, por is the deviation alarm probability, W2 is the deviation alarm matrix, and b2 is the deviation alarm bias vector;

[0140] The gated recurrent unit with a length of 5 performs feature extraction in groups of 5 time units, and performs a sliding window operation in 2 time units. When por>0.5 is detected, a deviation alarm is issued. When por≤0.5 is detected, the sliding window is moved once to perform deviation alarm detection for the next time unit.

[0141] For example, if there are 9 time units of data, GRU5() first takes the 1st, 2nd, 3rd, 4th, and 5th data as input. If pro>0.5 is detected, a deviation alarm is issued. If pro≤0.5 is detected, a sliding window operation is performed, and the next round of detection is performed with the 3rd, 4th, 5th, 6th, and 7th data as input.

[0142] Step S42 is intended to calculate the spreading deviation feature to evaluate whether the feed falls into the trough accurately and the degree of deviation during the spreading process. The present invention proposes to calculate the spreading deviation feature directly by analyzing the spreading image data, and adjust the motor speed accordingly. First, the preprocessed spreading image data is subjected to global, large and local feature extraction through a convolutional neural network. Then, these features are input into a multi-layer perceptron to calculate the spreading deviation category distribution p, where p represents the degree of deviation of the feed falling into the trough during the spreading process, and is divided into three categories: no deviation, deviation and severe deviation. Finally, the original spreading feature and the spreading deviation category distribution p are spliced ​​to obtain the spreading deviation feature, which is used in the subsequent spreading motor target speed calculation step to provide information for adjusting the motor speed.

[0143] S43: Calculate the target speed of the spreading motor by combining the reference speed of the spreading motor and the spreading deviation characteristics. The calculation method is:

[0144] N final =N base +MLP(F ′ feed )

[0145] Among them, N final is the target speed of the spreading motor;

[0146] For example, the moving speed of the spreading vehicle after pretreatment is V vehicle =0.5m / s, the corrected spreading amount A′=5.5kg, the proportional factor K=20, the reference offset B=50, the spreading distance between the spreading vehicle and the trough: D=1m, then RPM;

[0147] The calculated distribution of the spreading deviation category p is [0.1, 0.7, 0.2], which means that the probability of no deviation is 0.1, the probability of deviation is 0.7, and the probability of severe deviation is 0.2; MLP(F′ feed ) is calculated to be 10 rpm,

[0148] Therefore, the corrected target speed of the spreading motor is: N final =N base +MLP(F′ feed )=270+10=280 rpm.

[0149] S5: Control the material spreading vehicle to spread the material according to the calculated target speed of the material spreading motor;

[0150] S51: inputting the calculated target speed of the material spreading motor into the material spreading vehicle motor control system, and converting the target speed of the material spreading motor into a digital signal of the material spreading vehicle motor;

[0151] S52: Using PID control algorithm, adjusting the control signal according to the deviation between the actual speed of the spreading motor and the target speed of the spreading motor;

[0152] S53: Monitor the actual speed of the material spreading motor in real time and feed it back to the PID control algorithm to adjust the target speed of the material spreading motor in real time.

[0153] Embodiment 2: The present invention also discloses an image-based intelligent material spreading system, comprising:

[0154] Data acquisition and preprocessing module: collects and preprocesses the spreading data and image data at different spreading times to obtain preprocessed spreading data, preprocessed livestock image data and preprocessed spreading image data;

[0155] Livestock feature extraction module: based on the preprocessed livestock image data, extract livestock image features, calculate incremental features, and then calculate livestock weight based on the incremental features;

[0156] Feeding amount calculation module: calculates the feeding amount according to the weight of livestock, and corrects the feeding amount based on the incremental characteristics to obtain the corrected feeding amount;

[0157] Spreading motor target speed calculation module: calculates the spreading motor target speed according to the corrected spreading amount, preprocessed spreading data and preprocessed spreading image data;

[0158] Material spreading control module: controls the material spreading vehicle to spread materials according to the calculated target speed of the material spreading motor.

[0159] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0160] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in multiple embodiments of the present invention.

[0161] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image-based intelligent material spreading method, characterized in that: The following steps are involved: S1: collecting and preprocessing feeding data and image data at different feeding time, and obtaining preprocessed feeding data, preprocessed livestock image data and preprocessed feeding image data; S2: based on the preprocessed livestock image data, extract livestock image features, calculate incremental features, and then calculate livestock weight according to the incremental features; S3: Calculate the amount of feed spread according to the weight of the livestock, and correct the amount of feed spread in combination with the incremental characteristics to obtain the corrected amount of feed spread; S4: Calculate the target speed of the material spreading motor according to the corrected material spreading amount, the preprocessed material spreading data and the preprocessed material spreading image data; S5: Control the material spreading vehicle to spread the material according to the calculated target speed of the material spreading motor.

2. The image-based intelligent material spreading method according to claim 1, characterized in that: The step S1 comprises: S11: collecting material spreading data, wherein the material spreading data includes a material spreading distance between the material spreading vehicle and the trough and a moving speed of the material spreading vehicle; the material spreading distance between the material spreading vehicle and the trough is obtained by measuring the distance between the wheel track and the trough, and the moving speed of the material spreading vehicle is obtained by an acceleration sensor; S12: Traverse the moving speed of the feeding vehicle, use the 3σ principle to remove outliers, and obtain the preprocessed moving speed of the feeding vehicle; then use the mean interpolation method to interpolate every two data points for the feeding distance between the feeding vehicle and the trough, and obtain the preprocessed feeding distance. The preprocessed moving speed of the feeding vehicle and the preprocessed feeding distance constitute the preprocessed feeding data; S13: collecting image data at different material spreading moments at a frame rate of 30 frames per second through a video camera installed on the material spreading vehicle; S14: cropping the image data at different feeding moments, taking the part containing livestock as livestock image data and the remaining part as feeding image data, and then adjusting the size of the livestock image data and feeding image data to 640 pixels * 640 pixels to obtain preprocessed livestock image data and preprocessed feeding image data.

3. The image-based intelligent material spreading method according to claim 1, characterized in that: The step S2 comprises: Firstly, livestock image features are extracted based on the preprocessed livestock image data. Secondly, the contour features and geometric features of livestock at different feeding moments are extracted based on the livestock image features, and a gating mechanism is designed to calculate the incremental features. Finally, livestock weight is calculated based on the incremental features.

4. The image-based intelligent material spreading method according to claim 3, characterized in that: The step S2 comprises: S21: Extract the livestock image features at the time of spreading feed, and the calculation method is: in, is the original feature of the livestock image at the moment of spreading feed, CNN() is the convolutional neural network operation, is the preprocessed livestock image data at the time of spreading feed, is the livestock image feature at the time of spreading feed, W down is a dimensionality reduction matrix operation, Concat() is a concatenation operation, GAP() is a global pooling operation, CNN big () is a convolutional neural network operation with a convolution kernel size greater than or equal to 64, CNN small () is a convolutional neural network operation with a convolution kernel size less than or equal to 32; S22: Combined with livestock image features at the time of spreading feed The preprocessed livestock image data at the time of spreading the feed is subjected to hierarchical feature extraction to extract the first contour feature and the first geometric feature of the livestock. The calculation method is: G (1) =MLP(Concat(L,S)) in, is the original contour feature, Canny() is the Canny edge detection operation, is the first outline feature of livestock, L (1) ,S (1) are the first perimeter feature and the first area feature of livestock respectively. DenseNet() is the DenseNet network operation. G (1) It is the first geometric characteristic of livestock; The above steps of extracting the first contour feature and the first geometric feature of the livestock are uniformly expressed as follows: Among them, Feature() is a feature extraction operation; S23: Preprocessed livestock image data based on feeding time 2 The method in step S21 is used to extract the livestock image features at the second feeding time. Combined with livestock image features at feeding time 2 The method in step S22 is used to process the preprocessed livestock image data at the second feeding time. Perform hierarchical feature extraction to extract the second contour features of livestock and the second geometric characteristic G of livestock (2) , calculated as: S24: Design a gating mechanism to calculate the incremental features. The calculation method is: Gate=sigmoid(W1·(E (1) ,E (2) )+b) ΔF=Gate·(E (1) -AND (2) ) Among them, E (1) For the first livestock characteristic, E (2) is the second livestock feature, Gate is the incremental gate, sigmoid() is the sigmoid function, W1 is the incremental weight matrix, b is the incremental bias vector, ΔF is the incremental feature; S25: Calculate the livestock weight based on the incremental characteristics, the calculation method is: Among them, Weight is the weight of livestock.

5. The image-based intelligent material spreading method according to claim 4, characterized in that: The step S3 comprises: S31: Calculate the amount of feed to be spread based on the weight of the livestock. The calculation method is: A=SVR(Weight) Among them, A is the spreading amount, SVR() is the support vector regression model; S32: Correct the spreading amount according to the incremental feature to obtain the corrected spreading amount, which is calculated as follows: A′=A+SVR(ΔF) Among them, A ′ is the corrected spreading amount.

6. The image-based intelligent material spreading method according to claim 5, characterized in that: The step S4 comprises: Firstly, the reference speed of the spreading motor is calculated according to the corrected spreading amount and the moving speed of the spreading vehicle after preprocessing. Secondly, the original spreading features are extracted according to the preprocessed spreading image data, and the spreading deviation category distribution is calculated according to the original spreading features. Then, the spreading deviation features are calculated based on the spreading distance between the spreading vehicle and the trough and the spreading deviation category distribution. Finally, the target speed of the spreading motor is calculated based on the reference speed of the spreading motor and the spreading deviation features.

7. The image-based intelligent material spreading method according to claim 6, characterized in that: The step S4 comprises: S41: Calculate the reference speed of the material spreading motor according to the corrected material spreading amount and the moving speed of the material spreading vehicle after pre-processing. The calculation method is: Among them, N base is the base speed of the spreading motor, K is the proportional factor, V vehicle is the moving speed of the material spreading vehicle after pretreatment, and B is the reference offset; S42: extracting original spreading features according to the preprocessed spreading image data, and calculating the spreading deviation category distribution according to the original spreading features; and then calculating the spreading deviation features in combination with the spreading distance between the spreading vehicle and the trough and the spreading deviation category distribution; the calculation method is: F feed =(W′ down ·Concat(GAP(P feed ),CNN big (P feed ),CNN small (P feed )) p=softmax(MLP(D,F feed )) F′ feed =Concat(F feed ,p) Among them, F feed is the original spreading characteristic, W′ down is the weight matrix of the scattering and downsampling, P feed is the preprocessed scattering image data, p is the scattering deviation category distribution, softmax() is the softmax function, D is the preprocessed scattering distance, F ′ feed It is the material spreading deviation characteristic; S43: Calculate the target speed of the spreading motor by combining the reference speed of the spreading motor and the spreading deviation characteristics. The calculation method is: N final =N base +MLP(F′ feed ) Among them, N final It is the target speed of the spreading motor.

8. The image-based intelligent material spreading method according to claim 1, characterized in that: The step S5 comprises: S51: inputting the calculated target speed of the material spreading motor into the material spreading vehicle motor control system, and converting the target speed of the material spreading motor into a digital signal of the material spreading vehicle motor; S52: Using PID control algorithm, adjusting the control signal according to the deviation between the actual speed of the spreading motor and the target speed of the spreading motor; S53: Monitor the actual speed of the material spreading motor in real time and feed it back to the PID control algorithm to adjust the target speed of the material spreading motor in real time.

9. An image-based intelligent material spreading system, characterized in that: include: Data acquisition and preprocessing module: collects and preprocesses the spreading data and image data at different spreading times to obtain preprocessed spreading data, preprocessed livestock image data and preprocessed spreading image data; Livestock feature extraction module: based on the preprocessed livestock image data, extract livestock image features, calculate incremental features, and then calculate livestock weight based on the incremental features; Feeding amount calculation module: calculates the feeding amount according to the weight of livestock, and corrects the feeding amount based on the incremental characteristics to obtain the corrected feeding amount; Spreading motor target speed calculation module: calculates the spreading motor target speed according to the corrected spreading amount, preprocessed spreading data and preprocessed spreading image data; Material spreading control module: controls the material spreading vehicle to spread materials according to the calculated target speed of the material spreading motor; To realize an image-based intelligent material spreading method as described in any one of claims 1-8.

Images