Intelligent fusion control navigation system of vegetable collection and transportation all-in-one machine

Abstract

The invention relates to the technical field of agricultural equipment control, in particular to an intelligent fusion control navigation system of a vegetable collection and transportation all-in-one machine, which comprises an image acquisition and preprocessing module, a cabbage ridge line identification and positioning module, a multi-source information fusion decision module and a path tracking and cooperative control module. The method comprises the following steps: collecting ridge row images through a visual sensor, executing multi-scale fusion denoising, histogram equalization and perspective correction, and extracting a clear visual navigation datum line; gNSS and IMU data are fused with visual information, and extended Kalman filtering is adopted to realize high-precision pose estimation; switching different operation modes based on the operation state machine, and dynamically generating a corresponding expected path; path tracking control is completed in combination with a pure tracking algorithm and a speed mapping table, and intelligent cooperative control over the whole receiving and operating process is achieved. According to the method, the ridge row identification robustness, the path tracking precision and the system collaborative operation capability are improved, and the method is suitable for automatic harvesting operation of cabbages in a complex farmland environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of agricultural equipment control, and particularly relates to an intelligent fusion control navigation system of a vegetable harvesting and transporting integrated machine. BACKGROUND

[0002] In large-scale mechanized harvesting and transporting operations of cabbage and other vegetable crops, the intelligent degree of ridge and row recognition, path navigation and operation control directly determines the operation efficiency and operation quality. Traditional harvesting and transporting operations mostly rely on manual driving or single GNSS navigation control mode, and have problems such as low operation accuracy, heavy man-machine interaction burden and frequent path deviation. With the development of agricultural unmanned technology, the intelligent control system integrating visual navigation, satellite positioning and inertial measurement and other multi-source perception has gradually become a mainstream trend. Based on vehicle-mounted visual perception for ridge and row recognition, combined with GNSS and IMU information to realize high-precision path tracking control, which is becoming the core supporting technology for the automation transformation of cabbage and other vegetable harvesting and transporting operations.

[0003] The prior art still has many deficiencies in actual application in the field. On the one hand, the visual recognition algorithm is sensitive to light, mud and plant overlap, and the ridge and row boundary extraction is unstable, resulting in large path reference error; on the other hand, GNSS positioning has signal drift and shielding problems, and IMU is easily affected by noise accumulation, so a single information source cannot guarantee the continuity and accuracy of path tracking. In addition, the existing path control is mostly a single line control system, which lacks an intelligent mode switching mechanism linked with the operation state, and cannot realize high-level collaborative control in the harvesting, turning and unloading links, thereby limiting the overall operation efficiency and reliability of the system. SUMMARY

[0004] The present application provides an intelligent fusion control navigation system of a vegetable harvesting and transporting integrated machine, which integrates visual, satellite and inertial information, and has a cabbage harvesting and transporting integrated machine intelligent control navigation system with operation mode adaptive capability.

[0005] An intelligent fusion control navigation system of a vegetable harvesting and transporting integrated machine, comprising an image acquisition and preprocessing module, a cabbage ridge line recognition and positioning module, a multi-source information fusion decision module and a path tracking and collaborative control module, wherein; The image acquisition and preprocessing module is used for acquiring and processing original ridge and row images, and outputting preprocessed ridge and row images; The cabbage ridge line recognition and positioning module is used for receiving the preprocessed ridge and row images, recognizing and fitting the cabbage ridge line, and outputting the visual navigation reference line and its position information; The multi-source information fusion decision module is used for receiving the visual navigation reference line and its position information, fusing satellite positioning information and inertial measurement information, and according to the mode switching of the operation state machine, calculating and generating an expected travel path corresponding to the current operation mode; The path tracking and cooperative control module is configured to receive the expected travel path, calculate real-time steering control instructions and travel speed instructions of the walking chassis, and control action instructions of the harvesting mechanism, the conveying mechanism, or the lifting and unloading mechanism according to a current working mode.

[0006] Optionally, the image acquisition and preprocessing module comprises: An original ridge image containing the cabbage ridge is acquired by a visual sensor installed in front of the harvesting and conveying all-in-one machine; The original ridge image is subjected to multi-scale fusion denoising processing to eliminate noise interference in the image, thereby obtaining a denoised ridge image; The denoised ridge image is subjected to adaptive histogram equalization processing to enhance the contrast between the ridge and the soil background, thereby obtaining an enhanced ridge image; The enhanced ridge image is subjected to perspective transformation processing based on camera calibration parameters to correct image distortion caused by lens perspective, thereby obtaining a corrected ridge image; The corrected ridge image is output as a preprocessed ridge image to the cabbage ridge line recognition and positioning module.

[0007] Optionally, the multi-scale fusion denoising processing specifically adopts an algorithm combining wavelet transformation and bilateral filtering, comprising: wavelet multi-scale decomposition of the original ridge image to obtain high-frequency detail coefficients and low-frequency approximation coefficients; soft threshold denoising processing of the high-frequency detail coefficients, and bilateral filtering denoising processing of the low-frequency approximation coefficients; and inverse wavelet transformation reconstruction of the denoised high-frequency detail coefficients and low-frequency approximation coefficients to obtain the denoised ridge image.

[0008] Optionally, the cabbage ridge line recognition and positioning module comprises: The preprocessed ridge image from the image acquisition and preprocessing module is received, and an image segmentation is performed on the preprocessed ridge image by using a super green feature factor and an Otsu threshold segmentation algorithm to separate the cabbage ridge region from the soil background, thereby obtaining a segmented binary image; The segmented binary image is subjected to morphological closing operation processing to fill small cavities in the cabbage ridge region and smooth the ridge line edges, and at the same time, an area threshold method is used to filter out noise interference in the non-ridge region, thereby obtaining a morphologically optimized binary image; The morphologically optimized binary image is subjected to Canny edge detection to extract the complete contour of the cabbage ridge region, thereby obtaining an edge contour image of the cabbage ridge line; Based on the edge contour image of the cabbage ridge line, a least squares method is used to perform linear fitting on the left and right cabbage ridge line edge point sets, respectively, thereby obtaining two linear equations representing the positions of the cabbage ridge lines. According to the linear equations of the two cabbage ridge lines obtained by fitting, a middle line of the two straight lines is calculated as a final visual navigation reference line, and a lateral deviation and a heading deviation of the visual navigation reference line relative to a current position of the picking and transporting integrated machine are further calculated, which are collectively output as position information of the visual navigation reference line to the multi-source information fusion decision module.

[0009] Optionally, the straight line fitting by using the least square method specifically includes: respectively extracting edge point pixel coordinates of the left side ridge line and the right side ridge line in the edge contour image of the cabbage ridge line; for the left side edge point set and the right side edge point set, respectively applying the least square method principle to solve the straight line equation parameters that minimize the sum of squares of distances of all edge points to the fitting straight line, so as to obtain two straight line equations respectively representing positions of the left side cabbage ridge line and the right side cabbage ridge line.

[0010] Optionally, the multi-source information fusion decision module includes: receiving the visual navigation reference line and the position information thereof from the cabbage ridge line recognition and positioning module, and synchronously receiving satellite positioning information from a global satellite navigation system and inertial measurement information from an inertial measurement unit, and uniformly converting the received information to a picking and transporting integrated machine body coordinate system; taking the converted position information of the visual navigation reference line, the satellite positioning information and the inertial measurement information as observation inputs, and adopting an extended Kalman filtering algorithm to perform data fusion to estimate accurate pose state information of the picking and transporting integrated machine at the current time; driving the operation state machine to perform mode judgment and switching according to a preset operation logic and a current farmland environment feature, and a mode of the operation state machine at least includes a straight line harvesting mode, a head turning mode and a visual signal loss mode; based on the current operation mode determined by the operation state machine and the accurate pose state information of the picking and transporting integrated machine at the current time, calculating and generating an expected travel path corresponding to the current operation mode, and outputting the calculated and generated expected travel path to the path tracking and cooperative control module.

[0011] Optionally, the expected travel path includes: in the straight line harvesting mode, the expected travel path is a path parallel to the visual navigation reference line; in the head turning mode, the expected travel path is a circular arc path generated based on the satellite positioning information; and in the visual signal loss mode, the expected travel path is a straight line path calculated based on the inertial measurement information.

[0012] Optionally, the mode determination and switching logic of the job state machine comprises: switching to a straight-line harvesting mode when the position information of the visual navigation reference line is continuously valid and the heading deviation is less than a set threshold; switching to a headland turning mode when it is detected that the position information of the visual navigation reference line is invalid or the lateral deviation continuously increases beyond a set range; switching to a visual signal loss mode when the position information of the visual navigation reference line is suddenly lost in the straight-line harvesting mode or the headland turning mode.

[0013] Optionally, the path tracking and cooperative control module comprises: receiving the expected travel path from the multi-source information fusion decision module, and combining the real-time pose of the pickup and delivery integrated machine to calculate the lateral deviation and the heading deviation between the current position of the pickup and delivery integrated machine and the expected travel path; based on the calculated lateral deviation and heading deviation, using a pure tracking algorithm to search for a target preview point on the expected travel path with a look-ahead distance, and calculating a real-time steering control instruction required for the pickup and delivery integrated machine to track the expected travel path according to the target preview point; querying a preset speed mapping table according to the current job mode determined by the job state machine and the curvature of the expected travel path to determine a driving speed instruction suitable for the current job mode and the path curvature, wherein the straight-line harvesting mode adopts a high-speed gear and the headland turning mode adopts a low-speed gear; generating action instructions for the harvesting mechanism, the conveying mechanism or the lifting and unloading mechanism according to the current job mode determined by the job state machine, wherein the harvesting mechanism and the conveying mechanism are activated simultaneously in the straight-line harvesting mode, the harvesting mechanism and the conveying mechanism are stopped in the headland turning mode, and the lifting and unloading mechanism is activated and the walking chassis is paused in the lifting and unloading stage; outputting the calculated real-time steering control instruction, the driving speed instruction and the action instructions for the harvesting mechanism, the conveying mechanism or the lifting and unloading mechanism to the underlying actuators of the pickup and delivery integrated machine to complete closed-loop control.

[0014] The present application has the following advantages: The present application introduces multi-scale fusion denoising and adaptive histogram equalization technology through the image acquisition and preprocessing module, cooperates with ridge image segmentation based on super green features and Otsu algorithm, Canny edge extraction and least square method straight line fitting, and constructs a stable and reliable visual navigation reference line, which greatly improves the recognition robustness of the cabbage ridge under the conditions of light interference, background debris and irregular shape. At the same time, the visual navigation reference line is accurately mapped to the body coordinate system, the lateral deviation and the heading deviation are calculated, the path reference is ensured to have high reliability for vehicle control, and high-resolution and low-error input basis is provided for subsequent navigation control.

[0015] The present application utilizes a multi-source information fusion decision module to fuse visual navigation position information, GNSS plane positioning data and IMU inertial attitude information through an extended Kalman filtering algorithm, construct a multi-sensor cooperative positioning mechanism, and keep the system stable operation under partial failure or short-time interference of navigation signals. Meanwhile, the judgment logic of the operation state machine is combined to support dynamic switching between three modes of straight-line harvesting, turning at the head of the field and loss of visual signals, and automatically generate a standard circular arc turning path in the turning state at the head of the field, further improve the adaptability of the vehicle in the small field and between curved ridges, and realize the real sense of intelligent path planning in the field.

[0016] The present application constructs a complete path tracking and cooperative control module, realizes the look-ahead point guidance control by using a pure tracking algorithm, calculates the vehicle steering angle and speed command in real time, dynamically selects the matched running speed from the preset speed mapping table in combination with the path curvature and operation mode, and ensures the smooth and efficient operation of the vehicle. Meanwhile, the system can link control the start-stop logic of the harvesting mechanism, the conveying mechanism and the lifting and unloading mechanism, realize the automatic cooperative operation of the whole process of "harvesting-conveying-unloading", greatly reduce the frequency of manual intervention, and improve the consistency, safety and overall efficiency of the cabbage field operation. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only a part of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0018] Fig. 1 It is a system flowchart of the embodiment of the present application. Fig. 2 It is a multi-source information fusion decision module flowchart of the embodiment of the present application. DETAILED DESCRIPTION

[0019] The present application will be described in detail below in combination with the drawings and specific embodiments. It should be noted here that, in order to make the embodiments more detailed, the following embodiments are the best, preferred embodiments, and other alternative ways can also be used by those skilled in the art to implement some known technologies; and the drawings are only used to describe the embodiments more specifically, and are not intended to specifically limit the present application.

[0020] As shown in Figs. 1-2 An intelligent fusion control navigation system of a vegetable harvesting and conveying integrated machine, comprising an image acquisition and preprocessing module, a cabbage ridge line recognition and positioning module, a multi-source information fusion decision module and a path tracking and cooperative control module, wherein: The image acquisition and preprocessing module is used for acquiring and processing original ridge images, and outputting preprocessed ridge images, specifically: Original ridge image acquisition: The visual sensor of MER-132-43U3C high-resolution industrial camera is selected for continuous image acquisition of the front working area. The camera is installed at the front end of the cabbage harvesting and transporting machine, 1.5 meters above the ground, and the lens optical axis forms a 30° angle with the vertical line of the ground to ensure that the field of view of the acquired image can completely cover the ridge area within 3 to 5 meters in front. The image acquisition process is controlled by the FPGA-based image acquisition control unit, which triggers the camera to expose and image at a fixed interval of 50 milliseconds, continuously generating original ridge images containing cabbage ridges.

[0021] Multi-scale fusion denoising processing: To effectively suppress the natural light changes in the field, soil texture interference, and Gaussian noise and salt and pepper noise introduced by the sensor, multi-scale fusion denoising processing is performed on the original ridge image. This step uses a hybrid denoising algorithm based on wavelet transform and bilateral filtering, and the specific processing flow is as follows: Select Daubechies 4 wavelet basis, perform two-dimensional discrete wavelet transform on the original ridge image, and set the decomposition level to 2 layers to obtain the high-frequency detail coefficients (horizontal, vertical, and diagonal directions) of the first and second layers and the low-frequency approximation coefficients of the second layer; Apply the general soft threshold function to all high-frequency detail coefficients for denoising, and the soft threshold function is: ; Wherein is the wavelet coefficient, and the threshold According to the general threshold rule proposed by Donoho: ; Noise standard deviation Estimate by the median of the first layer high-frequency detail coefficients: ; Wherein and are the number of rows and columns of the image, respectively.

[0022] Perform bilateral filtering on the low-frequency approximation coefficients of the second layer, where the spatial domain standard deviation is set to 7 pixels, the color domain standard deviation is set to 20 gray levels, and the filter window size is 15x15 pixels.

[0023] The processed high-frequency and low-frequency coefficients are inversely wavelet transformed to reconstruct a denoised ridge image.

[0024] Adaptive histogram equalization processing: in order to enhance the local contrast between the cabbage ridge seedlings and the soil background, the denoised ridge image is subjected to adaptive histogram equalization processing. The limited contrast adaptive histogram equalization algorithm is adopted, and the specific parameters are as follows: The image is divided into a rectangular grid region of 32x32 pixels; Local histogram equalization processing is performed on each sub-region, and the histogram clipping limit threshold is set to 0.02 to prevent the local region from amplifying noise due to excessively high contrast; The boundaries between adjacent grids are fused by bilinear interpolation to smooth the brightness jumps between sub-regions; and an enhanced ridge image with enhanced contrast is output.

[0025] Perspective transformation processing: in order to obtain a standard view at a vertical perspective angle and eliminate geometric distortion introduced by the lens, the enhanced ridge image is subjected to perspective transformation processing, including the following steps: The geometric calibration of the vision sensor is completed in advance based on Zhang Zhengyou's calibration method to obtain its intrinsic matrix and distortion coefficients. The distortion coefficients include: Radial distortion coefficient: ; Tangential distortion coefficient: ; The above distortion coefficients are used to correct the distortion of the enhanced ridge image to eliminate edge stretching or compression caused by nonlinear distortion of the imaging system; Four feature points on the ground in the ridge region are selected as the pre-transformation point set to form a trapezoidal region, and the four corner points of the standard view rectangular region after transformation are defined as [0, 0], [640, 0], [640, 480], and [0, 480]; Based on the above correspondence, the homography matrix is calculated, and the image perspective remapping is performed using the matrix to finally obtain a corrected ridge image close to the orthographic view.

[0026] Preprocessed image output: the corrected ridge image obtained above is subjected to format standardization processing: First, the image is converted from the RGB color space to an 8-bit grayscale image to reduce the data dimension and facilitate subsequent recognition processing; Next, the image is subjected to size uniform scaling processing to unify the image size to 640x480 pixels; Finally, the processed image is output as a preprocessed ridge image to the cabbage ridge line recognition and positioning module to provide a clear and reliable input image source for subsequent ridge line recognition and position information extraction.

[0027] The cabbage ridge line recognition and positioning module receives the preprocessed ridge image, identifies and fits the cabbage ridge lines, and outputs the visual navigation baseline and its position information, specifically: Row area image segmentation: Receive preprocessed row images from the image acquisition and preprocessing module, and use the super green feature factor and Otsu threshold segmentation algorithm to separate the cabbage row area from the soil background area.

[0028] Specifically, the super-green feature value of each pixel in the image is first calculated pixel by pixel, using the following formula: ; in, , , The values ​​represent the red, green, and blue components of a pixel, respectively, thus constructing a super-green feature image.

[0029] Subsequently, the Otsu adaptive thresholding algorithm is applied to the ultra-green feature image. The core process is as follows: Calculate the image grayscale histogram; Traverse all possible grayscale thresholds Find the threshold that maximizes the inter-class variance between the foreground (cabbage rows) and background (soil) categories. ; The formula for calculating the between-class variance is: ; in, , The pixel ratio between the foreground and background. , The average gray value of the foreground and background. This represents the average grayscale value of the entire image.

[0030] Ultimately, the supergreen eigenvalue will be greater than Pixels with a value less than or equal to the threshold are set to 1 (white, representing cabbage rows), and pixels with a value less than or equal to the threshold are set to 0 (black, representing soil background), generating a segmented binary image.

[0031] Morphological closing operation and noise filtering: To enhance the connectivity and edge smoothness of the segmented ridge region, morphological processing and small-area noise removal are performed on the segmented binary image.

[0032] Morphological closing operations are performed on the image using 3×3 pixel rectangular structuring elements. This process includes dilation followed by erosion, which effectively fills small holes in the ridge area and connects discontinuous contour lines. Calculate the area of ​​all white connected regions in the image, and apply an area thresholding method to filter out connected regions with areas smaller than a set threshold. Preferably, with an image resolution of 640×480 pixels, the area threshold is set to 100 pixels. Output a morphologically optimized binary image after noise removal.

[0033] Ridge edge contour extraction: Canny edge detection is performed on the morphologically optimized binary image to accurately extract the boundary contours of the cabbage ridges. The specific steps are as follows: The image is pre-smoothed using a 5×5 Gaussian filter with a standard deviation of 1.4 to reduce high-frequency interference from edge detection. Using the Sobel operator to compute images in direction and gradient components of direction and And calculate the gradient magnitude and gradient direction for each pixel: ; Perform nonmaxima suppression, filter local maxima along the gradient direction, and refine edge lines; Edge concatenation is performed using a dual-threshold method, with high and low thresholds set as follows: and , the gradient value is greater than Pixels with gradient values ​​between the two and connected to the strong edges are marked as strong edges and retained; The final output is an image of the edge contour of the cabbage ridges, including the ridge edges.

[0034] Ridge line fitting and baseline calculation: Based on the edge contour image of cabbage ridges, the edge point sets on the left and right sides are extracted, and the least squares method is used to fit the lines respectively. The center line is then calculated as the visual navigation baseline.

[0035] 1. Using the vertical center line of the image as the dividing line, divide the edge pixels in the edge contour image into: Left edge point set: ; Right edge point set: ; 2. For the set of points on the left, fit a straight line using the least squares method: Its analytical solution is: ; 3. Similarly, fit a straight line to the set of points on the right edge: ; 4. Based on the line connecting the midpoints of the two straight lines at corresponding points at each horizontal level, calculate and construct the centerline of the row as the visual navigation baseline.

[0036] Visual navigation position information calculation: Based on the visual navigation baseline obtained above, calculate its positional relationship with the coordinate system of the collection and transportation unit, including lateral deviation and heading deviation.

[0037] 1. Set the origin of the coordinate system of the integrated collection and transportation machine. The vertical axis represents the center point of the lower boundary of the image. The direction is the direction in which the vehicle is moving; 2. Let the general form of the visual navigation baseline be: The coordinates of the origin are ; 3. Calculate the lateral deviation (i.e., the vertical distance from the origin to the navigation baseline), is calculated as follows: ; 4. Calculate the heading deviation (i.e., the navigation baseline and the vehicle's longitudinal axis) The included angle is calculated as follows: ; The calculated lateral deviation and heading deviation The position information, which serves as the baseline for visual navigation, is output to the multi-source information fusion decision module, serving as the basis for subsequent path calculation and control execution.

[0038] The multi-source information fusion decision module receives the visual navigation baseline and its position information, and fuses satellite positioning information and inertial measurement information. Based on the mode switching of the operational state machine, it calculates and generates the desired travel path corresponding to the current operational mode. Specifically: Unified coordinate transformation for heterogeneous data: Receives three types of heterogeneous input information, including: Position information of the visual navigation baseline, including lateral deviation. and heading deviation ; The satellite positioning information provided by the GNSS receiver is expressed in latitude and longitude form as follows: ; The inertial measurement information provided by the IMU includes triaxial acceleration, triaxial angular velocity, and the calculated heading angle. and yaw rate ; To achieve information fusion, all the above information will be uniformly converted to the coordinate system of the integrated collection and transportation machine. This coordinate system is defined as follows: the origin is the center of the vehicle's rear axle. The axis points to the left side of the vehicle. The axis points in the direction the vehicle is moving. The specific conversion method is as follows: Satellite positioning information conversion: First, UTM projection (Universal Transverse Mercator projection) is used to convert the satellite positioning information into satellite positioning information. Convert to planar coordinates Then, the two-dimensional rigid body transformation parameters are defined based on the starting point coordinates. ,Will Transformed into body coordinate system ; Visual navigation baseline position information conversion: based on camera extrinsic calibration parameters (installation height) and pitch angle ), in the image coordinate system and Approximate conversion to lateral position in body coordinate system and heading angle ; Under approximate conditions, we have: ; Inertial measurement information conversion: This is achieved through a rotation matrix constructed from the IMU mounting angles. and Expressions transformed from sensor coordinates to body coordinates.

[0039] The coordinate alignment process described above ensures the consistency of data in the spatial reference frame, laying the foundation for subsequent filtering, fusion, and path generation.

[0040] Extended Kalman Filter Data Fusion: To improve the accuracy of pose estimation for the integrated collection and transportation machine, an extended Kalman filter algorithm is used to fuse multi-source information and estimate the motion state at the current moment in real time. The processing procedure is as follows: Define the system state vector as follows: ; in, and This indicates the vehicle's position in the body coordinate system. Indicates the heading angle. Indicates longitudinal velocity. This represents the yaw rate.

[0041] Define the observation vector as: ; A simplified vehicle kinematics model is constructed as the state equation, and the state is updated during the prediction phase based on the following formula: ; ; ; ; ; Construct observation equations to establish the relationship between observations and states: ; ; ; ; ; In each filtering cycle In the middle, execute in sequence: Prediction step: Predict the current state using the state equation; Update step: using actual observation vectors Correct the state prediction results; Finally, the fused precise pose and status information of the collection and transportation machine is output: ; Job state machine mode determination and switching: based on the state vector obtained by filtering estimation The effectiveness of visual navigation information drives the operational state machine to perform pattern recognition and switching. The state machine includes three main modes: Straight-line harvesting mode: When the position information of the visual navigation baseline is continuously valid (e.g., output for 10 consecutive frames) and the visual heading deviation is... The system enters or remains in straight-line harvesting mode.

[0042] Ground Turning Mode: When visual information fails (e.g., no output for 5 consecutive frames) or when estimating lateral position. (Indicating that the vehicle has deviated from the center of the ridge), the system switches to the headway turning mode.

[0043] Visual signal loss mode: When visual navigation information is suddenly interrupted (3 consecutive frames lost) in straight-line harvesting or turning mode, but GNSS and IMU information are normal, the system switches to visual signal loss mode.

[0044] Through the pattern recognition logic described above, the system can achieve dynamic adaptation of the operation stage.

[0045] Desired path calculation: combining the current operation mode with the estimated state vector The calculation generates the expected path that matches the pattern, specifically including: Straight-line harvesting mode path generation: The path is a straight line coinciding with the visual navigation baseline; the point sequence is... ,in The intervals are equal vertical increments.

[0046] Path generation for turning at the edge of the field: The path is a segment Standard arc, center The location point recorded at the end of the previous operation, with a radius of... m represents the minimum turning radius of the machine. The circular path points are generated by parametric equations. ; Visual signal loss pattern path generation: The path is a straight line based on inertial guidance; the path direction is the heading angle. The point sequence is: ; ; The incremental step size has a safety margin of no more than 20 meters for the total path length.

[0047] Desired Path Output: The generated desired path is encapsulated as structured data and output to the path tracking and cooperative control module, specifically including: A series of path point coordinates ; Corresponding target heading angle ; Target speed The straight-line mode is set to a higher speed, and the turning mode is set to a lower speed to ensure safety. The above data is output in real time in the form of discrete sequences, which are then used by the subsequent control module to execute chassis path tracking and operation linkage control strategies.

[0048] The path tracking and coordinated control module receives the desired travel path, calculates and generates real-time steering control commands and travel speed commands for the chassis, and coordinates the action commands of the harvesting mechanism, conveying mechanism, or lifting and unloading mechanism according to the current operating mode. Specifically: Receive the desired travel path from the multi-source information fusion decision module, which is represented as a discrete sequence of path points: ; Simultaneously receive the precise pose status information of the collection and transportation machine at the current moment: ; Based on the above information, perform path deviation calculation, including: 1. Search the waypoint sequence for the current vehicle position. Euclidean distance to the nearest path ; 2. Calculate the current vehicle position To path segment The normal distance, as the lateral deviation The calculation formula is as follows: ; 3. Calculate the vehicle's current heading angle with path points Tangent direction The difference is used as the heading deviation. ,in; ; Real-time steering control command generation: Based on the path deviation, a pure tracking algorithm is used to generate real-time steering control commands. The specific steps are as follows: 1. The current driving speed command generated subsequently. Calculate the look-ahead distance The calculation formula is as follows: ; in The forward proportionality coefficient is preferred, with a value of 1.0. The preferred value for the basic look-ahead distance is 1.0m; 2. From the shortest path point Start by searching along the path to the current position. Approximately Target aiming point ; 3. Calculate the directional angle between the current vehicle's rear axle center and the target aiming point. : ; 4. Based on the vehicle's wheelbase (Preferred value is 2.0m) and the pure tracking algorithm formula are used to calculate the front wheel steering angle. : ; 5. Calculate the required steering angle As the real-time steering control command for the current cycle, it is output to the steering servo controller.

[0049] Driving speed command calculation: Generate adaptive driving speed commands based on the current operation mode and path curvature. Specifically, it includes: 1. Based on the current tracking point Centered on, using waypoints Perform differential estimation to calculate path curvature : First derivative: ; Second derivative: ; Curvature calculation formula: ; 2. Query the preset speed mapping table, based on the current operating mode and curvature. Choose an appropriate speed.

[0050] 3. Output the matched values The driving speed command for the current control cycle is used for steering control and chassis drive control.

[0051] Linked generation of action commands for operating mechanisms: Based on the current mode of the operating state machine, linked generation of action commands for the harvesting mechanism, conveying mechanism, and lifting and unloading mechanism is performed. The specific control logic is as follows: Straight-line harvesting mode: Send a "start" command to the hydraulic valve group of the harvesting mechanism and set the speed to 300 RPM; Send a "start" command to the frequency converter of the conveyor mechanism to drive the conveyor belt to run.

[0052] Turning mode at the edge of the field: Send a "stop" instruction to the harvesting facility; Send a "stop after 5 seconds" command to the conveying mechanism to ensure that the material is emptied.

[0053] Lifting and unloading phase (triggered by external signal): Send a "lift" command to the hydraulic valve group of the lifting and unloading mechanism; Send a "speed is 0" command to the walking motor controller to pause vehicle movement until a descent reset signal is received.

[0054] This coordinated control strategy ensures seamless switching between operational phases and coordinated operation of all subsystems.

[0055] Control command output and closed-loop execution: The control commands generated above are sent to each actuator with a control cycle of 50 milliseconds, forming a closed-loop control system. Real-time steering control commands: sent to the steering servo controller via the CAN bus; Travel speed command: sent to the travel motor controller; Operating mechanism action commands: sent to the corresponding hydraulic valve group or frequency converter (harvesting mechanism, conveying mechanism, lifting and unloading mechanism); The system continuously monitors the feedback status of each actuator to determine whether the command was executed successfully, and performs necessary fault tolerance or correction processing to ensure that path tracking accuracy, vehicle driving safety and operation efficiency are fully under control.

[0056] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0057] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An intelligent integrated control and navigation system for a vegetable harvesting and transportation machine, characterized in that, It includes an image acquisition and preprocessing module, a cabbage ridge line recognition and positioning module, a multi-source information fusion decision-making module, and a path tracking and collaborative control module, among which; The image acquisition and preprocessing module is used to acquire and process the original row image and output the preprocessed row image. The cabbage ridge line recognition and positioning module is used to receive the pre-processed ridge image, identify and fit the cabbage ridge line, and output the visual navigation baseline and its position information. The multi-source information fusion decision module is used to receive the visual navigation baseline and its position information, and fuse satellite positioning information and inertial measurement information. According to the mode switching of the operation state machine, it calculates and generates the expected travel path corresponding to the current operation mode. The path tracking and collaborative control module is used to receive the desired travel path, calculate and generate real-time steering control commands and travel speed commands for the chassis, and coordinate the action commands of the harvesting mechanism, conveying mechanism or lifting and unloading mechanism according to the current operation mode.

2. The intelligent integrated control and navigation system for a vegetable harvesting and transportation machine according to claim 1, characterized in that, The image acquisition and preprocessing module includes: The original row images containing cabbage rows are acquired by a vision sensor installed in front of the integrated collection and transportation machine; The original ridge image is subjected to multi-scale fusion denoising processing to eliminate noise interference in the image, resulting in a denoised ridge image. Adaptive histogram equalization is performed on the denoised ridge image to enhance the contrast between the ridges and the soil background, resulting in an enhanced ridge image. The enhanced ridge image is subjected to perspective transformation processing based on camera calibration parameters to correct the image distortion caused by lens perspective, and a corrected ridge image is obtained. The corrected ridge image is output as a preprocessed ridge image to the cabbage ridge line recognition and positioning module.

3. The intelligent integrated control and navigation system for a vegetable harvesting and transportation machine according to claim 2, characterized in that, The multi-scale fusion denoising process specifically employs an algorithm combining wavelet transform and bilateral filtering, including: performing wavelet multi-scale decomposition on the original row image to obtain high-frequency detail coefficients and low-frequency approximation coefficients; performing soft thresholding denoising on the high-frequency detail coefficients, while simultaneously performing bilateral filtering denoising on the low-frequency approximation coefficients; and reconstructing the denoised high-frequency detail coefficients and low-frequency approximation coefficients using inverse wavelet transform to obtain the denoised row image.

4. The intelligent integrated control and navigation system for a vegetable harvesting and transportation machine according to claim 3, characterized in that, The cabbage ridge line identification and positioning module includes: The image acquisition and preprocessing module receives the preprocessed row image and uses the super green feature factor and Otsu threshold segmentation algorithm to segment the preprocessed row image, separating the cabbage row area from the soil background to obtain the segmented binary image. The segmented binary image is processed by morphological closing operation to fill the small holes in the cabbage row area and smooth the row edge. At the same time, the area threshold method is used to filter out noise interference in the non-row area to obtain the morphologically optimized binary image. Canny edge detection is performed on the morphologically optimized binary image to extract the complete contour of the cabbage row region, resulting in the edge contour image of the cabbage row. Based on the edge contour image of the cabbage ridge line, the least squares method is used to fit the line to the edge point set of the cabbage ridge line on the left and right sides respectively, and two line equations representing the position of the cabbage ridge line are obtained. Based on the straight line equations of the two cabbage rows obtained by fitting, the midline of the two straight lines is calculated as the final visual navigation baseline. The lateral deviation and heading deviation of the visual navigation baseline relative to the current position of the collection and transportation machine are further calculated and output as the position information of the visual navigation baseline to the multi-source information fusion decision module.

5. The intelligent integrated control and navigation system for a vegetable harvesting and transportation machine according to claim 4, characterized in that, The method of using least squares to fit a straight line specifically includes: extracting the pixel coordinates of edge points belonging to the left and right ridge lines in the edge contour image of the cabbage ridges; applying the principle of least squares to the left and right edge point sets respectively to solve for the line equation parameters that minimize the sum of the squares of the distances from all edge points to the fitted line, thereby obtaining two line equations that respectively represent the positions of the left and right cabbage ridge lines.

6. The intelligent integrated control and navigation system for a vegetable harvesting and transportation machine according to claim 5, characterized in that, The multi-source information fusion decision module includes: The system receives the visual navigation baseline and its position information from the cabbage ridge identification and positioning module, and simultaneously receives satellite positioning information from the global satellite navigation system and inertial measurement information from the inertial measurement unit, and converts the received information into the coordinate system of the integrated collection and transportation machine. The position information of the converted visual navigation baseline, satellite positioning information and inertial measurement information are used as observation inputs. The extended Kalman filter algorithm is used to fuse the data and estimate the precise pose state information of the collection and transportation unit at the current moment. Based on the preset operation logic and the current farmland environment characteristics, the operation state machine is driven to perform mode judgment and switching. The operation state machine's modes include at least straight-line harvesting mode, field-end turning mode, and visual signal loss mode. Based on the current operating mode determined by the operating state machine and the precise pose information of the collection and transportation machine at the current moment, the desired travel path corresponding to the current operating mode is calculated and generated, and the calculated desired travel path is output to the path tracking and collaborative control module.

7. The intelligent integrated control and navigation system for a vegetable harvesting and transportation machine according to claim 6, characterized in that, The desired travel path includes: in straight-line harvesting mode, the desired travel path is a path parallel to the visual navigation baseline; in ground-turning mode, the desired travel path is an arc path generated based on satellite positioning information; and in visual signal loss mode, the desired travel path is a straight-line path calculated based on inertial measurement information.

8. The intelligent integrated control and navigation system for a vegetable harvesting and transportation machine according to claim 7, characterized in that, The mode judgment and switching logic of the operation state machine specifically includes: when the position information of the visual navigation baseline is continuously valid and the heading deviation is less than a set threshold, switching to the straight-line harvesting mode; when the position information of the visual navigation baseline is detected to be invalid or the lateral deviation continues to increase beyond the set range, switching to the ground turning mode; when the position information of the visual navigation baseline is suddenly lost in the straight-line harvesting mode or the ground turning mode, switching to the visual signal loss mode.

9. The intelligent integrated control and navigation system for a vegetable harvesting and transportation machine according to claim 8, characterized in that, The path tracking and cooperative control module includes: The desired travel path is received from the multi-source information fusion decision module, and combined with the real-time pose of the collection and transportation machine, the lateral deviation and heading deviation between the current position of the collection and transportation machine and the desired travel path are calculated. Based on the calculated lateral and heading deviations, a pure tracking algorithm is used to search for target preview points on the desired travel path using look-ahead distance, and the real-time steering control command required to make the collection and transportation unit track the desired travel path is calculated based on the target preview points. Based on the current operation mode determined by the operation state machine and the curvature of the desired travel path, a preset speed mapping table is queried to determine the travel speed command that is adapted to the current operation mode and the curvature of the path. The straight harvesting mode uses a high speed gear, and the turning mode at the edge of the field uses a low speed gear. Based on the current operating mode determined by the operating state machine, action commands for the harvesting mechanism, conveying mechanism, or lifting and unloading mechanism are generated in a coordinated manner. In the straight harvesting mode, the harvesting mechanism and conveying mechanism are activated simultaneously. In the field turning mode, the harvesting mechanism and conveying mechanism are stopped. In the lifting and unloading stage, the lifting and unloading mechanism is activated and the traveling chassis is paused. The calculated real-time steering control commands, driving speed commands, and action commands for the harvesting mechanism, conveying mechanism, or lifting and unloading mechanism are output to the bottom-level actuator of the integrated harvester to complete closed-loop control.

Images