Cotton plant topping system and method based on image perception and intelligent control

Abstract

The invention discloses a cotton plant topping system and a cotton plant topping method based on image perception and intelligent control, which integrate machine vision, posture perception and intelligent control technologies and realize automatic identification, spatial positioning and accurate cutting of cotton plant terminal buds. The system collects image and attitude information through an industrial camera, an inertial measurement unit (IMU) and a vehicle speed sensor, image processing and decision control are executed through an industrial personal computer, and topping path planning and intelligent execution are achieved. Meanwhile, the system can carry out detection and risk prediction on cotton plant leaf scab and insect pest areas, and assists in realizing cotton field health state monitoring. The system improves the automation and intelligence level of cotton field operation, and is suitable for the fields of cotton precision management and intelligent agricultural machinery equipment.

Description

Technical Field

[0001] This invention relates to the field of agricultural machine vision technology, and in particular to a cotton plant topping system and method based on image perception and intelligent control. Background Technology

[0002] Cotton, as one of my country's important economic crops, directly affects the raw material supply for the textile industry and the economic benefits for farmers. During the cotton plant's growth, topping is necessary at appropriate times to promote nutrient allocation to fruiting branches and increase boll setting rate. Topping effectively suppresses apical dominance and promotes balanced branch and leaf growth, making it a crucial step in high-yield and stable cotton management. However, traditional manual topping methods suffer from high labor intensity, low efficiency, poor precision, and strong reliance on manual labor, making it difficult to meet the demands of modern agricultural mechanization and intelligentization.

[0003] Existing automatic topping equipment mostly employs purely mechanical structures or relies on single sensing methods such as ultrasound and infrared to detect and control the removal of apical buds. While these devices improve operational efficiency to some extent, their positioning accuracy and environmental adaptability are limited. Especially under complex field lighting conditions, varying cotton plant postures, and differences between rows, they are prone to false detections, missed detections, and topping deviations. Furthermore, the lack of efficient image perception and intelligent control strategies makes it difficult for the system to achieve accurate identification and dynamic decision-making control of the target apical bud. For example, Chinese Patent Publication No. CN118435797A discloses a "fully automatic topping mechanism for cotton," whose system mainly includes a moving frame, a rotating frame topping mechanism, a positioning mechanism, and an apical bud detection mechanism. This solution achieves multi-point topping operations through a rotating disc, radius adjustment components, and lifting topping components. By utilizing mechanical structures to adjust the spacing and height differences between different cotton plants, it can achieve a certain degree of automated topping. However, the mechanism mainly relies on mechanical sensing and cylinder control to achieve fixed-height topping, lacking real-time visual recognition and intelligent decision-making capabilities. Furthermore, its overall structure is complex, with high coupling between modules, and it lacks the ability to adapt to different lighting conditions and changes in cotton plant posture.

[0004] Therefore, there is an urgent need for a cotton plant topping system and method based on image perception and intelligent control to provide an efficient and reliable intelligent solution for cotton field management. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to address the shortcomings of existing technologies by providing a cotton plant topping system and detection method based on image perception and intelligent control. This system integrates an industrial camera, an inertial measurement unit (IMU), and a vehicle speed sensor to achieve automatic identification and positioning of the cotton plant's terminal bud. By using an industrial control computer to perform image processing and path planning, it drives a lifting and shearing mechanism to complete precise topping, thereby achieving automatic detection, intelligent decision-making, and precise execution of cotton plant topping.

[0006] Technical Solution: The first aspect of this invention provides a cotton plant topping system based on image perception and intelligent control, including a body, an image and attitude perception module, a topping execution module, and an image processing and intelligent control module. The body includes a mobile platform, a metal frame, and wheels for carrying the various functional modules and moving between rows in the cotton field. The image and attitude perception module includes an industrial RGB camera, an inertial measurement unit (IMU), and a vehicle speed sensor mounted on the upper front of the body shell for synchronously acquiring images of the cotton plant canopy and vehicle motion attitude information. The topping execution module includes a lifting module, a guide slide, a shearing drive assembly, and a shearing end effector. The lifting module and guide slide are used to drive and constrain the shearing end effector to move vertically, and the shearing end effector is used to shear the top bud of the cotton plant. The image processing and intelligent control module includes an industrial computer, which is electrically connected to the industrial camera, the IMU, the vehicle speed sensor, and the topping execution module.

[0007] Preferably, the walking wheels are made of wear-resistant rubber material, and the surface of the walking wheels is provided with anti-slip patterns to improve the traction and stability of the device in soft soil environments in the field.

[0008] Preferably, the shearing end effector includes a rotating blade assembly, and the rotating blade assembly is provided with an annular limiting cover on the outside to prevent debris or foreign objects from being rolled into the blade edge area, thereby improving the safety and stability of shearing. The rotating blade assembly is made of corrosion-resistant stainless steel.

[0009] Preferably, the blades in the shearing end effector are arranged in a spiral pattern to create a smooth cutting path and reduce resistance during the shearing process.

[0010] Preferably, the industrial cameras are uniformly and symmetrically installed at the center of the front end of the machine body; each camera mounting bracket is equipped with a height adjustment mechanism, which can be finely adjusted up and down according to the height of the cotton plants and the requirements of the viewing angle, so as to ensure the integrity of the image acquisition area and the clarity of the imaging.

[0011] A second aspect of this invention provides a method for topping cotton plants based on image perception and intelligent control, comprising the following steps:

[0012] S1: Cotton canopy images and vehicle speed and attitude information are acquired synchronously through an industrial RGB camera, vehicle speed sensor and inertial measurement unit (IMU). Camera calibration and distortion correction are performed on the acquired images. Image stabilization, deblurring and illumination normalization are performed in combination with vehicle speed and attitude information to generate an enhanced image dataset containing the region of interest at the top of the cotton plant.

[0013] S2: In the enhanced image dataset, row band detection and canopy segmentation are performed on the cotton plant canopy to define the working row region and obtain the canopy mask; a top interest region is constructed in the upper region of the canopy, the top interest region is input into the key point recognition network, the pixel coordinates and confidence of the top bud key point are output, and the top bud key point is mapped from the image plane to the body coordinate system by combining the camera intrinsic parameters and the camera-body extrinsic parameters to obtain the spatial position of the top bud and the corresponding confidence;

[0014] S3: Within the effective search domain defined by the enhanced image and the canopy mask, calculate the color enhancement indicator, perform region division and feature extraction on the search domain, obtain candidate pest patches based on the anomaly function, input the sub-image corresponding to the candidate patch into the pest detection network to output the pest category, bounding box and confidence score, and calculate the pest risk score by combining the anomaly and confidence scores. When the risk score exceeds the preset threshold, complete the conversion from pixel coordinates to ground coordinates and generate pest risk reporting data.

[0015] S4: Based on the spatial location of the apical bud, confidence level, vehicle speed, attitude, and safety sensor status, the operation permission is determined. Under the permission conditions, the shearing trajectory planning and collision check are performed, the shearing end effector is driven to complete the shearing of the cotton plant apical bud, and after the operation is completed, the image processing process is called to re-inspect the shearing results, generate an operation record package containing timestamps, pose and trajectory data, execution parameters, result labels, and before and after comparison images, and when the conditions are met, it is associated with the pest risk reporting data and sent to the central end.

[0016] Furthermore, step S1 further includes:

[0017] S11: Under unified trigger control, the industrial RGB camera, vehicle speed sensor and inertial measurement unit (IMU) synchronously acquire data, and bind the distortion-corrected image with the corresponding speed and attitude information to form the original multi-source synchronous dataset.

[0018] S12: Calculate the intra-frame pose increment during the camera exposure time based on vehicle speed and attitude information, and construct the image plane homography matrix using a local plane approximation model. Perform inverse mapping and resampling on the original image to achieve image stabilization and jitter correction.

[0019] S13: Estimate the direction and length of motion blur based on velocity and attitude information, construct a motion blur kernel to deblur the stabilized image, and implement illumination normalization and vegetation color index enhancement to improve the separability of cotton plants from the background.

[0020] Preferably, the original multi-source synchronization dataset in S11 can be represented as:

[0021]

[0022] In the formula, This represents the i-th corrected canopy image, acquired by an industrial RGB camera after calibration and distortion correction. The timestamp for data collection is generated by a unified trigger control module or the system clock. The vehicle speed is measured by the vehicle speed sensor. Attitude information, measured by the inertial measurement unit, includes pitch angle, roll angle, and yaw angle.

[0023] Preferably, in step S12, the planar homography matrix The calculation formula is:

[0024]

[0025] In the formula, For the camera intrinsic parameter matrix, For the fixed extrinsic parameter rotation from the IMU to the camera coordinate system, This represents the relative rotation increment within the exposure time window. Camera extrinsic matrix transpose, This refers to the relative translation increment within the exposure time window; This is the transpose of the local plane unit normal vector in the camera coordinate system; Let be the directed distance to the local plane; It is the inverse of the camera intrinsic parameter matrix.

[0026] Preferably, step S2 includes the following steps:

[0027] S21: Input the enhanced image into the deep learning segmentation network to perform pixel-level semantic segmentation of the cotton plant apical domain and obtain the apical domain semantic mask.

[0028] S22: Perform connected component analysis and morphological filtering on the semantic mask to remove isolated noise points and non-target regions, obtain a cleaned mask, and construct a top candidate band based on the upper boundary of the cleaned mask, calculate its minimum bounding rectangle, generate rectangular regions of interest according to a preset expansion coefficient and perform boundary clipping, thereby forming the top ROI dataset.

[0029] S23: Input the top ROI dataset into the keypoint recognition network to obtain a candidate set of apical bud keypoints, and filter and select the uniqueness of the candidate set according to the confidence threshold and the maximum confidence criterion, and output a single apical bud keypoint and its confidence.

[0030] S24: Based on the obtained apical bud pixel coordinates, the key points are converted from pixel coordinates to three-dimensional coordinates in the camera coordinate system using the camera intrinsic parameter matrix and the apical depth or range parameter. Furthermore, the coordinate transformation is performed using the extrinsic parameters from the camera to the body to obtain the spatial position of the apical bud in the body coordinate system.

[0031] Preferably, step S3 includes the following steps:

[0032] S31: Perform superpixel segmentation within the effective search domain to obtain multiple sub-regions, calculate color, texture and structural features for each sub-region to form a feature vector, and establish an anomaly function based on the background feature distribution of healthy leaves. Sort and filter each sub-region to obtain a set of candidate pest patches.

[0033] S32: Normalize the cropped sub-images corresponding to the candidate pest patches to a fixed scale and input them into the pest detection network to output the pest category, patch bounding box and detection confidence. Perform non-maximum suppression and consistency constraints on multiple candidate results to obtain redundant detection results.

[0034] S33: Extract anomaly features within the bounding box of the detected target, and weight and fuse the anomaly with the detection confidence to obtain a pest risk score. When the pest risk score exceeds a preset threshold, perform pixel-to-ground plane coordinate mapping based on the camera intrinsic and extrinsic parameters under the near-ground plane assumption, and generate a risk reporting data packet containing timestamp, pest category, risk score, bounding box, ground coordinates and evidence sub-image.

[0035] Preferably, step S33 specifically involves using superpixel segmentation and Mahalanobis distance anomaly function to screen candidate pest patches within the effective search domain defined by the enhanced image and canopy mask. After identification by the pest detection network, a risk score is calculated by weighted fusion of anomaly and detection confidence. And when the threshold is exceeded, a risk data packet is generated and reported. The calculation formula is:

[0036] Triggering conditions:

[0037] In the formula, For the prior anomaly degree, This represents the prior anomaly degree.

[0038] Preferably, step S4 specifically includes the following steps:

[0039] S41: Based on the spatial location and confidence level of the apical bud, and combined with vehicle speed, attitude stability and safety sensor status, a work window is generated. When the apical bud confidence level is higher than the first threshold and the safety conditions are met, the apical bud execution process is allowed to proceed; otherwise, execution is prohibited and the reason code is recorded.

[0040] S42: Complete single shearing trajectory planning and soft limit / collision check within the operation window, drive the shearing end effector to enter close alignment and perform shearing operation, and automatically retreat to a safe position and retry according to the limit if abnormalities such as jamming or timeout occur. If the number of retry exceeds the limit, terminate the operation and mark the reason for failure.

[0041] S43: After the shearing operation is completed, use an industrial camera to collect images after shearing to re-inspect the topping effect and form a work record package containing timestamps, pose and trajectory summaries, execution current or torque curves, result labels, ground coordinates and before-and-after comparison images. When the pest risk score reaches the second threshold, associate the work record package with the corresponding pest risk data and report it to the central end. When communication bandwidth is limited, prioritize uploading high-risk summaries and cache detailed evidence data locally.

[0042] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0043] (1) This invention integrates an industrial camera, a fill light, an inertial measurement unit (IMU), and a vehicle speed sensor through an image and attitude perception module to achieve synchronous acquisition of image and motion state information; combined with illumination normalization and attitude compensation algorithms, it improves image quality and spatial positioning accuracy, providing highly reliable input for subsequent recognition and control.

[0044] (2) The present invention integrates key point recognition network and spatial mapping algorithm in image processing and intelligent control module, which can realize accurate detection and three-dimensional positioning of cotton plant apical buds; at the same time, combined with real-time path planning and control strategy, it significantly improves the accuracy and stability of topping operation.

[0045] (3) This invention introduces a pest detection and risk assessment mechanism in the canopy area, realizes pest level prediction through anomaly calculation, takes into account topping control and cotton plant health monitoring, and improves the overall intelligence level of the system.

[0046] (4) The present invention adopts a modular structure design, the lifting module and the shearing mechanism are independently adjustable, the blade is a spiral corrosion-resistant structure, and the ring limit cover effectively prevents the entrapment of debris, ensuring the safety of shearing and the continuity of operation.

[0047] (5) This invention realizes a closed-loop control process of image perception, attitude measurement, intelligent decision-making and precise execution, which can automatically complete the topping operation in complex field environment. It has the advantages of high precision, self-adaptation and scalability, and provides a new idea for the development of intelligent agricultural machinery equipment. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the overall structure of the present invention;

[0049] Figure 2This is a schematic diagram of the top-down execution module structure in this invention;

[0050] Figure 3 This is a flowchart of the algorithm of the present invention. Detailed Implementation

[0051] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0052] Example 1

[0053] like Figure 1-2 As shown, a cotton plant topping system based on image perception and intelligent control according to the present invention includes a body 1, an image and attitude perception module 2, a topping execution module 3, and an image processing and intelligent control module 4. The body 1 includes a mobile platform, a metal frame 11, and wheels 12, used to carry the various functional modules and move between rows in the cotton field. The image and attitude perception module 2 includes an industrial RGB camera 21, an inertial measurement unit (IMU) 22, and a vehicle speed sensor 23 installed on the front upper part of the body shell, used to synchronously acquire images of the cotton plant canopy and vehicle movement attitude information. The topping execution module 3 includes a lifting module 31, a guide slide bar 32, a shearing drive assembly, and a shearing end effector 33. The lifting module 31 and the guide slide bar 32 are used to drive and constrain the shearing end effector to move in the vertical direction, and the shearing end effector 33 is used to shear the top bud of the cotton plant. The image processing and intelligent control module 4 includes an industrial computer, which is electrically connected to the industrial camera 21, the inertial measurement unit (IMU) 22, the vehicle speed sensor 23, and the topping execution module 3.

[0054] Industrial cameras 21 are uniformly and symmetrically installed at the center of the front end of the machine body 1; each industrial RGB camera 21 mounting bracket is equipped with a height adjustment mechanism, which can be finely adjusted up and down according to the height of the cotton plants and the requirements of the viewing angle to ensure the integrity of the image acquisition area and the clarity of the imaging.

[0055] The walking wheel 12 is made of wear-resistant rubber material, and the surface of the walking wheel is provided with anti-slip pattern to improve the traction and stability of the device in soft soil environment in the field.

[0056] The shearing end effector 33 includes a rotating blade assembly. The blades in the shearing end effector 33 are arranged in a helical pattern to create a smooth cutting path and reduce resistance during the shearing process. An annular retaining cover is provided on the outside of the rotating blade assembly to prevent debris or foreign objects from being caught in the blade edge area, thereby improving the safety and stability of the shearing process. The rotating blade assembly is made of corrosion-resistant stainless steel.

[0057] Example 2

[0058] The method in this embodiment can be implemented using the system described in Embodiment 1, such as... Figure 3As shown in the figure, this embodiment provides a method for detecting the main body of cotton plant topping based on image perception and intelligent control, as follows:

[0059] After the equipment is started, the industrial RGB camera is connected to the vehicle speed sensor and inertial measurement unit. Through unified trigger control, it synchronously acquires image and motion state information, thereby obtaining a one-to-one corresponding canopy image sequence and velocity and attitude data. The acquired images are further calibrated and distortion corrected to obtain geometrically distorted images, which are then bound to the corresponding velocity and attitude information. This ultimately forms a raw multi-source synchronous dataset that can be directly used for subsequent processing, represented as:

[0060]

[0061] In the formula, This represents the i-th corrected canopy image, acquired by an industrial RGB camera after calibration and distortion correction. The timestamp for data collection is generated by a unified trigger control module or the system clock. The vehicle speed is measured by the vehicle speed sensor. Attitude information, measured by the inertial measurement unit, includes pitch angle, roll angle, and yaw angle.

[0062] Then in the original synchronized dataset Based on this, the vehicle speed of the corresponding image frame is called. With attitude information Integrating the value over the camera exposure time yields the intra-frame pose increment. Combined with camera intrinsic parameter matrix Average distance from the canopy The pose increment is mapped to the image plane homography matrix using a local plane approximation model. Under the influence of this homography matrix, the original image Perform inverse mapping and resampling to obtain a stable image. This process achieves motion compensation and jitter correction; further, it utilizes velocity and attitude to estimate the direction and length of motion blur, constructs a blur kernel function, performs deblurring, and outputs a geometrically stable image. Subsequently Illumination normalization was implemented to reduce the differences between dawn and dusk and shadows, and vegetation color index was calculated to enhance the contrast between plants and the background, ultimately resulting in an enhanced image with uniform illumination, enhanced color, and geometric stability. Based on this, an augmented dataset is formed. Among them, the image plane homography matrix of the corresponding frame. The calculation formula is as follows:

[0063]

[0064] In the formula, Rotation with fixed extrinsic parameters from the IMU to the camera coordinate system; The relative rotation increment within the exposure time window, originally derived from the IMU angular velocity. The result is obtained by integration; Camera extrinsic matrix transpose; This refers to the relative translation increment within the exposure time window; This is the transpose of the local plane unit normal vector in the camera coordinate system; The directed distance (scalar) to the local plane; It is the inverse of the camera intrinsic parameter matrix.

[0065] Next, in image enhancement First, the work row area is determined using the row strip detection method. By limiting the processing range and excluding inter-row background interference, canopy segmentation is performed within this area based on vegetation color index and adaptive threshold to obtain a canopy mask. This allows for effective differentiation between the canopy, soil, and weeds. Based on this, the upper and lower boundaries of the canopy are extracted, and the canopy height is calculated. and according to the preset cutting coefficient The upper canopy region is longitudinally segmented, while also incorporating buffer parameters. Preserve information about the periphery of the apex to obtain the apical region of interest containing the apical bud. This results in the top region dataset:

[0066]

[0067] In the formula, For image Pixel field; This is the planting row area; Indicates the first Canopy mask region in the image The minimum longitudinal coordinate value corresponds to the upper boundary position of the plant canopy; This is the top clipping factor; This is the longitudinal buffer amount.

[0068] In the target localization phase, based on the enhanced image dataset, the i-th image after image stabilization and illumination normalization is denoted as... ,in Representing the pixel domain. The image is input into the SegFormer deep learning network to perform pixel-level segmentation of the cotton plant apical band domain. Its output semantic mask is denoted as... : Regions with a value of 1 correspond to the apical zone of the cotton plant, while regions with a value of 0 correspond to the background. Connectivity analysis and morphological filtering are performed to remove isolated noise points and non-target small regions, resulting in a cleaned mask. Based on this, according to Construct a candidate top band region along the upper boundary and compute its minimum bounding rectangle. Let the center and width / height of the rectangle be respectively... and To ensure complete preservation of information around the apical bud, the information is expanded according to a preset expansion coefficient. Expand outwards in both the horizontal and vertical directions to generate a rectangular region of interest. and to Boundary cropping is performed to meet image size constraints. Through the above processing, a sequence of top regions of interest (ROIs) for subsequent keypoint recognition is obtained, forming the top ROI dataset. .in Defined as:

[0069] Next, in the top ROI dataset Based on this, the i-th region of interest Input the keypoint recognition network to obtain the pixel coordinates and confidence scores of the apical keypoints. The keypoint recognition network preferably uses YOLO-Pose to obtain stable keypoint localization results while maintaining real-time performance. The candidate results are filtered by thresholding and a uniqueness selection is performed based on the maximum confidence criterion to ensure the stability and usability of the output results; the candidate set is denoted as... With confidence threshold Determining a unique solution using the maximum confidence criterion Thus determined This ensures that even in cases with multiple candidates or low confidence, a single keypoint result that meets the lower confidence limit is output, guaranteeing the stability of subsequent coordinate mapping and execution control. The unique solution is... The announcement is as follows:

[0070] Finally, the obtained pixel domain keypoint coordinates and their confidence levels were analyzed. Based on this, the key points are mapped step by step from the image plane to the body coordinate system to obtain spatial position parameters that can be used by the actuator. This mapping process includes two stages: camera intrinsic parameter calculation and extrinsic parameter transformation; in the intrinsic parameter calculation process, the camera intrinsic parameter matrix is ​​used. The pixel coordinates are normalized and combined with depth or range parameters. Determine the 3D position vector of the key points in the camera coordinate system. This is to achieve the recovery of two-dimensional pixels into three-dimensional geometric points. During the extrinsic parameter transformation process, the extrinsic parameters from the camera to the machine body (…) , )right Perform a coordinate transformation to obtain the spatial position of the apical bud in the body coordinate system. , obtained With confidence level The data is output together and used as control input to drive the budding actuator, thereby achieving a controllable transformation from image key points to the machine's spatial position. The bud spatial position in the machine coordinate system... The formula is as follows:

[0071]

[0072] In the formula, Represents the camera's rotation matrix; Represents the camera's translation vector; This represents the inverse matrix of the intrinsic parameters.

[0073] During pest detection, the i-th frame standardized image is obtained by calling the previously stabilized image and illumination normalization. reuse line tape Canopy segmentation mask To eliminate non-cotton plant background elements such as soil, weeds, sky, and row gaps, and to simultaneously suppress interference from canopy segmentation spillover and row boundary offset on subsequent detection, the effective search domain is defined as follows: Its definition is as follows:

[0074] In the formula Morphological corrosion; radius . structural elements.

[0075] Then Intradomain calculation of color enhancement indicators to improve the separability of lesions and healthy leaves; VARI is preferred as a robust chromaticity characterization, and is aligned and normalized with the RGB channels of the standard frame to form the detection input tensor for subsequent steps. To suppress the effects of low light, shadows, and slight exposure drift, an improved indicator with a stabilization term is used. To effectively avoid ratio divergence caused by the denominator approaching zero under low brightness or shadow conditions, it is defined as follows:

[0076]

[0077] In the formula, The standard frame in pixels The red, green, and blue channel values ​​at the location; It is the stability constant.

[0078] Next in the valid domain The internal execution region is divided into sub-regions, and the superpixel segmentation SLIC algorithm is used to obtain a set of sub-regions. The detection input tensor constructed based on S32 For each sub-region Calculate three statistical features: color, texture, and structure, to construct a feature vector. This includes Lab / HSV channel mean and variance, improved VARI / ExG index, local entropy and energy, LBP histogram, edge density, and porosity. A robust statistical estimation model was used to establish the background feature distribution of healthy leaves. Based on this, an anomaly function is defined to sort and filter each sub-region, thereby obtaining a set of candidate pest patches. Here is the anomaly scoring function. Using Mahalanobis distance can simultaneously consider the correlation and scale differences between features; The definition is as follows:

[0079]

[0080] In the formula, Indicates the first The first frame of the image is obtained by dividing it within the effective domain. Sub-regions; These are the feature mean and covariance matrices established based on healthy leaf samples, respectively. This is the anomaly threshold.

[0081] Finally, within the valid domain, a clipping window is defined for the minimum bounding rectangle of the candidate patches. Normalization to a fixed scale Forming subgraphs Subgraph Input an SSD-Lite detection network and output pest categories. Patch bounding box With confidence level ; Patch bounding box Then offset with the cropping window Combined Non-maximum suppression and threshold consistency constraints are applied to overlapping candidate results to obtain a redundant identification set. For targets requiring execution / source tracing and localization, the internal parameters obtained from the aforementioned calibration are used here. With external references Under the near-ground plane assumption, homography is used to perform pixel-to-ground coordinate transformation to obtain the homogeneous coordinates of image pixels. The formula is as follows:

[0082]

[0083] In the formula, These are the homogeneous coordinates of the image pixels; The coordinates are homogeneous coordinates on the ground plane; This is the camera intrinsic parameter matrix; From external references The first two columns and the translation vector constitute the symbol " "Indicates equal proportions"

[0084] Finally, for each detected target, within its bounding box... The anomaly map is obtained by normalization and region pooling. ; Prior anomaly With detection confidence The risk score is obtained by weighted synthesis. When the score exceeds the threshold, a reporting data packet is generated and sent to the central end through the communication link. The data packet contains a timestamp. Pest categories Risk Score Full frame bounding box Ground coordinates Evidence Subgraph Information such as frame identifiers is also included. To adapt to different operating conditions, the weighting coefficients and trigger thresholds can be adaptively adjusted according to insect pressure levels, operating speed, and lighting conditions. Furthermore, in cases of communication anomalies or bandwidth limitations, high-risk targets are prioritized for reporting, while low-risk targets are cached to delay synchronization. Risk scoring is also included. The calculation formula is as follows:

[0085] Triggering conditions:

[0086] During the execution phase, based on the spatial location and confidence level of the apical bud obtained from the aforementioned steps, combined with vehicle speed, attitude stability, and safety sensor status, a work window is generated and work permission is determined. When both the confidence level and safety threshold are met, the current target is locked and the execution process begins. If any condition is not met, the action is prohibited and a reason code is recorded. Then, within the work window, a single trajectory planning and soft limit / collision check are completed, close alignment is entered, and the end shearing mechanism is triggered. The entire process is protected by three types of interlocks: human-machine safety, obstacle detection, and attitude abnormality. If jamming or timeout occurs, the system automatically retreats to a safe position and retryes within a limited number of attempts. If it still fails, the operation is terminated and the reason for failure is marked. Finally, after the operation is completed, the in-situ re-inspection of the shearing effect is performed, and an operation record package is generated (timestamp, pose and trajectory summary, execution current / torque curve, result label, ground coordinates, before and after comparison). When the pest risk score reaches the threshold, it is associated with this record and reported to the central end. When bandwidth is limited, the summary is uploaded and evidence is cached locally. If it fails, it enters the retry queue.

[0087] The above-described specific embodiments are merely preferred embodiments of the present invention and are not intended to limit the implementation of the present invention or the scope of the claims. All equivalent changes and modifications made in accordance with the scope of patent protection of the present invention should be included within the scope of the present invention patent application.

Claims

1. A cotton plant topping system based on image perception and intelligent control, characterized in that, The system includes a body (1), an image and attitude perception module (2), a topping execution module (3), and an image processing and intelligent control module (4). The body (1) includes a mobile platform, a metal frame (11), and wheels (12) for carrying the various functional modules and moving between rows in the cotton field. The image and attitude perception module (2) includes an industrial RGB camera (21), an inertial measurement unit (IMU) (22), and a vehicle speed sensor (23) installed on the front upper part of the body shell for synchronously acquiring images of the cotton canopy and vehicle motion attitude information. The topping execution module... Block (3) includes a lifting module (31), a guide slide (32), a shearing drive assembly and a shearing end effector (33). The lifting module (31) and the guide slide (32) are used to drive and constrain the shearing end effector (33) to move in the vertical direction. The shearing end effector (33) is used to shear the top bud of the cotton plant. The image processing and intelligent control module (4) includes an industrial computer. The industrial computer is electrically connected to an industrial camera (21), an inertial measurement unit (IMU) (22), a vehicle speed sensor (23) and the topping execution module (3).

2. The cotton plant topping system based on image perception and intelligent control according to claim 1, characterized in that, The walking wheel (12) is made of wear-resistant rubber material, and the surface of the walking wheel (12) is provided with anti-slip pattern.

3. The cotton plant topping system based on image perception and intelligent control according to claim 1, characterized in that, The shearing end effector (33) includes a rotating blade assembly with an annular limiting cover on the outside of the rotating blade assembly, and the rotating blade assembly is made of corrosion-resistant stainless steel.

4. A method for topping cotton plants based on image perception and intelligent control, characterized in that, Includes the following steps: S1: Cotton canopy images and vehicle speed and attitude information are acquired synchronously through an industrial RGB camera, vehicle speed sensor and inertial measurement unit (IMU). Camera calibration and distortion correction are performed on the acquired images. Image stabilization, deblurring and illumination normalization are performed in combination with vehicle speed and attitude information to generate an enhanced image dataset containing the region of interest at the top of the cotton plant. S2: In the enhanced image dataset, row band detection and canopy segmentation are performed on the cotton plant canopy to define the working row region and obtain the canopy mask; a top interest region is constructed in the upper region of the canopy, the top interest region is input into the key point recognition network, the pixel coordinates and confidence of the top bud key point are output, and the top bud key point is mapped from the image plane to the body coordinate system by combining the camera intrinsic parameters and the camera-body extrinsic parameters to obtain the spatial position of the top bud and the corresponding confidence; S3: Within the effective search domain defined by the enhanced image and the canopy mask, calculate the color enhancement indicator, perform region division and feature extraction on the search domain, obtain candidate pest patches based on the anomaly function, input the sub-image corresponding to the candidate patch into the pest detection network to output the pest category, bounding box and confidence score, and calculate the pest risk score by combining the anomaly and confidence scores. When the risk score exceeds the preset threshold, complete the conversion from pixel coordinates to ground coordinates and generate pest risk reporting data. S4: Based on the spatial location of the apical bud, confidence level, vehicle speed, attitude, and safety sensor status, the operation permission is determined. Under the permission conditions, the shearing trajectory planning and collision check are performed, the shearing end effector is driven to complete the shearing of the cotton plant apical bud, and after the operation is completed, the image processing process is called to re-inspect the shearing results, generate an operation record package containing timestamps, pose and trajectory data, execution parameters, result labels, and before and after comparison images, and when the conditions are met, it is associated with the pest risk reporting data and sent to the central end.

5. The cotton plant topping method based on image perception and intelligent control according to claim 4, characterized in that, Step S1 further includes: S11: Under unified trigger control, the industrial RGB camera, vehicle speed sensor and inertial measurement unit (IMU) synchronously acquire data, and bind the distortion-corrected image with the corresponding speed and attitude information to form the original multi-source synchronous dataset. S12: Calculate the intra-frame pose increment during the camera exposure time based on vehicle speed and attitude information, and construct the image plane homography matrix using a local plane approximation model. Perform inverse mapping and resampling on the original image to achieve image stabilization and jitter correction. S13: Estimate the direction and length of motion blur based on velocity and attitude information, construct a motion blur kernel to deblur the stabilized image, and implement illumination normalization and vegetation color index enhancement to improve the separability of cotton plants from the background.

6. The cotton plant topping method based on image perception and intelligent control according to claim 5, characterized in that, In step S12, the planar homography matrix The calculation formula is: ， In the formula, For the camera intrinsic parameter matrix, For the fixed extrinsic parameter rotation from the IMU to the camera coordinate system, This represents the relative rotation increment within the exposure time window. Camera extrinsic matrix transpose, This refers to the relative translation increment within the exposure time window; This is the transpose of the local plane unit normal vector in the camera coordinate system; The directed distance (scalar) to the local plane; It is the inverse of the camera intrinsic parameter matrix.

7. The cotton plant topping method based on image perception and intelligent control according to claim 4, characterized in that, Step S2 includes the following steps: S21: Input the enhanced image into the deep learning segmentation network to perform pixel-level semantic segmentation of the cotton plant apical domain and obtain the apical domain semantic mask. S22: Perform connected component analysis and morphological filtering on the semantic mask to remove isolated noise points and non-target regions, obtain a cleaned mask, and construct a top candidate band based on the upper boundary of the cleaned mask, calculate its minimum bounding rectangle, generate rectangular regions of interest according to a preset expansion coefficient and perform boundary clipping, thereby forming the top ROI dataset. S23: Input the top ROI dataset into the keypoint recognition network to obtain a candidate set of apical bud keypoints, and filter and select the uniqueness of the candidate set according to the confidence threshold and the maximum confidence criterion, and output a single apical bud keypoint and its confidence. S24: Based on the obtained apical bud pixel coordinates, the key points are converted from pixel coordinates to three-dimensional coordinates in the camera coordinate system using the camera intrinsic parameter matrix and the apical depth or range parameter. Furthermore, the coordinate transformation is performed using the extrinsic parameters from the camera to the body to obtain the spatial position of the apical bud in the body coordinate system.

8. The cotton plant topping method based on image perception and intelligent control according to claim 4, characterized in that, Step S3 includes the following steps: S31: Perform superpixel segmentation within the effective search domain to obtain multiple sub-regions, calculate color, texture and structural features for each sub-region to form a feature vector, and establish an anomaly function based on the background feature distribution of healthy leaves. Sort and filter each sub-region to obtain a set of candidate pest patches. S32: Normalize the cropped sub-images corresponding to the candidate pest patches to a fixed scale and input them into the pest detection network to output the pest category, patch bounding box and detection confidence. Perform non-maximum suppression and consistency constraints on multiple candidate results to obtain redundant detection results. S33: Extract anomaly features within the bounding box of the detected target, and weight and fuse the anomaly with the detection confidence to obtain a pest risk score. When the pest risk score exceeds a preset threshold, perform pixel-to-ground plane coordinate mapping based on the camera intrinsic and extrinsic parameters under the near-ground plane assumption, and generate a risk reporting data packet containing timestamp, pest category, risk score, bounding box, ground coordinates and evidence sub-image.

9. The cotton plant topping method based on image perception and intelligent control according to claim 8, characterized in that, Specifically, step S33 involves using superpixel segmentation and Mahalanobis distance anomaly function to screen candidate pest patches within the effective search domain defined by the enhanced image and canopy mask. After identification by the pest detection network, a risk score is calculated by weighted fusion of anomaly and detection confidence. And when the threshold is exceeded, a risk data packet is generated and reported. The calculation formula is: Triggering conditions: , In the formula, For the prior anomaly degree, This represents the prior anomaly degree.

10. The cotton plant topping method based on image perception and intelligent control according to claim 4, characterized in that, Step S4 specifically includes the following steps: S41: Based on the spatial location and confidence level of the apical bud, and combined with vehicle speed, attitude stability and safety sensor status, a work window is generated. When the apical bud confidence level is higher than the first threshold and the safety conditions are met, the apical bud execution process is allowed to proceed; otherwise, execution is prohibited and the reason code is recorded. S42: Complete single shearing trajectory planning and soft limit / collision check within the operation window, drive the shearing end effector to enter close alignment and perform shearing operation, and automatically retreat to a safe position and retry according to the limit if abnormalities such as jamming or timeout occur. If the number of retry exceeds the limit, terminate the operation and mark the reason for failure. S43: After the shearing operation is completed, use an industrial camera to collect images after shearing to re-inspect the topping effect and form a work record package containing timestamps, pose and trajectory summaries, execution current or torque curves, result labels, ground coordinates and before-and-after comparison images. When the pest risk score reaches the second threshold, associate the work record package with the corresponding pest risk data and report it to the central end. When communication bandwidth is limited, prioritize uploading high-risk summaries and cache detailed evidence data locally.

Images