Intelligent centering method and system for hedge trimmer

Abstract

The present invention relates to the field of hedge trimming control, and more specifically, to an intelligent centering method and system for hedge trimmers. The method first obtains the current tool vibration coefficient based on the tool's short-term tilt angle, operating power, acceleration data, and point cloud position changes on the trimming path. The current tool centering coefficient is obtained based on the depth value difference between the pixel points on the edge of the tool area on one side of the trimmed hedge area and the pixel points in the trimmed hedge area, as well as the correlation between the depth value changes of the pixel points between each reference line and the center line of the tool area. Based on the tool vibration coefficient and the tool centering coefficient, the trimming posture deviation is obtained. The hedge offset is obtained based on the length of the branch edge line and the hue and saturation values ​​of the pixel points in the untrimmed hedge area. The trimming posture deviation and the hedge offset are combined to perform real-time centering control on the hedge trimmer. The present invention can improve the centering effect of the hedge trimmer.

Description

Technical Field

[0001] The present invention relates to the field of hedge trimmer control, and in particular to an intelligent centering method and system for a hedge trimmer. Background Art

[0002] A hedge trimmer is a piece of mechanical equipment used in landscaping operations. It is widely used in highway medians, parks, communities and other places to trim shrubs and hedges to make them neat and beautiful. Centering is usually used to describe the adjustment of the position of an object in space to place it in a central or symmetrical position. Centering of a hedge trimmer refers to dynamically aligning the trimmer's cutting tool with the geometric center axis of the hedge or a preset trimming path during operation, thereby ensuring that the trimmed hedge surface is flat and symmetrical, and adapting to the trimming needs of complex shapes. With the development of modern agriculture and garden management, higher requirements are placed on the functionality, adaptability, endurance and ease of maintenance of garden operation equipment. Especially in complex terrains such as outdoor mountains, traditional garden operation equipment has gradually been unable to meet the needs of agronomic operations.

[0003] At present, hedge trimming equipment is still mainly operated manually. However, this method requires the operator to judge the alignment status of the cutter head and the center line of the hedge with the naked eye, which is easily affected by subjective factors such as fatigue and visual angle error, resulting in an uneven trimming surface and difficulty in achieving the ideal greening trimming effect. At the same time, current automatic hedge trimmers often use preset paths or mechanical guides for trimming, and cannot achieve real-time and flexible centering adjustments based on the trimming effect and the actual operating status of the trimmer's tool, thereby reducing the centering effect of the hedge trimmer. Summary of the Invention

[0004] In order to solve the technical problem that existing methods cannot achieve real-time and flexible centering adjustment based on the natural growth form of the hedge and the actual operating status of the trimmer's blades, thereby reducing the centering effect of the hedge trimmer, the purpose of the present invention is to provide an intelligent centering method and system for a hedge trimmer. The technical solutions adopted are as follows:

[0005] The present invention provides an intelligent centering method for a hedge trimmer, the method comprising:

[0006] The pruning path is determined using point cloud data of the hedge to be pruned. During the pruning process, the tool's tilt angle, acceleration data, operating power, RGB depth image of the tool direction, and pixel hue, brightness, and saturation values ​​are acquired in real time.

[0007] The tool vibration coefficient at the current moment is obtained based on the tilt angle, operating power, acceleration data at each moment in a preset period before the current moment, as well as the point cloud position change of the pruning path; the tool area, the trimmed hedge area, and the untrimmed hedge area are identified in the RGB depth image; multiple reference lines parallel to the center line of the tool area are set in the trimmed hedge area, and the tool centering coefficient at the current moment is obtained based on the difference in depth values ​​between the edge line pixels of the tool area and the pixels of the trimmed hedge area, and the change in depth values ​​of the pixels between the reference lines and the center line of the tool area; based on the tool vibration coefficient and the tool centering coefficient, the pruning posture deviation at the current moment is obtained;

[0008] Perform edge detection on the untrimmed hedge area to obtain multiple branch edge lines. According to the length of the branch edge lines and the hue and saturation values ​​of each pixel point, the hedge offset at the current moment is obtained.

[0009] Based on the trimming posture deviation and hedge offset, the hedge trimmer is controlled in real time.

[0010] Furthermore, determining the pruning path based on the point cloud data includes:

[0011] Using the ICP algorithm and an initial pruning model constructed based on the point cloud data of the hedge to be pruned, the hedge portion in the initial pruning model is segmented using the PointNet++ algorithm, and outliers are removed using a statistical filtering algorithm to obtain an initial hedge point cloud model;

[0012] Using the RANSAC algorithm, fitting the initial hedge point cloud model to obtain an optimized hedge point cloud model, and deleting point cloud data belonging to the optimal hedge point cloud model from the initial hedge point cloud model to obtain a hedge surface point cloud model;

[0013] The ant colony algorithm is used to process the surface point cloud model of the hedge to obtain a pruning path.

[0014] Furthermore, obtaining the tool vibration coefficient at the current moment includes:

[0015] The tilt angle and the operating power at each moment in a preset time period are integrated and averaged to obtain the tool vibration intensity at the current moment;

[0016] The absolute value of the difference between the tilt angle at each moment and the next adjacent moment within the preset period is used as the tilt angle change at each moment within the preset period; the tilt angle change at each moment within the preset period and the acceleration data are integrated and averaged to obtain the first path deflection degree at the current moment;

[0017] Performing curve fitting on the point cloud positions on the trimming path traversed by the tool within a preset time period to obtain a fitted path curve, taking the absolute value of the difference in slope between each point cloud position and the next adjacent point cloud position on the fitted path curve as the position change of each point cloud position on the fitted path curve, and taking the average of the position changes of all point cloud positions on the fitted path curve as the second path deflection degree at the current moment; combining the first path deflection degree and the second path deflection degree to obtain the path deflection complexity at the current moment;

[0018] The tool vibration intensity and the path deflection complexity are integrated to obtain the tool vibration coefficient at the current moment.

[0019] Furthermore, identifying the tool area, the trimmed hedge area, and the untrimmed hedge area in the RGB depth image at the current moment includes:

[0020] Using the YOLO algorithm, the knife area and hedge area are identified in the RGB depth image at the current moment;

[0021] Taking the tool area as the boundary and following the moving direction of the trimmer tool, the hedge area before the tool area is regarded as the untrimmed hedge area, and the hedge area after the tool area is regarded as the trimmed hedge area.

[0022] Furthermore, obtaining the tool centering coefficient at the current moment includes:

[0023] The edge line of the tool area on the side of the trimmed hedge area is used as the trimming edge line of the tool area, and the pixel point closest to each pixel point on the trimming edge line and located in the trimmed hedge area is used as the comparison pixel point for each pixel point on the trimming edge line. The depth value differences between each pixel point on the trimming edge line and the comparison pixel point are averaged and negative correlation mapping is performed to obtain the surface proximity between the tool and the trimmed hedge at the current moment.

[0024] Taking any reference line as the target reference line, synthesizing the brightness value and saturation value of each pixel point of the target reference line to obtain a trimming evaluation value of each pixel point; sorting the trimming evaluation values ​​of all pixels of the target reference line to obtain an evaluation value sequence of the target reference line; determining a first selection boundary value of the target reference line based on two adjacent trimming evaluation values ​​with the largest difference selected from the evaluation value sequence; and selecting pixels on the target reference line whose trimming evaluation values ​​are greater than the first selection boundary value as trimming pixels of the target reference line;

[0025] Selecting a reference pixel point of the tool area center line with respect to the target reference line on the tool area center line, wherein the position sequence number of the reference pixel point on the tool area center line is the same as the position sequence number of the trimming pixel point on the target reference line;

[0026] Sort the depth values ​​of the trimming pixels of the target reference line and the depth values ​​of the reference pixels of the tool area center line according to the position sequence of the pixels, to obtain a first depth value sequence of the tool area center line and a second depth value sequence of the target reference line;

[0027] Analyzing the correlation between the first-order difference sequence of the first depth value sequence and the first-order difference sequence of the second depth value sequence to obtain a height change correlation of the target reference line, and taking an average of the height change correlations of all reference lines as the height similarity of the trimmed hedge at the current moment;

[0028] The surface proximity and the height similarity are integrated to obtain the tool centering coefficient at the current moment.

[0029] Furthermore, obtaining the trimming posture deviation at the current moment includes:

[0030] Performing negative correlation mapping on the tool centering coefficient at the current moment to obtain the tool offset coefficient at the current moment;

[0031] The tool offset coefficient and the tool vibration coefficient at the current moment are integrated to obtain the trimming posture deviation degree at the current moment.

[0032] Furthermore, performing edge detection on the untrimmed hedge area to obtain multiple branch edge lines includes:

[0033] Perform edge detection on the untrimmed hedge area to obtain multiple edge lines of the untrimmed hedge area, perform straight line fitting on the pixel points on each branch edge line of the untrimmed hedge area to obtain a fitting straight line for each branch edge line;

[0034] A branch edge line is selected from all edge lines of the untrimmed hedge area, wherein an extension line of the fitting straight line of the branch edge line passes through the tool area.

[0035] Furthermore, obtaining the hedge offset at the current moment includes:

[0036] After performing negative correlation mapping and normalization processing on the fitting errors between each branch-stem edge line and the corresponding fitting straight line, a weight parameter of each branch-stem edge line is obtained. The weight parameter of each branch-stem edge line is used to perform weighted summation on the lengths of each branch-stem edge line to obtain the first hedge offset coefficient at the current moment;

[0037] Sort the depth values ​​of all pixels in the untrimmed hedge area to obtain a third depth value sequence for the untrimmed hedge area. Select two depth values ​​from the third depth value sequence whose difference between two adjacent depth values ​​has the largest absolute value. Use the maximum of the two selected depth values ​​as a second selected boundary value for the untrimmed hedge area. Pixels in the untrimmed hedge area whose depth values ​​are greater than the second selected boundary value are used as leaf pixels in the untrimmed hedge area. Perform negative correlation mapping on the average value of the hue value and the saturation value of each leaf pixel in the untrimmed hedge area to obtain a second hedge offset coefficient at the current moment.

[0038] The first hedge offset coefficient and the second hedge offset coefficient are integrated to obtain the hedge offset degree at the current moment.

[0039] Furthermore, the real-time centering control of the hedge trimmer includes:

[0040] Combining the pruning posture deviation and the hedge offset to obtain the tool centering adjustment strength at the current moment;

[0041] The angle between the tangent plane of the optimal hedge point cloud model at the current position of the tool and the horizontal plane is used as the expected tilt angle at the current moment, and the absolute value of the difference between the tilt angle at the current moment and the expected tilt angle is used as the tilt angle error value at the current moment;

[0042] Based on the calculation formula of the tool centering adjustment parameter, the tool centering adjustment parameter at the current moment is obtained. The calculation formula of the tool centering adjustment parameter is:

[0043]

[0044] in, Indicates the tool centering adjustment parameters at the current moment; Indicates the tool centering adjustment strength at the current moment; Indicates the tilt angle error value at the current moment; represents the normalization function;

[0045] The tool centering adjustment parameter at the current moment is used as the proportional gain coefficient of the PID controller, and the PID controller is used to perform real-time centering control on the trimmer tool.

[0046] The present invention also proposes an intelligent centering system for a hedge trimmer, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the system implements any one of the steps of an intelligent centering method for a hedge trimmer.

[0047] The present invention has the following beneficial effects:

[0048] The present invention takes into account that the existing methods cannot achieve real-time and flexible adjustment of centering based on the natural growth form of the hedge and the actual operating state of the trimmer tool, thereby reducing the centering effect of the hedge trimmer. Therefore, the trimming path is first determined, and the inclination angle, acceleration data, operating power and RGB depth image of the tool in the direction of the tool are collected in real time during the advancement of the trimmer along the trimming path. Considering that the operating posture of the trimmer tool is affected by the vibration generated by the actual operation process of the tool, the real-time vibration degree of the tool during the advancement of the trimmer is first reflected by the acquired tool vibration coefficient. At the same time, considering that when the trimmer tool has a better centering effect, its trimming effect on the hedge is also better, at this time the distance between the trimmed hedge and the tool surface is closer, and the height of each position of the trimmed hedge is higher The centering coefficient of the tool is close to and similar to the height distribution of each position of the tool. Therefore, the centering state of the trimmer tool at the current moment can be reflected by the obtained tool centering coefficient, and the degree of deviation of the trimming posture of the trimmer at the current moment can be reflected by the trimming posture deviation. Considering that during the hedge trimming process, when the centering state of the trimmer tool is unstable, the blade fails to directly contact the hedge stem, causing the hedge stem to be pushed and offset. Therefore, the degree of hedge offset caused by the unstable tool centering at the current moment can be reflected by the obtained hedge offset. Then, the hedge trimmer can be controlled in real time by combining the trimming posture deviation and the hedge offset. Therefore, more flexible centering adjustment can be made based on the trimming effect and the actual operating state of the trimmer tool, thereby improving the centering effect of the hedge trimmer. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 A flow chart of an intelligent centering method for a hedge trimmer provided by one embodiment of the present invention;

[0051] Figure 2 A schematic side view of a trimmer provided by one embodiment of the present invention;

[0052] Figure 3 A schematic diagram of an initial pruning model provided by one embodiment of the present invention;

[0053] Figure 4A schematic diagram of an optimal hedge point cloud model and a hedge surface point cloud model provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0054] To further illustrate the technical means and effectiveness of the present invention in achieving its intended objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of an intelligent centering method and system for a hedge trimmer according to the present invention. In the following description, references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0055] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0056] The specific scheme of the intelligent centering method and system for a hedge trimmer provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0057] See also Figure 1 , which shows a flow chart of an intelligent centering method for a hedge trimmer provided by one embodiment of the present invention, the method comprising:

[0058] Step S1: Determine the pruning path using the point cloud data of the hedge to be pruned. During the pruning process, obtain the tool's tilt angle, acceleration data, operating power, RGB depth image of the tool direction, and hue, brightness, and saturation values ​​of the pixel points in real time.

[0059] Common hedges are generally rotating bodies, such as cylinders, cones, and spheres. Currently, mechanized equipment for hedge pruning is equipped with a 360° rotating tool at the robotic arm position. The working method of hedge pruning equipment is generally to manually drive a vehicle to the vicinity of the hedge to be pruned, and then manually operate the robotic arm to the top of the center position of the hedge, and then use the installed tool to rotate and prune with the central axis of the hedge as the rotation center. The degree of automation of this operation method is extremely low, and the alignment process of the tool position and the hedge is completely dependent on manual labor, mainly through manual alignment by human eyes through video. As a result, this method is not only time-consuming and labor-intensive, but also requires a high level of proficiency of the operator. Therefore, the efficiency of hedge pruning is difficult to improve and the pruning quality is difficult to guarantee.

[0060] The embodiment of the present invention first installs a structured light camera and an IMU inertial motion unit on the trimmer. Figure 2, which shows a side view schematic diagram of a trimmer provided by an embodiment of the present invention. When it is necessary to trim the hedge, the robotic arm is first operated, and the structured light camera installed on the robotic arm is used to surround and shoot the hedge to be trimmed to obtain multiple RGB depth images of the hedge to be trimmed. In one embodiment of the present invention, a spherical hedge is taken as an example, wherein the four channels of the pixel points of the RGB depth image are R value, G value, B value and depth value, respectively, and the depth value represents the distance from the pixel point to the camera. Then, the camera intrinsic parameters and the imaging principle of the camera are combined with the multiple RGB depth images of the hedge to be trimmed to obtain the corresponding point cloud data. Among them, converting multiple RGB depth images into point cloud data is a technical means well known to those skilled in the art and will not be elaborated here.

[0061] A pruning path is then determined based on the point cloud data, and a subsequent trimmer can trim the hedge along the pruning path.

[0062] Preferably, in one embodiment of the present invention, the method for obtaining the pruning path specifically includes:

[0063] The initial pruning model is constructed using the ICP algorithm based on the point cloud data of the hedge to be pruned. Figure 3 , which shows a schematic diagram of an initial pruning model provided by an embodiment of the present invention. Since the initial pruning model contains not only point cloud data belonging to the hedge part, but also point cloud data belonging to the background part and outlier point cloud data, the hedge part in the initial pruning model can be segmented using the PointNet++ algorithm, and the outliers can be removed using the statistical filtering algorithm to obtain the initial hedge point cloud model.

[0064] The RANSAC algorithm is used to fit the initial hedge point cloud model to obtain an optimized hedge point cloud model. The RANSAC algorithm can optimize the point cloud model and optimize the point cloud model into a regular shape. Since the hedge to be pruned in one embodiment of the present invention is spherical, the optimized hedge point cloud model after the RANSAC algorithm optimization process is close to a sphere. Then, in the initial hedge point cloud model, the point cloud data belonging to the optimal hedge point cloud model is deleted to obtain the hedge surface point cloud model. Please refer to Figure 4 , which shows a schematic diagram of the optimal hedge point cloud model and the hedge surface point cloud model provided by an embodiment of the present invention. Then, the hedge surface point cloud model can be processed using the ant colony algorithm to obtain the pruning path. Determining the path by using the ant colony algorithm is a technical means well known to those skilled in the art and will not be described in detail here.

[0065] After determining the trimming path for the hedge, the trimmer can perform trimming operations along the trimming path. During the trimming process, the IMU inertial motion unit collects the tool's inclination angle and acceleration data in real time. The inclination angle represents the angle between the tool and the horizontal direction, and its value range is 0 to 90 degrees. The trimmer's power monitoring system collects the tool's operating power in real time, and the structured light camera collects the RGB depth image of the tool direction in real time. The RGB depth image contains the tool image features and the hedge image features.

[0066] At the same time, the embodiment of the present invention also needs to perform color space conversion on the RGB depth image based on the R, G, and B three-channel values ​​of the pixels in the RGB depth image, convert it into the HSV color space, and obtain the hue value, brightness value, and saturation value of the pixels in the RGB depth image, so as to facilitate the subsequent analysis of the color features of the pixels in the RGB depth image.

[0067] It should be noted that the collection frequencies of various types of data may be different. Therefore, during the data collection process, it is necessary to synchronously process various types of data through sampling or interpolation methods to facilitate subsequent data analysis.

[0068] Step S2: Obtain the tool vibration coefficient at the current moment based on the tilt angle, operating power, acceleration data at each moment in a preset period before the current moment, and the point cloud position change of the pruning path; identify the tool area, the trimmed hedge area, and the untrimmed hedge area in the RGB depth image; set multiple reference lines parallel to the center line of the tool area in the trimmed hedge area, and obtain the tool centering coefficient at the current moment based on the difference in depth values ​​between the edge line pixels of the tool area and the pixels of the trimmed hedge area, and the change in depth values ​​of the pixels between the reference line and the center line of the tool area; obtain the pruning posture deviation at the current moment based on the tool vibration coefficient and the tool centering coefficient.

[0069] The pruning posture of the trimmer tool is affected by the actual operation of the tool. The hedge trimmer completes the pruning operation through the high-speed reciprocating cutting blade. The inertia of the blade movement will directly cause mechanical vibration, which may cause the tool to fail to prune at the expected angle. In addition, at different inclination angles of the tool, the inertia generated by the tool movement has different effects on the vibration of the trimmer. For example, when the blade surface of the trimmer tool is more perpendicular to the ground, the direction of the inertial force generated by the tool movement is closer to the direction of the gravity of the trimmer, the stronger the vibration is, and the worse the centering stability of the trimmer tool posture. At the same time, when the pruning path is more complex, the robotic arm needs to rotate and change direction more frequently, which will also cause the tool to vibrate. However, the centering stability of the tool is reduced, and the tool is often affected by the unstable factors described above for a period of time. Therefore, the inclination angle, operating power, acceleration data of the tool at each moment in the preset time period before the current moment, and the changes in the position of each point cloud on the pruning path can be analyzed, and the obtained tool vibration coefficient can be used to reflect the real-time vibration degree of the tool during the operation of the trimmer. The length of the preset time period is usually in the range of 10~30. In one embodiment of the present invention, the length of the preset time period is set to 15, that is, the preset time period includes 15 moments before the current moment. The specific length of the preset time period can also be set by the implementer according to the specific implementation scenario, and is not limited here.

[0070] Preferably, in one embodiment of the present invention, the method for obtaining the tool vibration coefficient at the current moment specifically includes:

[0071] First, the larger the inclination angle of the tool and the closer it is to 90 degrees, the more perpendicular the tool is to the ground. At this time, the greater the operating power of the trimmer, the stronger the vibration generated. Therefore, the inclination angle and operating power at each moment in the preset time period can be combined and averaged to obtain the tool vibration intensity at the current moment. The greater the tool vibration intensity, the more obvious the tool vibration in the short term.

[0072] In an embodiment of the present invention, the integration of the tilt angle and the operating power can be achieved by calculating the sum or product of the tilt angle and the operating power at a certain moment, which is not limited here, and the subsequent steps of the integrated processing of two or more data can also be achieved using the same method.

[0073] Then, the absolute value of the difference in the inclination angle between each moment and the next adjacent moment in the preset time period is used as the inclination angle change at each moment in the preset time period. The inclination angle change and acceleration data at each moment in the preset time period are integrated and averaged to obtain the first path deflection degree at the current moment. The larger the first path deflection degree, the more obvious the change in the tool inclination angle and speed of the trimmer during the short-term advancement process, and thus the more obvious the vibration of the tool caused in the short term.

[0074] It should be noted that there is no adjacent next moment to the last moment of the preset period. In this case, the average value of the tilt angle changes of all moments before the last moment of the preset period can be used as the tilt angle change of the last moment to facilitate subsequent calculations.

[0075] Curve fitting is performed on the point cloud positions on the pruning path passed by the tool within a preset time period to obtain a fitted path curve, wherein the least squares method or other methods can be used to implement curve fitting, which is not limited here, and then the absolute value of the difference in slope between each point cloud position and the next adjacent point cloud position on the fitted path curve is used as the position change of each point cloud position on the fitted path curve, and the average value of the position change of all point cloud positions on the fitted path curve is used as the second path deflection degree at the current moment. The larger the second path deflection degree, the more frequent the propulsion direction of the trimmer changes in the short term, and the more obvious the vibration of the tool in the short term. Then, the first path deflection degree and the second path deflection degree are combined to obtain the path deflection complexity at the current moment.

[0076] It should be noted that there is no adjacent next point cloud position to the last point cloud position on the fitting path curve. In this case, the average value of the position changes of all point cloud positions before the last point cloud position on the fitting path curve can be used as the position change of the last point cloud position to facilitate subsequent calculations.

[0077] Finally, the tool vibration intensity and path deflection complexity are integrated to obtain the tool vibration coefficient at the current moment.

[0078] As an example, in one embodiment of the present invention, the expression of the tool vibration coefficient at the current moment may be specifically, for example, as follows:

[0079]

[0080]

[0081]

[0082] in, Indicates the tool vibration coefficient at the current moment; Indicates the tool vibration intensity at the current moment; Indicates the path deflection complexity at the current moment; Indicates the first The tilt angle at a moment; Indicates the first The operating power at each moment; Indicates the number of moments in the preset period before the current moment of the tool; Indicates the first The change in tilt angle at each moment; Indicates the first Acceleration data at each moment; Indicates the first The position change of each point cloud position; Indicates the first path deflection degree at the current moment; Indicates the second path deflection degree at the current moment; Indicates the number of point cloud positions on the fitted path curve.

[0083] Considering that when the trimmer tool is well centered, its trimming effect on the hedge is also better, the distance between the trimmed hedge and the tool surface is relatively close, and the heights of each position of the trimmed hedge are relatively close and similar to the height distribution of each position of the tool, the embodiment of the present invention first identifies the tool area, the trimmed hedge area and the untrimmed hedge area in the RGB depth image at the current moment.

[0084] Preferably, in one embodiment of the present invention, the method for obtaining the tool area, the trimmed hedge area, and the untrimmed hedge area in the RGB depth image specifically includes:

[0085] The existing YOLO algorithm is used to identify the tool area and hedge area in the RGB depth image at the current moment. Then, with the tool area as the boundary, along the moving direction of the trimmer tool, the hedge area before the tool area is regarded as the untrimmed hedge area, and the hedge area after the tool area is regarded as the trimmed hedge area.

[0086] Then, a preset number of reference lines parallel to the center line of the tool area are evenly set in the trimmed hedge area, wherein the preset number is set to 20~40. In one embodiment of the present invention, the preset number is set to 30. The specific length of the preset number can also be set by the implementer according to the specific implementation scenario, which is not limited here. It should be noted that the trimmer tool is generally long and strip-shaped, and the center line of the tool area refers to the central axis of the tool.

[0087] From the above analysis, it can be seen that when the trimmer tool is well-centered, the distance between the surface of the trimmed hedge and the tool surface is relatively close. At this time, the difference in depth values ​​between the pixel points of the edge line of the tool area on the side of the trimmed hedge area and the pixel points of the trimmed hedge area is small. At the same time, when the trimmer tool is well-centered, the heights of each position of the trimmed hedge are relatively close and similar to the height distribution of each position of the tool. At this time, the depth value change of the pixel points on the reference line and the depth value change of the pixel points on the center line of the tool area are relatively consistent. Therefore, the difference in depth values ​​between the pixel points of the edge line of the tool area on the side of the trimmed hedge area and the pixel points of the trimmed hedge area, as well as the correlation of the depth value changes of the pixel points between each reference line and the center line of the tool area can be analyzed. The obtained tool centering coefficient reflects the centering status of the trimmer tool at the current moment. The larger the tool centering coefficient, the better the centering effect of the trimmer tool at the current moment.

[0088] Preferably, in one embodiment of the present invention, the method for obtaining the tool centering coefficient at the current moment specifically includes:

[0089] First, the edge line of the tool area on the side of the trimmed hedge area is used as the trimming edge line of the tool area, and the pixel point closest to each pixel point of the trimming edge line and located in the trimmed hedge area is used as the comparison pixel point of each pixel point of the trimming edge line. The absolute value of the difference in depth value between each pixel point of the trimming edge line and the corresponding comparison pixel point is averaged and negative correlation mapping is performed to obtain the surface proximity between the tool and the trimmed hedge at the current moment. The greater the surface proximity, the closer the distance between the trimmed hedge surface and the tool surface is, and the better the trimmer tool alignment effect is.

[0090] Then, considering that the trimmed hedge area contains not only pixel points representing the trimmed hedge cuts, but also pixel points representing the untrimmed internal branches and leaves, in the subsequent correlation analysis process, pixel points representing the trimmed hedge cuts should be selected as much as possible for analysis to improve the evaluation accuracy. Compared with the untrimmed internal branches and leaves that are blocked for a long time and have insufficient light, the trimmed hedge cuts or leaf sections receive stronger light, and their pixel points have higher brightness and saturation.

[0091] Therefore, first, any reference line is used as the target reference line, and the brightness value and saturation value of each pixel point of the target reference line are integrated to obtain the pruning evaluation value of each pixel point of the target reference line. The larger the pruning evaluation value of a pixel point of the target reference line, the more likely the pixel point is to represent a hedge cut. The pruning evaluation values ​​of all pixels of the target reference line are sorted to obtain the evaluation value sequence of the target reference line. The sorting method can be either increasing or decreasing according to the pruning evaluation value. In the evaluation value sequence, the two pruning evaluation values ​​with the largest absolute value of the difference between the two adjacent pruning evaluation values ​​are selected, and the selected pruning evaluation values ​​are sorted. The maximum value of the two pruning evaluation values ​​is used as the first selected boundary value of the target reference line, and the pixel point on the target reference line whose pruning evaluation value is greater than the first selected boundary value is used as the pruning pixel point of the target reference line. The pruning pixel point is the pixel point of the pruned hedge incision. Then, the reference pixel point of the tool area center line with respect to the target reference line is selected on the tool area center line, wherein the position serial number of the reference pixel point on the tool area center line is the same as the position serial number of the pruning pixel point on the target reference line, that is, a reference pixel point of the target reference line and a pruning pixel point on the corresponding target reference line are in the same row.

[0092] According to the position order of the pixel points, for example, the depth values ​​of the trimmed pixel points of the target reference line and the depth values ​​of the reference pixel points of the tool area center line can be sorted respectively in the order from top to bottom or from bottom to top of the RGB depth image to obtain a first depth value sequence of the tool area center line and a second depth value sequence of the target reference line. Then, the absolute value of the Pearson correlation coefficient between the first-order difference sequence of the first depth value sequence and the first-order difference sequence of the second depth value sequence is used as the height change correlation between the target reference line and the tool area center line. The same method as above can be used to obtain the height change correlation between each reference line and the tool area center line. The larger the height change correlation, the closer the heights of the trimmed hedges at various positions along the tool direction and the more similar the height distribution of the various positions in the tool direction. Then, the average value of the height change correlations between all reference lines and the tool area center line can be used as the height similarity of the trimmed hedges at the current moment.

[0093] Finally, the surface proximity and height similarity are integrated to obtain the tool centering coefficient at the current moment.

[0094] As an example, in one embodiment of the present invention, the expression of the tool centering coefficient at the current moment may be specifically, for example, as follows:

[0095]

[0096]

[0097]

[0098] in, Indicates the tool centering coefficient at the current moment; represents the surface proximity between the cutter and the trimmed hedge at the current moment; represents the height similarity of the trimmed hedge at the current moment; Indicates the trimming edge line The depth value of each pixel; Indicates the trimming edge line The depth value of the pixel compared with the pixel; Indicates the number of pixels on the trimming edge line; Indicates the The height variation correlation between the reference line and the center line of the tool area; Indicates the number of reference lines in the trimmed hedge area, i.e. the preset number; Indicates the preset first adjustment parameter, which is used to prevent the denominator from being 0. The value range is In one embodiment of the present invention, Set to 0.01, The specific value of can also be set by the implementer according to the specific implementation scenario and is not limited here.

[0099] It should be noted that, in other embodiments of the present invention, negative correlation mapping may also be achieved through other basic mathematical operations, which is not limited here.

[0100] The more obvious the vibration of the trimmer tool at the current moment and the worse the tool centering effect, the greater the deviation of the trimming posture of the trimmer tool at this moment. Therefore, the trimming posture deviation at the current moment can be obtained based on the tool vibration coefficient and the tool centering coefficient. The trimming posture deviation reflects the real-time deviation degree of the trimming posture. Subsequently, the trimmer can be adjusted in real time based on the trimming posture deviation to improve its centering effect.

[0101] Preferably, in one embodiment of the present invention, the method for obtaining the trimming posture deviation at the current moment specifically includes:

[0102] The tool centering coefficient at the current moment is negatively mapped to obtain the tool offset coefficient at the current moment, and the tool offset coefficient and tool vibration coefficient at the current moment are integrated to obtain the trimming posture deviation at the current moment.

[0103] As an example, in one embodiment of the present invention, the expression of the trimming posture deviation at the current moment can be specifically, for example, as follows:

[0104]

[0105] in, Indicates the trimming posture deviation at the current moment; Indicates the tool centering coefficient at the current moment; Indicates the tool offset coefficient at the current moment; Indicates the tool vibration coefficient at the current moment; Indicates the preset second adjustment parameter, which is used to prevent the denominator from being 0. The value range is In one embodiment of the present invention, Set to 0.01, The specific value of can also be set by the implementer according to the specific implementation scenario and is not limited here.

[0106] At this point, the analysis of the real-time deviation of the trimmer's centering trimming posture is completed.

[0107] Step S3: Perform edge detection on the untrimmed hedge area to obtain multiple branch edge lines, and obtain the hedge offset at the current moment based on the length of the branch edge lines and the hue and saturation values ​​of each pixel point.

[0108] During the hedge trimming process, when the centering state of the pruning tool is unstable, the blade fails to directly contact the hedge branches, causing the hedge branches to be pushed and offset. The offset hedge branches are mainly reflected in the state of the hedge plants in the unpruned area. After being pushed away by the hedge trimmer, the offset hedge branches tend to spread outward from the center of the trimmer. In the RGB depth image, part of the hedge branches are exposed to the camera's field of view. Compared with the leaf edges on the surface, these branches have longer and nearly straight edge features. Therefore, edge detection can be performed on the unpruned hedge area first to obtain multiple branch edge lines.

[0109] Preferably, in one embodiment of the present invention, the method for obtaining multiple branch edge lines in an untrimmed hedge area specifically includes:

[0110] Edge detection is performed on the untrimmed hedge area to obtain multiple edge lines of the untrimmed hedge area, and straight line fitting is performed on the pixel points on each branch edge line of the untrimmed hedge area to obtain a fitting straight line for each branch edge line, wherein the existing least squares method or other methods can be used to achieve straight line fitting, which is not limited here, and then the branch edge line is selected from all edge lines of the untrimmed hedge area, wherein the extension line of the fitting straight line of the branch edge line passes through the tool area.

[0111] From the above analysis, it can be seen that the branches have long edge features. When the branches of the hedge are pushed and offset, in addition to the exposed branches, the back of the pushed hedge leaves are also exposed in the camera field of view. Compared with the front of the leaves, the back of the leaves has a relatively rough surface and strong diffuse reflection, so its saturation in the image is lower. In addition, compared with the back of the leaves, the color of the front of the leaves is closer to pure green, so the hue value of the back of the leaves is lower. Therefore, the length of the edge line of each branch and the hue and saturation values ​​of each pixel point in the untrimmed hedge area can be analyzed. The hedge offset degree obtained reflects the degree of hedge offset caused by the unstable tool centering at the current moment. Subsequently, the hedge trimmer can be controlled in real time based on the trimming posture deviation at the current moment.

[0112] Preferably, in one embodiment of the present invention, the method for obtaining the hedge offset at the current moment specifically includes:

[0113] First, the fitting error between each branch edge line and the corresponding fitting straight line is negatively correlated and normalized to obtain the weight parameter of each branch edge line, where the fitting error can be represented by the root mean square error, which is not limited here. Since the hedge branches have longer edge features and the shape is approximately straight, the larger the weight parameter, the more likely the branch edge line is to represent the real hedge branch. Therefore, the weight parameter of each branch edge line can be used to perform weighted summation on the length of each branch edge line to obtain the first hedge offset coefficient at the current moment.

[0114] As an example, in one embodiment of the present invention, the expression of the first hedge offset coefficient at the current moment may be specifically, for example, as follows:

[0115]

[0116]

[0117] in, Indicates the first hedge offset coefficient at the current moment; Indicates the first The weight parameter of each branch edge line; Indicates the first The length of the branch edge line, which can be represented by the number of pixels on the branch edge line; Indicates the The fitting error between the edge line of each branch and the corresponding fitting straight line; Indicates the The fitting error between the edge line of each branch and the corresponding fitting straight line; Indicates the number of stem edge lines in the unpruned hedge area; Indicates the preset third adjustment parameter, which is used to prevent the denominator from being 0. The value range is In one embodiment of the present invention, Set to 0.01, The specific value of can also be set by the implementer according to the specific implementation scenario and is not limited here.

[0118] in, Used for Normalization is performed so that the sum of the weight parameters of all branch edge lines is equal to 1.

[0119] Then, the depth values ​​of all pixels in the untrimmed hedge area are sorted to obtain a third depth value sequence of the untrimmed hedge area, wherein the sorting method can be in ascending or descending order of depth values, which is not limited here. In the third depth value sequence, two depth values ​​with the largest absolute value of the difference between two adjacent depth values ​​are selected, and the maximum value of the two selected depth values ​​is used as the second selected boundary value of the untrimmed hedge area. The pixels in the untrimmed hedge area with depth values ​​greater than the second selected boundary value are used as leaf pixels of the untrimmed hedge area. The average value of the hue value and saturation value of each leaf pixel in the untrimmed hedge area is negatively correlated with each other to obtain the second hedge offset coefficient at the current moment.

[0120] As an example, in one embodiment of the present invention, the expression of the second hedge offset coefficient at the current moment may be specifically, for example, as follows:

[0121]

[0122] in, Indicates the second hedge offset coefficient at the current moment; Indicates the area of ​​untrimmed hedges. The hue value of each leaf pixel; Indicates the area of ​​untrimmed hedges. The saturation value of each leaf pixel; The number of leaf pixels representing the untrimmed hedge area; Indicates the preset fourth adjustment parameter, which is used to prevent the denominator from being 0. The value range is In one embodiment of the present invention, Set to 0.01, The specific value of can also be set by the implementer according to the specific implementation scenario and is not limited here.

[0123] The larger the first hedge offset coefficient and the second hedge offset coefficient, the greater the degree of offset of the untrimmed hedge at the current moment due to being pushed. The first hedge offset coefficient and the second hedge offset coefficient can then be combined to obtain the hedge offset degree at the current moment.

[0124] As an example, in one embodiment of the present invention, the expression of the hedge offset at the current moment may be specifically, for example, as follows:

[0125]

[0126] in, Indicates the hedge offset at the current moment; Indicates the first hedge offset coefficient at the current moment; Indicates the second hedge offset coefficient at the current moment.

[0127] At this point, the analysis of the situation in which the hedge branches are pushed and deflected due to the unstable centering state of the pruning tool is completed.

[0128] Step S4: Based on the trimming posture deviation and the hedge offset, the hedge trimmer is subjected to real-time centering control.

[0129] The larger the trimming posture deviation and hedge offset at the current moment, the worse the trimmer's centering state is. Therefore, the hedge trimmer can be controlled in real time based on the trimming posture deviation and hedge offset, thereby improving the centering effect of the hedge trimmer.

[0130] Preferably, in one embodiment of the present invention, the method for real-time centering control of a hedge trimmer specifically includes:

[0131] The trimming posture deviation and hedge offset are combined to obtain the tool centering adjustment strength at the current moment. The greater the tool centering adjustment strength, the greater the need for adjustment of the trimmer tool to ensure the centering state.

[0132] The angle between the section of the tool position of the optimal hedge point cloud model at the current moment and the horizontal plane is used as the expected tilt angle at the current moment, and the absolute value of the difference between the current tilt angle and the expected tilt angle is used as the tilt angle error value at the current moment. The larger the tilt angle error value, the more the tilt angle at the current moment deviates from the ideal angle.

[0133] Based on the calculation formula of the tool centering adjustment parameter, the tool centering adjustment parameter at the current moment is obtained. The calculation formula of the tool centering adjustment parameter is:

[0134]

[0135] in, Indicates the tool centering adjustment parameters at the current moment; Indicates the tool centering adjustment strength at the current moment; Indicates the tilt angle error value at the current moment; Represents the normalization function, which is used to limit the normalized value to , which makes The value range is In one embodiment of the present invention, the normalization process may be, for example, maximum and minimum value normalization. In other embodiments of the present invention, other normalization methods may be selected according to a specific range of numerical values, which will not be described in detail.

[0136] The tool centering adjustment parameter at the current moment is then used as the proportional gain coefficient of the PID controller, and the PID controller is used to perform real-time centering control of the trimmer tool. Specifically, the PID controller realizes the centering adjustment of the trimmer tool by adjusting the relative position and tilt angle of the tool.

[0137] One embodiment of the present invention provides an intelligent centering system for a hedge trimmer, the system comprising a memory, a processor, and a computer program, wherein the memory is used to store the corresponding computer program, and the processor is used to run the corresponding computer program. When the computer program runs in the processor, it can implement the method described in steps S1 to S4.

[0138] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0139] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. An intelligent centering method for a hedge trimmer, characterized in that: The method comprises: The pruning path is determined using point cloud data of the hedge to be pruned. During the pruning process, the tool's tilt angle, acceleration data, operating power, RGB depth image of the tool's direction, and the hue, brightness, and saturation values ​​of the pixels are acquired in real time. The tool vibration coefficient at the current moment is obtained based on the tilt angle, operating power, acceleration data at each moment in a preset period before the current moment, as well as the point cloud position change of the pruning path; the tool area, the trimmed hedge area, and the untrimmed hedge area are identified in the RGB depth image; multiple reference lines parallel to the center line of the tool area are set in the trimmed hedge area, and the tool centering coefficient at the current moment is obtained based on the difference in depth values ​​between the edge line pixels of the tool area and the pixels of the trimmed hedge area, and the change in depth values ​​of the pixels between the reference lines and the center line of the tool area; based on the tool vibration coefficient and the tool centering coefficient, the pruning posture deviation at the current moment is obtained; Perform edge detection on the untrimmed hedge area to obtain multiple branch edge lines. According to the length of the branch edge lines and the hue and saturation values ​​of each pixel point, the hedge offset at the current moment is obtained. Based on the trimming posture deviation and hedge offset, the hedge trimmer is controlled in real time. Determining the pruning path includes: Using the ICP algorithm and an initial pruning model constructed based on the point cloud data of the hedge to be pruned, the hedge portion in the initial pruning model is segmented using the PointNet++ algorithm, and outliers are removed using a statistical filtering algorithm to obtain an initial hedge point cloud model; Using the RANSAC algorithm, fitting the initial hedge point cloud model to obtain an optimized hedge point cloud model, and deleting point cloud data belonging to the optimal hedge point cloud model from the initial hedge point cloud model to obtain a hedge surface point cloud model; Using an ant colony algorithm, the point cloud model of the hedge surface is processed to obtain a pruning path; The obtaining of the trimming posture deviation at the current moment includes: Performing negative correlation mapping on the tool centering coefficient at the current moment to obtain the tool offset coefficient at the current moment; The tool offset coefficient and the tool vibration coefficient at the current moment are combined to obtain the trimming posture deviation at the current moment; The real-time centering control of the hedge trimmer includes: Combining the pruning posture deviation and the hedge offset to obtain the tool centering adjustment strength at the current moment; The angle between the tangent plane of the optimal hedge point cloud model at the current position of the tool and the horizontal plane is used as the expected tilt angle at the current moment, and the absolute value of the difference between the tilt angle at the current moment and the expected tilt angle is used as the tilt angle error value at the current moment; Based on the calculation formula of the tool centering adjustment parameter, the tool centering adjustment parameter at the current moment is obtained. The calculation formula of the tool centering adjustment parameter is: in, Indicates the tool centering adjustment parameters at the current moment; Indicates the tool centering adjustment strength at the current moment; Indicates the tilt angle error value at the current moment; represents the normalization function; The tool centering adjustment parameter at the current moment is used as the proportional gain coefficient of the PID controller, and the PID controller is used to perform real-time centering control on the trimmer tool.

2. The intelligent centering method for a hedge trimmer according to claim 1, characterized in that: The obtaining of the tool vibration coefficient at the current moment includes: The tilt angle and the operating power at each moment in a preset time period are integrated and averaged to obtain the tool vibration intensity at the current moment; The absolute value of the difference between the tilt angle at each moment and the next adjacent moment within the preset period is used as the tilt angle change at each moment within the preset period; the tilt angle change at each moment within the preset period and the acceleration data are integrated and averaged to obtain the first path deflection degree at the current moment; Performing curve fitting on the point cloud positions on the trimming path traversed by the tool within a preset time period to obtain a fitted path curve, taking the absolute value of the difference in slope between each point cloud position and the next adjacent point cloud position on the fitted path curve as the position change of each point cloud position on the fitted path curve, and taking the average of the position changes of all point cloud positions on the fitted path curve as the second path deflection degree at the current moment; combining the first path deflection degree and the second path deflection degree to obtain the path deflection complexity at the current moment; The tool vibration intensity and the path deflection complexity are integrated to obtain the tool vibration coefficient at the current moment.

3. The intelligent centering method for a hedge trimmer according to claim 1, characterized in that: The identifying of the tool area, the trimmed hedge area, and the untrimmed hedge area in the RGB depth image at the current moment includes: Using the YOLO algorithm, the knife area and hedge area are identified in the RGB depth image at the current moment; Taking the tool area as the boundary and following the moving direction of the trimmer tool, the hedge area before the tool area is regarded as the untrimmed hedge area, and the hedge area after the tool area is regarded as the trimmed hedge area.

4. The intelligent centering method for a hedge trimmer according to claim 1, characterized in that: The tool centering coefficient obtained at the current moment includes: The edge line of the tool area on the side of the trimmed hedge area is used as the trimming edge line of the tool area, and the pixel point closest to each pixel point on the trimming edge line and located in the trimmed hedge area is used as the comparison pixel point for each pixel point on the trimming edge line. The depth value differences between each pixel point on the trimming edge line and the comparison pixel point are averaged and negative correlation mapping is performed to obtain the surface proximity between the tool and the trimmed hedge at the current moment. Taking any reference line as the target reference line, synthesizing the brightness value and saturation value of each pixel point of the target reference line to obtain a trimming evaluation value of each pixel point; sorting the trimming evaluation values ​​of all pixels of the target reference line to obtain an evaluation value sequence of the target reference line; determining a first selection boundary value of the target reference line based on two adjacent trimming evaluation values ​​with the largest difference selected from the evaluation value sequence; and selecting pixels on the target reference line whose trimming evaluation values ​​are greater than the first selection boundary value as trimming pixels of the target reference line; Selecting a reference pixel point of the tool area center line with respect to the target reference line on the tool area center line, wherein the position sequence number of the reference pixel point on the tool area center line is the same as the position sequence number of the trimming pixel point on the target reference line; Sort the depth values ​​of the trimming pixels of the target reference line and the depth values ​​of the reference pixels of the tool area center line according to the position sequence of the pixels, to obtain a first depth value sequence of the tool area center line and a second depth value sequence of the target reference line; Analyzing the correlation between the first-order difference sequence of the first depth value sequence and the first-order difference sequence of the second depth value sequence to obtain a height change correlation of the target reference line, and taking an average of the height change correlations of all reference lines as the height similarity of the trimmed hedge at the current moment; The surface proximity and the height similarity are integrated to obtain the tool centering coefficient at the current moment.

5. The intelligent centering method for a hedge trimmer according to claim 1, characterized in that: The performing edge detection on the untrimmed hedge area to obtain multiple branch edge lines includes: Perform edge detection on the untrimmed hedge area to obtain multiple edge lines of the untrimmed hedge area, perform straight line fitting on the pixel points on each branch edge line of the untrimmed hedge area to obtain a fitting straight line for each branch edge line; A branch edge line is selected from all edge lines of the untrimmed hedge area, wherein an extension line of the fitting straight line of the branch edge line passes through the tool area.

6. The intelligent centering method for a hedge trimmer according to claim 5, characterized in that: The obtaining of the hedge offset at the current moment includes: After performing negative correlation mapping and normalization processing on the fitting errors between each branch-stem edge line and the corresponding fitting straight line, a weight parameter of each branch-stem edge line is obtained. The weight parameter of each branch-stem edge line is used to perform weighted summation on the lengths of each branch-stem edge line to obtain the first hedge offset coefficient at the current moment; Sort the depth values ​​of all pixels in the untrimmed hedge area to obtain a third depth value sequence for the untrimmed hedge area. Select two depth values ​​from the third depth value sequence whose difference between two adjacent depth values ​​has the largest absolute value. Use the maximum of the two selected depth values ​​as a second selected boundary value for the untrimmed hedge area. Pixels in the untrimmed hedge area whose depth values ​​are greater than the second selected boundary value are used as leaf pixels in the untrimmed hedge area. Perform negative correlation mapping on the average value of the hue value and the saturation value of each leaf pixel in the untrimmed hedge area to obtain a second hedge offset coefficient at the current moment. The first hedge offset coefficient and the second hedge offset coefficient are integrated to obtain the hedge offset degree at the current moment.

7. An intelligent centering system for a hedge trimmer, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images