Control method, device and equipment

Abstract

The embodiment of the invention discloses a control method, device and equipment, the method is applied to agricultural equipment provided with a camera device, and the method comprises the following steps: controlling the agricultural equipment to run between two adjacent rows of crops longitudinally arranged in a target area according to a preset planning path, obtaining target image data of the agricultural equipment in the driving process according to a camera device; acquiring a first included angle between the navigation line and a longitudinal axis in the image coordinate system of the target image data, wherein the longitudinal axis is the same as the driving direction of the agricultural equipment; according to the first component of the agricultural equipment in the longitudinal axis direction and the second component of the agricultural equipment in the transverse axis direction in the moving coordinate system, the angular deviation of the agricultural equipment in the driving direction is determined; determining the transverse deviation of the agricultural equipment according to the first included angle, the first component and the second component; and determining a target angular velocity according to the angular deviation and the transverse deviation, and controlling the agricultural equipment to continuously run according to the target angular velocity.

Description

Technical Field

[0001] This document relates to the field of computer technology, and in particular to a control method, apparatus and device. Background Art

[0002] With the rapid development of navigation and vehicle control technologies, agricultural machinery is gradually moving towards automation and precision. Among these technologies, autonomous navigation and trajectory tracking control are key to achieving agricultural machinery automation.

[0003] When navigating agricultural machinery, the equipment can be positioned using a world coordinate system, and path planning can be performed based on the positioning information. However, this method relies on high-precision satellite positioning. In indoor locations such as greenhouses, satellite navigation signals are easily interfered with by factors such as the ceiling and weather conditions, resulting in poor positioning accuracy and inadequate navigation performance. Therefore, this specification provides a technical solution to improve the navigation performance of indoor agricultural machinery. Summary of the Invention

[0004] The purpose of the embodiments in this specification is to provide a technical solution to improve the navigation effect of indoor agricultural machinery equipment.

[0005] To achieve the above technical solution, the embodiments in this specification are implemented as follows: This specification provides a control method applied to agricultural machinery equipped with a camera. The method includes: controlling the agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in a target area, and acquiring target image data of the agricultural machinery during its travel using the camera; acquiring a first angle between the navigation line and the longitudinal axis of the image coordinate system of the target image data, which is the same as the travel direction of the agricultural machinery; determining an angular deviation of the travel direction of the agricultural machinery based on a first component of the longitudinal axis and a second component of the transverse axis in the moving coordinate system; determining a lateral deviation of the agricultural machinery based on the first angle, the first component, and the second component; determining a target angular velocity based on the angular deviation and the lateral deviation, and controlling the agricultural machinery to continue traveling according to the target angular velocity.

[0006] This specification provides a control device comprising: an image acquisition module for controlling agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in a target area, and acquiring target image data of the agricultural machinery during its travel using a camera device; an angle acquisition module for acquiring a first angle between the navigation line and the longitudinal axis of the image coordinate system of the target image data, which is the same as the travel direction of the agricultural machinery; a first deviation determination module for determining an angular deviation of the agricultural machinery's travel direction based on a first component of the longitudinal axis and a second component of the transverse axis in the moving coordinate system; a second deviation determination module for determining a lateral deviation of the agricultural machinery based on the first angle, the first component, and the second component; and a device control module for determining a target angular velocity based on the angular deviation and the lateral deviation, and controlling the agricultural machinery to continue traveling according to the target angular velocity.

[0007] This specification provides an embodiment of a control device, comprising: a processor; and a memory arranged to store computer-executable instructions, wherein, when executed, the processor: controls agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in a target area, and acquires target image data of the agricultural machinery during its travel using a camera device; acquires a first angle between the navigation line and the longitudinal axis of the image coordinate system of the target image data, which is the same as the travel direction of the agricultural machinery; determines an angular deviation of the travel direction of the agricultural machinery based on a first component of the longitudinal axis and a second component of the transverse axis in the moving coordinate system; determines a lateral deviation of the agricultural machinery based on the first angle, the first component, and the second component; and determines a target angular velocity based on the angular deviation and the lateral deviation, and controls the agricultural machinery to continue traveling according to the target angular velocity.

[0008] This specification also provides a storage medium for storing computer-executable instructions. When executed by a processor, these instructions implement the following process: controlling agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in a target area, and acquiring target image data of the agricultural machinery during its travel using a camera device; acquiring a first angle between the navigation line and the longitudinal axis of the image coordinate system of the target image data, which is the same as the travel direction of the agricultural machinery; determining the angular deviation of the agricultural machinery's travel direction based on a first component of the longitudinal axis and a second component of the transverse axis in the moving coordinate system; determining the lateral deviation of the agricultural machinery based on the first angle, the first component, and the second component; determining a target angular velocity based on the angular deviation and the lateral deviation, and controlling the agricultural machinery to continue traveling according to the target angular velocity.

[0009] This specification also provides a computer program product, including a computer program that, when executed by a processor, implements the following process: controlling agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in a target area, and acquiring target image data of the agricultural machinery during its travel using a camera device; acquiring a first angle between the navigation line and the longitudinal axis of the image coordinate system of the target image data, which is the same as the travel direction of the agricultural machinery; determining the angular deviation of the agricultural machinery's travel direction based on a first component of the longitudinal axis direction and a second component of the transverse axis direction in the moving coordinate system; determining the lateral deviation of the agricultural machinery based on the first angle, the first component, and the second component; determining a target angular velocity based on the angular deviation and the lateral deviation, and controlling the agricultural machinery to continue traveling according to the target angular velocity. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is one embodiment of a control method described in this specification; Figure 2 This is a schematic diagram of image data in this specification; Figure 3 This is a schematic diagram of an image coordinate system as described in this specification; Figure 4This is yet another embodiment of the control method described in this specification; Figure 5 This is a schematic diagram illustrating the dynamic change of a feature point in this specification; Figure 6 This is a schematic diagram of one type of deviation angle in this specification; Figure 7 This is yet another embodiment of the control method described in this specification; Figure 8 This is a schematic diagram of a top view of one of the fields of view in this specification; Figure 9 This is a schematic side view of a camera device for agricultural machinery, as described in this specification. Figure 10 This is a schematic diagram of a navigation line in this specification; Figure 11 This is one embodiment of a control device described in this specification; Figure 12 This is one embodiment of a control device described in this specification. Detailed Implementation

[0011] This specification provides a control method, apparatus, and device through its embodiments.

[0012] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0013] This specification provides a technical solution to improve the navigation effect of indoor agricultural machinery. With the rapid development of navigation and vehicle control technologies, agricultural machinery is gradually moving towards automation and precision. Autonomous navigation and trajectory tracking control are key technologies for achieving agricultural machinery automation. When navigating agricultural machinery, it can be positioned using a world coordinate system, and path planning can be performed based on the positioning information. However, this method relies on high-precision satellite positioning. In indoor locations such as greenhouses, satellite navigation signals are easily interfered with by factors such as the ceiling and weather conditions, resulting in poor positioning accuracy and navigation performance. Therefore, this specification provides a technical solution to improve the navigation effect of indoor agricultural machinery. In this scheme, agricultural machinery can be controlled to travel along a pre-planned path between two adjacent rows of crops arranged longitudinally in a target area. The system uses a camera to acquire target image data of the machinery during its movement. The first angle between the navigation line and the vertical axis (which is the same as the machinery's direction of travel) in the image coordinate system of the target image data is then determined. Based on the first component of the vertical axis and the second component of the horizontal axis in the moving coordinate system, the angular deviation of the machinery's direction of travel is determined. Furthermore, based on the first angle, the first component, and the second component, the lateral deviation of the machinery is determined. Finally, the target angular velocity is determined based on the angular and lateral deviations, and the machinery is controlled to continue traveling according to this target angular velocity. This approach avoids the inaccuracy issues associated with satellite positioning, which is susceptible to indoor environmental influences. In other words, image servo technology enables precise navigation of agricultural machinery in indoor environments. On the other hand, during the path tracking process of agricultural machinery, the target angular velocity of the agricultural machinery can be accurately determined by the angular deviation of the agricultural machinery in the direction of travel and the lateral deviation of the agricultural machinery. The direction of travel of the agricultural machinery can be controlled by the target angular velocity, thereby improving the navigation effect of agricultural machinery in indoor places.

[0014] like Figure 1 As shown in the embodiments of this specification, a control method is provided. The execution subject of this method can be a server, and this method can be applied to agricultural machinery equipped with a camera device. The camera device can be any device capable of acquiring image data, and the agricultural machinery can be any agricultural machinery capable of performing agricultural operations. Specifically, the method may include the following steps: In step S102, the agricultural machinery is controlled to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in the target area, and target image data of the agricultural machinery during the travel process is acquired by the camera device.

[0015] The target area can be any area where agricultural operations need to be carried out. For example, the target area can be an indoor area such as a greenhouse. The preset planning path can be a planning path determined based on the agricultural operations to be carried out by the agricultural machinery and the crop information in the target area (such as the arrangement of crops, the height of crops, etc.).

[0016] In practice, the camera device can be installed at any location on the agricultural machinery that can acquire image data of the agricultural machinery during its operation. For example, the installation location of the camera device on the agricultural machinery can be determined based on the agricultural operation activities to be carried out by the agricultural machinery and the crop information in the target area.

[0017] During the operation of agricultural machinery, camera devices can acquire real-time target image data of the machinery. This target image data can include visual feature information such as the position of crops along the direction of the machinery's movement. For example, in situations like... Figure 2 The target image data shown may contain visual feature information such as the position information of two adjacent rows of crops (i.e., the two rows of crops marked by red dotted lines) in the direction of travel of the agricultural machinery.

[0018] In step S104, the first angle between the navigation line and the vertical axis in the image coordinate system of the target image data that is the same as the driving direction of the agricultural machinery is obtained.

[0019] In implementation, in such Figure 3 The target image data shown can be represented by a blue dashed line consisting of points A and D, and the image coordinate system can be a coordinate system consisting of the origin O, the vertical Y-axis, and the horizontal X-axis. The server can determine the angle between the navigation line and the Y-axis in the image coordinate system as the first angle.

[0020] In step S106, the angular deviation of the agricultural machinery's travel direction is determined based on the first component of the longitudinal axis and the second component of the transverse axis in the moving coordinate system.

[0021] The translation coordinate system can be the Frenet coordinate system; for example, the translation coordinate system can be based on... Figure 3 The coordinate system is formed by the midpoint A as the origin, the direction of the navigation line (i.e., the s-axis direction or the vertical axis direction) and the direction of the navigation line normal (i.e., the d-axis direction or the horizontal axis direction). The s value in the vertical axis direction can refer to the distance of the desired position point in the vertical axis direction, and the d value in the horizontal axis direction can refer to the distance of the desired position point in the horizontal axis direction, which is the distance of the desired position point from the navigation line.

[0022] In practice, the server can obtain the first component (such as the s value) of the agricultural machinery in the vertical axis direction and the second component (such as the d value) in the horizontal axis direction of the moving coordinate system, and determine the angular deviation of the agricultural machinery in the direction of travel (i.e., the angular deviation between the target point on the navigation line and the direction of travel of the agricultural machinery) based on the obtained s value and d value.

[0023] For example, the angle deviation can be ,in, , The first included angle, It can be The target value, in the image coordinate system ,in, The derivative of the second component d. The derivative of the first component s, i.e., the angular deviation. .

[0024] In step S108, the lateral deviation of the agricultural machinery is determined based on the first included angle, the first component, and the second component.

[0025] In implementation, the lateral deviation can be... , , This can be the intersection of the navigation line and the bottom of the target image data (i.e., the side closest to the agricultural machinery), such as... Figure 3 point shown The horizontal coordinate in the image coordinate system. It can be the x-coordinate of the desired location point in the image coordinate system, determined according to a preset planned path. Therefore, in the image coordinate system... .

[0026] That is, lateral deviation .

[0027] In step S110, the target angular velocity is determined based on the angular deviation and lateral deviation, and the agricultural machinery is controlled to continue traveling according to the target angular velocity.

[0028] In practice, the server can correct the angular deviation of the agricultural machinery in the direction of travel based on the angular deviation. At the same time, it can also correct the lateral deviation of the agricultural machinery in the direction of travel based on the lateral deviation, so as to obtain the target angular velocity at the next moment and control the agricultural machinery to continue to travel according to the target angular velocity.

[0029] This specification provides a control method that controls agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in a target area. The method involves acquiring target image data of the machinery during its movement using a camera device, determining the first angle between the navigation line and the longitudinal axis (which is the same as the machinery's direction of travel) in the image coordinate system of the target image data, and then determining the angular deviation of the machinery's direction of travel based on the first component of the longitudinal axis and the second component of the lateral axis in the movement coordinate system. Furthermore, the lateral deviation of the machinery is determined based on the first angle, the first component, and the second component. Finally, the target angular velocity is determined based on the angular and lateral deviations, and the machinery is controlled to continue traveling according to this target angular velocity. This approach avoids the inaccuracy issues associated with satellite positioning, which is susceptible to indoor environmental influences. In other words, image servo technology enables precise navigation of agricultural machinery in indoor environments. On the other hand, during the path tracking process of agricultural machinery, the target angular velocity of the agricultural machinery can be accurately determined by the angular deviation of the agricultural machinery in the direction of travel and the lateral deviation of the agricultural machinery. The direction of travel of the agricultural machinery can be controlled by the target angular velocity, thereby improving the navigation effect of agricultural machinery in indoor places.

[0030] In practical applications, the specific processing method for determining the target angular velocity based on angular deviation and lateral deviation in step S110 above can vary. One optional processing method is provided below, such as... Figure 4 As shown, the specific process may include the following steps S1102 to S1104.

[0031] In step S1102, based on the agricultural machinery's travel speed, the first correlation relationship, and the first vector, a compensation term is determined to compensate for the deviation component of the agricultural machinery on the horizontal axis of the image coordinate system of the target image data.

[0032] Among them, the first correlation relationship can be the relationship between the visual features of the target image data and the speed of the camera device, and the first vector can be the first column vector of the homogeneous transformation vector of the driving speed of the agricultural machinery and the speed of the camera device.

[0033] In implementation, image servoing technology can be used for modeling. =

[0034] in, It is the x-coordinate of the tracked point in the image coordinate system. These are the travel speed and yaw rate of the agricultural machinery. This can be used to determine the relationship between visual features and the speed of the camera device, i.e. It can be the first relevant relation:

[0035] in, As the first relevant relation, The focal length of the camera device, The tilt angle of the camera device. The angle between the navigation line and the vertical axis of the image coordinate system. Let (X, Y) be the height of the camera device, and (X, Y) be the point on the image coordinate system where the tracking point is located. .

[0036] Since the speed of agricultural machinery can be determined based on the linear velocity of the rear wheel center, and the position of the camera device is different from that of the agricultural machinery, the velocity components of the agricultural machinery and the camera device are different. Therefore, the server can construct a homogeneous transformation vector of the speed of agricultural machinery and the speed of camera device to represent the conversion relationship between the speed of agricultural machinery and the speed of camera device.

[0037]

[0038] in, Let be the homogeneous transformation vector between the traveling speed of the agricultural machinery and the speed of the camera device. The horizontal distance between the camera device and the rear wheels of the agricultural machinery.

[0039] like Figure 5 As shown, when the agricultural machinery moves forward, point P will be oriented towards the vector. The direction of movement, while the vector In the image coordinate system The axis has a component, corresponding to , It is a Jacobian matrix The first element, namely ,in, , yes The first column vector (i.e., the first vector).

[0040] The compensation term corresponds to factors not controlled by visual servo (such as vehicle speed). The motion of the target point generated by this motion, in order to counteract this motion, It can be determined by driving speed Points generated The negative value of the motion, that is, the server can determine the compensation term used to compensate for the deviation component of the agricultural machinery on the horizontal axis of the image coordinate system of the target image data based on the traveling speed of the agricultural machinery, the first correlation, and the first vector, namely: .

[0041] In step S1104, the deviation in the preset state equation is offset by a compensation term to obtain the offset state equation. Based on the offset state equation, the angular deviation, and the lateral deviation, the target angular velocity is determined.

[0042] In practical applications, the specific processing method for determining the target angular velocity in step S1104 above based on the canceled state equation, angular deviation, and lateral deviation can be varied. The following provides another optional processing method, which may include the following steps A1 to A4.

[0043] In step A1, the first matrix is ​​determined based on the first correlation and the second vector.

[0044] The second vector can be the second column vector of the homogeneous transformation vector of the agricultural machinery's speed and the camera's speed.

[0045] In implementation, the server can determine the first matrix based on the first correlation and the second vector: , in, For the first matrix, yes The second column vector (i.e., the second vector).

[0046] In step A2, the target angle is determined based on the angle deviation and the first included angle.

[0047] In practice, the server can determine the target angle as the difference between the angle deviation and the first included angle. , in, From the target perspective.

[0048] In step A3, the first deviation is determined based on the target x-coordinate and lateral deviation.

[0049] The target x-coordinate can be the x-coordinate of the end of the first center line between two adjacent rows of crops in the target image data that is closer to the shooting device, in the image coordinate system.

[0050] In implementation, such as Figure 6 As shown, the first deviation The desired location point can be defined as its x-coordinate in the image coordinate system. The server can determine the difference between the x-coordinate and the lateral deviation as the first deviation. .

[0051] In step A4, the target angular velocity is determined based on the canceled state equation, the first matrix, the angular deviation, the lateral deviation, the target angle, the first deviation, and the positive gain diagonal matrix.

[0052] The positive gain diagonal matrix can be constructed from preset parameters corresponding to the angular deviation and preset parameters corresponding to the lateral deviation.

[0053] In practice, image-based servo control technology tracks fixed points in the world coordinate system, while... Figure 5 As shown, agricultural machinery moving forward in a straight line will generate points. In vector The state moves upward. Therefore, a compensation term can be added to the state equation, meaning the preset state equation can be:

[0054] After the preset state equation is canceled out by the compensation term, the first matrix, angular deviation, lateral deviation, target angle, first deviation, and positive gain diagonal matrix can be substituted into the canceled state equation. , The target angular velocity is obtained, where, For the target angular velocity, The Moore-Penrose pseudo-inverse of the first matrix is... It is a positive gain diagonal matrix. , For lateral deviation, For angular deviation, The derivative of the first deviation, The derivative of the target angle.

[0055] In practical applications, before step S102 above, a preset planned path can be determined. The specific processing method for determining the preset planned path can vary. The following provides another optional processing method, such as... Figure 7 As shown, the specific process may include the following steps S702 to S714.

[0056] In step S702, initial image data for two adjacent rows of crops is acquired using a camera device.

[0057] In practice, the server can control the camera device to acquire initial image data for two adjacent rows of crops at the initial moment. For example, the acquired initial image data can be image data as shown in Figure 2.

[0058] In step S704, based on the initial image data, the angular deviation between the agricultural machinery and the navigation line, and the spacing information between two adjacent rows of crops are determined.

[0059] In practical applications, there are various ways to determine the angle deviation between the agricultural machinery and the navigation line based on the initial image data in step S704 above. Here is another optional processing method, which may include the following steps B1 to B4.

[0060] In step B1, based on the distance between the first endpoint and the second endpoint in the initial image data, the first depth information, and the focal length of the camera device, the first distance between the point corresponding to the first endpoint in the target area and the point corresponding to the second endpoint in the target area is determined.

[0061] In step B2, based on the distance between the third endpoint and the fourth endpoint in the initial image data, the second depth information, and the focal length of the camera device, the second distance between the point corresponding to the third endpoint in the target area and the point corresponding to the fourth endpoint in the target area is determined.

[0062] The first centerline can be the centerline of the initial image data, the second centerline can be the centerline between two adjacent rows of crops in the initial image data, the first endpoint and the second endpoint have the same vertical axis component in the image coordinate system, and the third endpoint and the fourth endpoint have the same vertical axis component in the image coordinate system.

[0063] In implementation, the initial image data is used as follows: Figure 3 Taking the image data shown as an example, the first median line can be the line segment between point B and point C, the second median line can be the line segment between point A and point D, the first endpoint can be point B, the second endpoint can be point A, the third endpoint can be point C, and the fourth endpoint can be point D.

[0064] The server can determine the distance between point A and point B in the image coordinate system. Mapped to, for example Figure 8 The corresponding point in the top view of the field of view of the camera device shown Arrive The distance between ,Right now ,in, The first distance, This is the depth information (i.e., the first depth information) of any point between point A and point B in the initial image data.

[0065] Similarly, the server can determine the distance between point C and point D in the image coordinate system. Mapped to, for example Figure 8 The corresponding point in the top view of the field of view of the camera device shown Arrive The distance between ,Right now ,in, The second distance, This is the depth information (i.e., the second depth information) of any point between point C and point D in the initial image data.

[0066] In step B3, the fifth distance of the first centerline is determined based on the third and fourth distances.

[0067] The third distance can be the distance between the first intersection point and the front axis. The first intersection point can be the intersection of the straight line corresponding to the ground below the viewfinder of the camera device and the ground. The fourth distance can be the distance between the second intersection point and the front axis. The second intersection point can be the intersection of the straight line corresponding to the ground above the viewfinder of the camera device and the ground.

[0068] In implementation, such as Figure 9 As shown, the camera device can be located perpendicular to the front wheel of the agricultural machinery, and the third distance can be... Figure 9 In The fourth distance can be found in the graph. The fifth distance from the first center line can be as follows: Figure 8 The corresponding point in the top view of the field of view of the camera device shown Arrive The distance between The server can determine the fifth distance as the difference between the fourth distance and the third distance. .

[0069] In step B4, the angular deviation between the agricultural machinery and the navigation line is determined based on the first distance, the second distance, and the fifth distance.

[0070] In practice, the angular deviation between the agricultural machinery and the navigation line can be: , in, This refers to the angular deviation between the agricultural machinery and the navigation line.

[0071] In practical applications, there are various ways to determine the spacing information between two adjacent rows of crops based on the initial image data in step S704 above. Here is another optional processing method, which may include the following steps C1 to C2.

[0072] In step C1, the seventh distance between two adjacent rows of crops in the target area is determined based on the sixth distance between two adjacent rows of crops in the initial image data, the first depth information, and the focal length of the camera device.

[0073] In implementation, the initial image data is used as follows: Figure 3 Taking the image data shown as an example, the sixth distance is taken as the point. and points The distance between them.

[0074] Based on the transformation relationship between the coordinate system of the camera device and the image coordinate system, it can be... Figure 3 Midpoint of the image coordinate system , The distance between The distance between corresponding points mapped onto the ground, i.e., the seventh distance, is:

[0075] in, This is the seventh distance.

[0076] In step C2, the spacing information between two adjacent rows of crops is determined based on the first depth information, the second depth information, the fifth distance, the first distance, the second distance, the seventh distance, and the focal length of the camera device.

[0077] In practice, the first depth information, second depth information, fifth distance, first distance, second distance, seventh distance, and the focal length of the camera device can be substituted into the following formula. , Obtain the spacing information between two adjacent rows of crops, where D is the spacing information between two adjacent rows of crops.

[0078] In step S706, the first depth information of the line segment between the first endpoint of the first median line and the second endpoint of the second median line in the initial image data, and the second depth information of the line segment between the third endpoint of the first median line and the fourth endpoint of the second median line are obtained.

[0079] The first centerline can be the centerline of the initial image data, the second centerline can be the centerline between two adjacent rows of crops in the initial image data, the first endpoint and the second endpoint have the same vertical axis component in the image coordinate system, and the third endpoint and the fourth endpoint have the same vertical axis component in the image coordinate system.

[0080] In implementation, the first depth information can be determined based on the height of the camera device, the longitudinal field of view of the camera device, and a third distance, and the second depth information can be determined based on the height of the camera device, the longitudinal field of view of the camera device, and a fourth distance. , ,in, This refers to the longitudinal field of view of the camera device.

[0081] In step S708, the lateral travel range of the agricultural machinery in the target area is determined based on the angular deviation between the agricultural machinery and the navigation line, the spacing information, the focal length of the camera device, the wheelbase of the front or rear wheels of the agricultural machinery, the first depth information, and the second depth information.

[0082] In implementation, for lateral movement, the left and right sampling boundary lines can define the drivable area of ​​agricultural machinery, that is, to ensure that the agricultural machinery can move freely laterally as much as possible while ensuring that the wheels do not run over crops. The maximum value of lateral sampling can be determined based on the spacing information and the wheelbase of the front or rear wheels of the agricultural machinery.

[0083] in, This can be the maximum value of the horizontal sampling. It can be the wheelbase of the front or rear wheels of agricultural machinery.

[0084] Furthermore, based on the maximum value of the lateral sampling, the angular deviation between the agricultural machinery and the navigation line, and the first and second depth information, the lateral travel range of the agricultural machinery in the target area can be determined. The lateral travel range can be, for example, as follows: Figure 10 point shown ,point ,point and points The scope of the composition.

[0085] Among them, the maximum range of horizontal sampling ( The mapping is within the top and bottom borders of the image. 2

[0086] 2

[0087] in, Midpoint of the initial image data ,point The distance between them Midpoint of the initial image data and points The distance between them.

[0088] In step S710, the longitudinal planning of the agricultural machinery is determined based on the preset longitudinal speed and preset longitudinal acceleration at the initial moment and the preset longitudinal speed and preset longitudinal acceleration at the final moment.

[0089] In implementation, the server can use the navigation line as the baseline for path planning, i.e. Figure 10 Horizontal distance of all points on the blue navigation line Longitudinal distance at point A .

[0090] For longitudinal trajectory planning, if only the constant-speed cruise condition is considered, the planning of the target's longitudinal position can be ignored, and only the longitudinal velocity and acceleration of the agricultural machinery need to be planned. Therefore, the known conditions for longitudinal planning can be the initial time. longitudinal velocity The derivative of longitudinal velocity and longitudinal acceleration Termination time The derivative of the longitudinal velocity and longitudinal acceleration .

[0091] Since the five known conditions allow us to solve for the five parameters of the quartic polynomial, the longitudinal programming here can be described by a quartic polynomial with respect to time t: , in, ( i = 0, 1, 2, 3, 4) can be the coefficients of the longitudinal programming polynomial. Substituting the above five known conditions will yield the coefficient values.

[0092] In step S712, the lateral planning of the agricultural machinery is determined based on the longitudinal planning and lateral travel range of the agricultural machinery.

[0093] In practice, when agricultural machinery is traveling at low speeds, it is necessary to plan the combination of lateral and longitudinal movements. Longitudinal sampling was obtained based on time sampling. In horizontal planning, it can be based on get Because lateral sampling needs to cover the entire sampling boundary (i.e., the lateral driving range), the planning of the target's lateral position cannot be ignored. Therefore, the known conditions for lateral planning can be the initial time step. Horizontal position The derivative of the transverse velocity and lateral acceleration Termination time Horizontal position The derivative of the transverse velocity and lateral acceleration .

[0094] Since the six known conditions can be used to solve for the six parameters of a fifth-degree polynomial, the lateral programming here can be used with respect to the vertical distance. sA fifth-degree polynomial description: (Equation 13) in, ( i = 0, 1, 2, 3, 4, 5) can be the coefficients of the horizontal programming polynomial. Substituting the above six known conditions will yield the coefficient values.

[0095] In step S714, a preset planning path is determined based on the longitudinal and lateral planning of the agricultural machinery.

[0096] In implementation, the server samples the initial and final states of the agricultural machinery, obtaining a series of lateral trajectory sets and a series of longitudinal trajectory sets. The server can then match the lateral and longitudinal trajectories at each time point within these sets, and perform piecewise fitting using cubic spline curves to obtain a complete planned trajectory bundle. Finally, by setting a heuristic cost function, an optimal trajectory is determined; this trajectory is the preset planned path and can serve as a reference trajectory for subsequent path tracking.

[0097] In this way, by planning the path in the image coordinate system, the planned path can be directly applied to image servo control, and longitudinal control of agricultural machinery can be added to the existing image servo control.

[0098] In summary, this agricultural machinery navigation method, which eliminates the need for explicit positioning, overcomes the difficulty of obtaining accurate global map positioning in indoor environments such as greenhouses, providing a practical new approach for achieving intelligent and autonomous operation of agricultural machinery. This method has the following advantages: (1) Apply image servoing technology to path tracking, and add compensation terms to take into account the dynamic changes of the target point to achieve a more accurate image servoing tracking method.

[0099] (2) The width of the double-column crop row within the camera's field of view is directly calculated from the image features. Then, taking into account the horizontal sampling range within the image, curve path fitting is directly performed in the image.

[0100] (3) Combine path planning and image servo tracking control to add the longitudinal control variable that is missing in traditional image servo control.

[0101] This specification provides a control method that controls agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in a target area. The method involves acquiring target image data of the machinery during its movement using a camera device, determining the first angle between the navigation line and the longitudinal axis (which is the same as the machinery's direction of travel) in the image coordinate system of the target image data, and then determining the angular deviation of the machinery's direction of travel based on the first component of the longitudinal axis and the second component of the lateral axis in the movement coordinate system. Furthermore, the lateral deviation of the machinery is determined based on the first angle, the first component, and the second component. Finally, the target angular velocity is determined based on the angular and lateral deviations, and the machinery is controlled to continue traveling according to this target angular velocity. This approach avoids the inaccuracy issues associated with satellite positioning, which is susceptible to indoor environmental influences. In other words, image servo technology enables precise navigation of agricultural machinery in indoor environments. On the other hand, during the path tracking process of agricultural machinery, the target angular velocity of the agricultural machinery can be accurately determined by the angular deviation of the agricultural machinery in the direction of travel and the lateral deviation of the agricultural machinery. The direction of travel of the agricultural machinery can be controlled by the target angular velocity, thereby improving the navigation effect of agricultural machinery in indoor places.

[0102] The above describes the control method provided in the embodiments of this specification. Based on the same idea, the embodiments of this specification also provide a control device, such as... Figure 11 As shown.

[0103] The control device includes: an image acquisition module 1101, an angle acquisition module 1102, a first deviation determination module 1103, a second deviation determination module 1104, and a device control module 1105, wherein: The image acquisition module 1101 is used to control the agricultural machinery to travel between two adjacent rows of crops arranged longitudinally in the target area according to a preset planned path, and to acquire target image data of the agricultural machinery during the travel process according to the camera device. Angle acquisition module 1102 is used to acquire the first angle between the navigation line and the longitudinal axis in the image coordinate system of the target image data that is the same as the driving direction of the agricultural machinery in the same way as the travel direction of the agricultural machinery in the target image data. The first deviation determination module 1103 is used to determine the angle deviation of the agricultural machinery's travel direction based on the first component of the longitudinal axis and the second component of the transverse axis in the moving coordinate system. The second deviation determination module 1104 is used to determine the lateral deviation of the agricultural machinery based on the first included angle, the first component, and the second component. The equipment control module 1105 is used to determine the target angular velocity based on the angular deviation and the lateral deviation, and control the agricultural machinery to continue traveling according to the target angular velocity.

[0104] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the control device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0105] This specification provides a control device that can control agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in a target area. It acquires target image data of the agricultural machinery during its movement using a camera device, and obtains the first angle between the navigation line and the longitudinal axis (which is the same as the direction of travel) in the image coordinate system of the target image data. Based on the first component of the longitudinal axis and the second component of the transverse axis in the moving coordinate system, it determines the angular deviation of the agricultural machinery's travel direction. Then, based on the first angle, the first component, and the second component, it determines the lateral deviation of the agricultural machinery. Finally, based on the angular and lateral deviations, it determines the target angular velocity and controls the agricultural machinery to continue traveling according to the target angular velocity. In this way, the target angular velocity can be determined solely from the target image data acquired by the camera device, and the agricultural machinery can be controlled to continue traveling according to the target angular velocity. This avoids the problem of poor positioning accuracy caused by indoor environmental factors when using satellite or other positioning methods. In other words, image servo technology can achieve precise navigation of agricultural machinery in indoor environments. On the other hand, during the path tracking process of agricultural machinery, the target angular velocity of the agricultural machinery can be accurately determined by the angular deviation of the agricultural machinery in the direction of travel and the lateral deviation of the agricultural machinery. The direction of travel of the agricultural machinery can be controlled by the target angular velocity, thereby improving the navigation effect of agricultural machinery in indoor places.

[0106] The above are the control devices provided in the embodiments of this specification. Based on the same idea, the embodiments of this specification also provide a control device, such as... Figure 12 As shown.

[0107] The control device can be a terminal device or a server, as described in the above embodiments.

[0108] The control device can vary considerably depending on its configuration or performance, and may include one or more processors 1201 and memory 1202. The memory 1202 may store one or more application programs or data. The memory 1202 may be temporary or persistent storage. The application programs stored in the memory 1202 may include one or more modules (not shown in the figures), each module may include a series of computer-executable instructions for the control device. Furthermore, the processor 1201 may be configured to communicate with the memory 1202 and execute the series of computer-executable instructions stored in the memory 1202 on the control device. The control device may also include one or more power supplies 1203, one or more wired or wireless network interfaces 1204, one or more input / output interfaces 1205, and one or more keyboards 1206.

[0109] Specifically, in this embodiment, the control device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the control device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following: The agricultural machinery is controlled to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in the target area, and the target image data of the agricultural machinery during the travel process is acquired by the camera device. In the target image data, obtain the first angle between the navigation line and the vertical axis in the image coordinate system of the target image data, which is the same as the travel direction of the agricultural machinery. The angular deviation of the agricultural machinery's travel direction is determined based on the first component of the longitudinal axis and the second component of the transverse axis in the moving coordinate system. The lateral deviation of the agricultural machinery is determined based on the first included angle, the first component, and the second component. Based on the angular deviation and the lateral deviation, the target angular velocity is determined, and the agricultural machinery is controlled to continue traveling according to the target angular velocity.

[0110] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the control device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0111] This specification provides a control device that can control agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in a target area. It acquires target image data of the agricultural machinery during its movement using a camera device, and obtains the first angle between the navigation line and the longitudinal axis (which is the same as the direction of travel) in the image coordinate system of the target image data. Based on the first component of the longitudinal axis and the second component of the transverse axis in the moving coordinate system, it determines the angular deviation of the agricultural machinery's travel direction. Then, based on the first angle, the first component, and the second component, it determines the lateral deviation of the agricultural machinery. Finally, based on the angular and lateral deviations, it determines the target angular velocity and controls the agricultural machinery to continue traveling according to the target angular velocity. In this way, the target angular velocity can be determined solely from the target image data acquired by the camera device, and the agricultural machinery can be controlled to continue traveling according to the target angular velocity. This avoids the problem of poor positioning accuracy caused by indoor environmental factors when using satellite or other positioning methods. In other words, image servo technology can achieve precise navigation of agricultural machinery in indoor environments. On the other hand, during the path tracking process of agricultural machinery, the target angular velocity of the agricultural machinery can be accurately determined by the angular deviation of the agricultural machinery in the direction of travel and the lateral deviation of the agricultural machinery. The direction of travel of the agricultural machinery can be controlled by the target angular velocity, thereby improving the navigation effect of agricultural machinery in indoor places.

[0112] Furthermore, based on the above Figures 1 to 10 The method shown in this specification, along with one or more embodiments, also provides a storage medium for storing computer-executable instruction information. In one specific embodiment, the storage medium can be a USB flash drive, optical disc, hard disk, etc. When the computer-executable instruction information stored in the storage medium is executed by a processor, it can achieve the following process: The agricultural machinery is controlled to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in the target area, and the target image data of the agricultural machinery during the travel process is acquired by the camera device. In the target image data, obtain the first angle between the navigation line and the vertical axis in the image coordinate system of the target image data, which is the same as the travel direction of the agricultural machinery. The angular deviation of the agricultural machinery's travel direction is determined based on the first component of the longitudinal axis and the second component of the transverse axis in the moving coordinate system. The lateral deviation of the agricultural machinery is determined based on the first included angle, the first component, and the second component. Based on the angular deviation and the lateral deviation, the target angular velocity is determined, and the agricultural machinery is controlled to continue traveling according to the target angular velocity.

[0113] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the above-described storage medium embodiment is basically similar to the method embodiment, so the description is relatively simple; relevant parts can be referred to the description of the method embodiment.

[0114] This specification provides a storage medium that can control agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in a target area. It acquires target image data of the agricultural machinery during its movement using a camera device. The system obtains the first angle between the navigation line and the longitudinal axis (which is the same as the direction of travel) in the image coordinate system of the target image data. Based on the first component of the longitudinal axis and the second component of the transverse axis in the moving coordinate system, the angular deviation of the agricultural machinery's travel direction is determined. Then, based on the first angle, the first component, and the second component, the lateral deviation of the agricultural machinery is determined. Finally, based on the angular and lateral deviations, the target angular velocity is determined, and the agricultural machinery is controlled to continue traveling according to the target angular velocity. In this way, the target angular velocity can be determined solely from the target image data acquired by the camera device, and the agricultural machinery can be controlled to continue traveling according to the target angular velocity. This avoids the problem of poor positioning accuracy caused by indoor environmental factors when using satellite or other positioning methods. In other words, image servo technology can achieve precise navigation of agricultural machinery in indoor environments. On the other hand, during the path tracking process of agricultural machinery, the target angular velocity of the agricultural machinery can be accurately determined by the angular deviation of the agricultural machinery in the direction of travel and the lateral deviation of the agricultural machinery. The direction of travel of the agricultural machinery can be controlled by the target angular velocity, thereby improving the navigation effect of agricultural machinery in indoor places.

[0115] Furthermore, based on the above Figures 1 to 10 The method shown in this specification, along with one or more embodiments, also provides a computer program product including a computer program that, when executed by a processor, performs the following process: The agricultural machinery is controlled to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in the target area, and the target image data of the agricultural machinery during the travel process is acquired by the camera device. In the target image data, obtain the first angle between the navigation line and the vertical axis in the image coordinate system of the target image data, which is the same as the travel direction of the agricultural machinery. The angular deviation of the agricultural machinery's travel direction is determined based on the first component of the longitudinal axis and the second component of the transverse axis in the moving coordinate system. The lateral deviation of the agricultural machinery is determined based on the first included angle, the first component, and the second component. Based on the angular deviation and the lateral deviation, the target angular velocity is determined, and the agricultural machinery is controlled to continue traveling according to the target angular velocity.

[0116] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the above-described embodiment of a computer program product is relatively simple in description because it is fundamentally similar to the method embodiment; relevant parts can be referred to the description of the method embodiment.

[0117] This specification provides a computer program product that can control agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in a target area. It acquires target image data of the agricultural machinery during its movement using a camera device, and obtains the first angle between the navigation line and the longitudinal axis (which is the same as the direction of travel) in the image coordinate system of the target image data. Based on the first component of the longitudinal axis and the second component of the transverse axis in the moving coordinate system, it determines the angular deviation of the agricultural machinery's travel direction. Then, based on the first angle, the first component, and the second component, it determines the lateral deviation of the agricultural machinery. Finally, based on the angular and lateral deviations, it determines the target angular velocity and controls the agricultural machinery to continue traveling according to the target angular velocity. In this way, the target angular velocity can be determined solely from the target image data acquired by the camera device, and the agricultural machinery can be controlled to continue traveling according to the target angular velocity. This avoids the problem of poor positioning accuracy caused by indoor environmental factors when using satellite or other positioning methods. In other words, image servo technology can achieve precise navigation of agricultural machinery in indoor environments. On the other hand, during the path tracking process of agricultural machinery, the target angular velocity of the agricultural machinery can be accurately determined by the angular deviation of the agricultural machinery in the direction of travel and the lateral deviation of the agricultural machinery. The direction of travel of the agricultural machinery can be controlled by the target angular velocity, thereby improving the navigation effect of agricultural machinery in indoor places.

[0118] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0119] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0120] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0121] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0122] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0123] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] Embodiments in this specification are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable parallel device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable parallel device, generate instructions for implementing the flowchart illustrations. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable fraud device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0127] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0128] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0129] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0130] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0131] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0133] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0134] The above description is merely an embodiment of this specification and is not intended to limit this document. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A control method, the method being applied to agricultural machinery equipped with a camera device, the method comprising: Controlling the agricultural machinery to travel between two adjacent rows of crops arranged vertically in a target area according to a preset planned path, and obtaining target image data of the agricultural machinery during the travel process according to the camera device; Acquire a first angle between a navigation line in the target image data and a longitudinal axis in an image coordinate system of the target image data which is the same as the driving direction of the agricultural machinery; Determining an angular deviation of a driving direction of the agricultural machinery according to a first component of a longitudinal axis direction and a second component of a transverse axis direction of the agricultural machinery in a mobile coordinate system; determining a lateral deviation of the agricultural machinery according to the first angle, the first component and the second component; A target angular velocity is determined according to the angular deviation and the lateral deviation, and the agricultural machinery is controlled to continue traveling according to the target angular velocity.

2. The method according to claim 1, wherein determining the target angular velocity according to the angular deviation and the lateral deviation comprises: Determining a compensation item for compensating a deviation component of the agricultural machinery on the horizontal axis of the image coordinate system of the target image data according to the driving speed of the agricultural machinery, a first correlation and a first vector, wherein the first correlation is a relationship between a visual feature of the target image data and a speed of the camera device, and the first vector is a first column vector of a homogeneous transformation vector of the driving speed of the agricultural machinery and the speed of the camera device; The compensation term is used to offset the deviation in the preset state equation to obtain the offset state equation, and the target angular velocity is determined based on the offset state equation, the angle deviation and the lateral deviation.

3. The method according to claim 2, wherein determining the target angular velocity according to the offset state equation, the angle deviation and the lateral deviation comprises: Determine a first matrix according to the first correlation and the second vector, wherein the second vector is a second column vector of a homogeneous transformation vector of a travel speed of the agricultural machinery and a speed of the camera device; Determining a target angle according to the angle deviation and the first angle; determining a first deviation according to a target horizontal coordinate and the lateral deviation, wherein the target horizontal coordinate is a horizontal coordinate of an end of a first center line between two adjacent rows of crops in the target image data, close to the photographing device, on the image coordinate system; The target angular velocity is determined based on the offset state equation, the first matrix, the angle deviation, the lateral deviation, the target angle, the first deviation and a positive gain diagonal matrix, wherein the positive gain diagonal matrix is ​​constructed by preset parameters corresponding to the angle deviation and preset parameters corresponding to the lateral deviation.

4. The method according to claim 3, wherein determining the target angular velocity according to the offset state equation, the first matrix, the angle deviation, the lateral deviation, the target angle, the first deviation and a positive gain diagonal matrix comprises: Substituting the first matrix, the angle deviation, the lateral deviation, the target angle, the first deviation and the positive gain diagonal matrix into the state equation after the offset, ， The target angular velocity is obtained, where is the target angular velocity, is the pseudo-inverse matrix of the first matrix, is the positive gain diagonal matrix, , is the lateral deviation, is the angle deviation, is the derivative of the first deviation, is the derivative of the target angle.

5. The method according to claim 1, before controlling the agricultural machinery to travel along a preset planned path between two adjacent rows of crops arranged longitudinally in the target area, further comprising: Acquiring initial image data for the two adjacent rows of crops by means of the camera device; Determining, based on the initial image data, an angle deviation between the agricultural machinery and the navigation line, and spacing information between two adjacent rows of crops; Acquire first depth information of a line segment between a first endpoint of a first center line and a second endpoint of a second center line in the initial image data, and second depth information of a line segment between a third endpoint of the first center line and a fourth endpoint of the second center line, wherein the first center line is the center line of the initial image data, the second center line is the center line between two adjacent rows of crops in the initial image data, the first endpoint and the second endpoint have the same component in the longitudinal direction of the image coordinate system, and the third endpoint and the fourth endpoint have the same component in the longitudinal direction of the image coordinate system; Determine the lateral driving range of the agricultural machinery in the target area according to the angle deviation between the agricultural machinery and the navigation line, the spacing information, the focal length of the camera device, the wheelbase of the front wheels or the rear wheels of the agricultural machinery, the first depth information and the second depth information; Determining the longitudinal plan of the agricultural machinery according to the preset longitudinal speed and the preset longitudinal acceleration of the agricultural machinery at the initial moment and the preset longitudinal speed and the preset longitudinal acceleration at the end moment; Determining the lateral planning of the agricultural machinery equipment according to the longitudinal planning of the agricultural machinery equipment and the lateral driving range; The preset planning path is determined according to the longitudinal planning and the lateral planning of the agricultural machinery.

6. The method according to claim 5, wherein determining the angle deviation between the agricultural machinery and the navigation line according to the initial image data comprises: Determine, according to the distance between the first endpoint and the second endpoint in the initial image data, the first depth information, and the focal length of the camera device, a first distance between a point corresponding to the first endpoint in the target area and a point corresponding to the second endpoint in the target area; Determine, according to the distance between the third endpoint and the fourth endpoint in the initial image data, the second depth information, and the focal length of the camera device, a second distance between a point corresponding to the third endpoint in the target area and a point corresponding to the fourth endpoint in the target area; Determine a fifth distance of the first center line according to a third distance and a fourth distance, wherein the third distance is a distance between a first intersection and the front axis, the first intersection is an intersection of a straight line below the viewfinder of the camera device corresponding to the ground and the ground, the fourth distance is a distance between a second intersection and the front axis, and the second intersection is an intersection of a straight line above the viewfinder of the camera device corresponding to the ground and the ground; An angular deviation between the agricultural machinery and the navigation line is determined according to the first distance, the second distance and the fifth distance.

7. The method according to claim 6, wherein determining the spacing information between the two adjacent rows of crops according to the initial image data comprises: determining a seventh distance between the two adjacent rows of crops in the target area according to the sixth distance between the two adjacent rows of crops in the initial image data, the first depth information, and the focal length of the camera device; The spacing information between the two adjacent rows of crops is determined according to the first depth information, the second depth information, the fifth distance, the first distance, the second distance, the seventh distance, and the focal length of the camera device.

8. A control device, comprising: An image acquisition module is used to control the agricultural machinery to travel between two adjacent rows of crops arranged vertically in the target area according to a preset planned path, and to obtain target image data of the agricultural machinery during the driving process according to a camera device; An angle acquisition module, used for acquiring a first angle between a navigation line in the target image data and a longitudinal axis in an image coordinate system of the target image data which is the same as the driving direction of the agricultural machinery; A first deviation determination module, used to determine the angular deviation of the driving direction of the agricultural machinery according to a first component of the longitudinal axis direction and a second component of the transverse axis direction of the agricultural machinery in a mobile coordinate system; a second deviation determination module, configured to determine a lateral deviation of the agricultural machinery according to the first angle, the first component and the second component; The equipment control module is used to determine a target angular velocity according to the angular deviation and the lateral deviation, and control the agricultural machinery to continue traveling according to the target angular velocity.

9. A control device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the control method according to any one of claims 1 to 7 when executed by the processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the control method according to any one of claims 1 to 7 when being executed by a processor.

Images