A human-machine integrated intelligent agricultural machinery automatic driving formation transfer system and method

Abstract

The present invention discloses a human-machine integrated intelligent agricultural machinery automatic driving formation transfer system and method. The system is installed on a pilot host and several follower slaves. The pilot host and each follower slave are equipped with the same driving status collection system. The pilot host is also provided with a formation control terminal system, a router main station and a host WiFi module connected in sequence. Each follower slave is provided with a slave WiFi module that wirelessly communicates with the host WiFi module. All collected information is transmitted to the formation control terminal system via the router main station. The on-board computer in the formation control terminal system obtains the machine adjustment decision of the follower slave. The pilot host operator transmits the human decision to the on-board computer through a human-machine interaction interface. The on-board computer integrates the human decision and the machine adjustment decision to generate a formation decision and a control instruction for the control quantity input. The present invention realizes the complementary advantages between human experience and machine decision, and improves the reliability of the multi-machine formation transfer system.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent agricultural machinery, and specifically relates to a technology for automatic driving formation and transfer of intelligent agricultural machinery, which realizes formation and transfer of agricultural machinery. Background Art

[0002] Currently, smart agriculture, which integrates information technology with agricultural production, has effectively improved operational efficiency and planting benefits and is a key development direction for agricultural modernization. However, the advancement of smart agriculture requires centralized, large-scale, and industrialized implementation, and the traditional operator-driven agricultural machinery model is no longer adaptable. Numerous application scenarios have demonstrated that the "collaborative and team-based" operation of multiple machines can significantly improve operational efficiency and quality, effectively reducing operator workload. Applying team-based collaborative operations to agricultural machinery operations such as tillage, sowing, spraying, fertilizing, and harvesting is of great significance for improving the efficiency and quality of agricultural machinery operations and promoting the development of smart agriculture.

[0003] For agricultural machinery that works in formation and in collaboration, there is a Chinese patent publication number CN110286676A, entitled "A collaborative operation system for multiple agricultural machinery based on automatic navigation", which includes an airborne collaborative operation general control module, a working component that cooperates with the airborne collaborative operation general control module, a wireless data transmission module, and a working component that cooperates with the wireless data transmission module; and a built-in module that works in collaboration with the airborne control module. The built-in module is located in the agricultural machinery and includes five sub-modules: a braking system control module, an engine control module, a steering system control module, an implement suspension control module, and a transmission control module. The sub-modules of the built-in module communicate with each other through a CAN-BUS bus connection to achieve collaborative operation of multiple agricultural machinery. The Chinese patent publication number is CN112034839A, and the name is "A collaborative control method, device, cloud control equipment and agricultural machinery system for a group of agricultural machinery". During control, the method first obtains the first driving information and first position information of the manually driven active agricultural machinery in the agricultural machinery group, and obtains the second driving information and second position information of the driven agricultural machinery in the agricultural machinery group; based on the first driving information and first position information of the active agricultural machinery and the second driving information and second position information of the driven agricultural machinery, control instruction information for controlling the driving of the driven agricultural machinery is generated, and the control instruction information is sent to the driven agricultural machinery, so that the driven agricultural machinery and the active agricultural machinery can work in coordination, thereby realizing the collaborative operation of the agricultural machinery group with less human resources.

[0004] The problem with the above-mentioned agricultural machinery coordination system is that when agricultural machinery is in an uncertain and complex environment, the human driving characteristics are difficult to be fully imitated by the agricultural machinery control system. Therefore, the coordination and adaptability to the complex transfer environment are poor, and the time-varying response to sudden abnormal situations is poor, and the safety and stability of the agricultural machinery group during the transfer process cannot be guaranteed. Summary of the Invention

[0005] The purpose of this invention is to solve the problems of poor environmental adaptability and poor time-varying response to sudden abnormal situations in existing agricultural machinery collaborative systems, and to propose a human-machine integrated intelligent agricultural machinery automatic driving formation transfer system, as well as a formation transfer method for intelligent agricultural machinery to ensure the safety and stability of agricultural machinery groups during transfer.

[0006] The technical solution adopted by the human-machine integrated intelligent agricultural machinery automatic driving formation transfer system described in the present invention is: it is installed on a pilot host and several follower machines, the pilot host is a manually driven agricultural machinery, the pilot host and several follower machines form a formation transfer group, the pilot host and each follower machine are equipped with the same driving status acquisition system, the pilot host is also provided with a formation control terminal system, a router master station and a host WiFi module connected in sequence, and each follower machine is provided with a slave WiFi module that wirelessly communicates with the host WiFi module; the driving status acquisition system collects: the distance L and the relative angle θ between the machine and the transfer environment obstacles, environmental vehicles, and other agricultural machinery in the group within the measurement range; the distance d between the machine and the lane line within the measurement range, and the relative speed v with the environmental obstacles, environmental vehicles, and other agricultural machinery in the group e , image information in front of each agricultural machine, traffic light status information; driving speed v of each agricultural machine, agricultural machine yaw angle φ; position information (x, y) of each agricultural machine, the relative distance l between each agricultural machine and other agricultural machines in the group, and the relative angle β between the driving direction of each agricultural machine and the position direction of other agricultural machines;

[0007] All collected information is transmitted to the formation control terminal system via the router master station. The formation control terminal system includes an on-board computer and a human-computer interaction interface that are interconnected. The on-board computer obtains the machine adjustment decision of the follower machine based on all collected information.

[0008] The pilot host operator transmits human decisions to the on-board computer through the human-computer interaction interface. The on-board computer integrates the human decisions with the machine adjustment decisions to generate control instructions for formation decisions and control quantity inputs. The control instructions are sequentially transmitted to the driving control system of each following slave via the router master station, the host WiFi module, and the slave WiFi module.

[0009] The technical solution adopted by the formation transfer method of the intelligent agricultural machinery automatic driving formation transfer system includes:

[0010] S1. The driving status collection system of each agricultural machine obtains information, packages the information and uploads it to the formation control terminal system;

[0011] S2. The pilot host operator sets up the human-computer interaction interface and saves the set information to the formation control terminal system;

[0012] S3, the formation control terminal system fits the leading machine's trajectory point into the trailing machine's predicted following path f based on the position information (x, y) and the agricultural machine's yaw angle φ ref ;

[0013] S4. Construct the environmental potential field U of the rear engine based on environmental obstacles and lane line control factors. APF ;

[0014] S5, follow the path f according to the prediction of the subsequent machine ref 、Environmental potential field U APF Obtain a model predictive following controller based on the model predictive following controller and the control constraint 0≤v≤1.5·V i , |δ|≤δ i , the control increment and the rear machine control input are the rear wheel speed v and the front wheel angle δ of the agricultural machinery; V i is the initial target speed, δ i is the initially set limit deflection angle;

[0015] S6. The formation control terminal system displays the information collected by each agricultural machine on the human-machine interaction interface. The pilot host operator monitors the group's transfer process in real time. If the group encounters a high-confidence scenario where the machine cannot be adjusted during transfer, the pilot host operator inputs a human decision-making instruction based on his or her experience and judgment into the formation control terminal system.

[0016] S7. The formation control terminal system integrates human-machine decisions and sends the rear wheel speed v, front wheel turning angle δ and start-stop instructions to each following slave machine.

[0017] The beneficial effects of the present invention after adopting the above technical solution are:

[0018] 1. The present invention can reduce the difficulty of formation transfer control through the method of human-machine integration, add a decision-making method for human formation transfer control, realize human-machine integration of perception, interaction, decision-making and control of agricultural machinery formation transfer, improve control accuracy, and comprehensively improve the reliability of the multi-machine formation transfer system.

[0019] 2. The present invention realizes the system function of manually driving a single head vehicle and unmanned following vehicles in formation, and transferring multiple vehicles under the supervision of one person.

[0020] 3. The present invention realizes the complementary advantages between human experience and machine decision-making, reduces the occurrence of unexpected situations, improves the adaptability and safety of the formation transfer, reduces labor costs, improves work efficiency, and enables the transfer of agricultural machinery from labor-saving to labor-saving. It is an important technical support for the construction of unmanned farms. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a structural block diagram of a human-machine integrated intelligent agricultural machinery automatic driving formation transfer system of the present invention;

[0022] Figure 2 yes Figure 1 Internal structure diagram;

[0023] Figure 3 This is a flow chart of the human-machine fusion formation method of the autonomous driving formation transfer system of the present invention;

[0024] Figure 4 Design diagram of the "Formation System Main Menu" interface for the human-computer interaction interface;

[0025] Figure 5 Design diagram of the "formation initial creation" interface for the human-computer interaction interface;

[0026] Figure 6 Design diagram of the "formation slave viewing" interface for the human-computer interaction interface;

[0027] Figure 7 Design diagram of the "Formation Command Adjustment" interface of the human-computer interaction interface. DETAILED DESCRIPTION

[0028] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0029] refer to Figure 1 The human-machine integrated intelligent agricultural machinery automatic driving formation transfer system described in the present invention is installed on a pilot host and several follower machines. The pilot host is an agricultural machine driven by humans, and the several follower machines are agricultural machines that can automatically follow the leading machine. The pilot host and the several follower machines constitute a formation transfer group.

[0030] The pilot host is equipped with a host vehicle status acquisition system, a host real-time communication system, and a formation control terminal system. The host vehicle status acquisition system is connected to the host real-time communication system via serial communication, and the host real-time communication system is bidirectionally interconnected with the formation control terminal system.

[0031] Each following slave machine is provided with a slave machine driving state acquisition system, a slave machine real-time communication system and a slave machine driving control system which are connected in sequence.

[0032] The host real-time communication system and the slave real-time communication system are bidirectionally interconnected.

[0033] The master machine's driving status acquisition system collects data from the pilot machine and transmits it to the master machine's real-time communication system. The slave machine's driving status acquisition system collects data from each following machine and transmits it to the corresponding slave machine's real-time communication system. Real-time communication between each machine is achieved through the master machine's real-time communication system and the slave machine's real-time communication system.

[0034] The slave machine driving control system controls the corresponding follower machines respectively. The formation control terminal system provides a human-machine interaction interface and integrates machine adjustment algorithms to implement a human-machine fusion formation method, thereby achieving human-machine integration of perception, interaction, decision-making, and control in the transfer of agricultural machinery formations. Specifically:

[0035] like Figure 2 As shown, the structures of the host vehicle driving state acquisition system and the slave vehicle driving state acquisition system are exactly the same, both of which include an environment state acquisition module, a driving state acquisition module, and a positioning state acquisition module.

[0036] Among them, the environmental status acquisition module includes lidar and visual camera:

[0037] The laser radar is used to obtain the relative distance L and angle θ between the agricultural machine and the transition environment obstacles, environmental vehicles, and other agricultural machines in the group within the measurement range.

[0038] The visual camera is used to obtain the distance d between the agricultural machine and the lane line within the measurement range, and the relative speed v to environmental obstacles, environmental vehicles, and other agricultural machines in the group. e , as well as image information in front of each agricultural machine and traffic light status information.

[0039] The information collected by the environmental status acquisition module is used to constrain the safety distance of the following machine's following control, ensuring that each machine in the group maintains a safe distance from other machines and obstacles in the transfer environment; it is also used for lane line constraints to ensure that the following machine drives along the lane line and determines whether the lane line is compliant when changing lanes; it is also used for traffic light constraints to ensure that following stops when the light is red and continues when the light is green.

[0040] The driving state acquisition module uses the IMU inertial measurement unit to obtain the driving speed v and yaw angle φ of each agricultural machine.

[0041] The positioning status acquisition module includes a GPS module and an UWB module. The GPS module obtains the location information (x, y) of each agricultural machine, and the UWB module obtains the relative distance l between each agricultural machine and other agricultural machines in the group, and the relative angle β between the driving direction of each agricultural machine and the position direction of other agricultural machines.

[0042] The host vehicle driving status acquisition system transmits all the information it collects to the host real-time communication system via serial communication, and each slave vehicle driving status acquisition system transmits all the information it collects to the slave real-time communication system of the same slave vehicle via serial communication.

[0043] During the transfer of agricultural machinery from the hangar to the field, if there is a low GPS coverage in indoor or outdoor transition areas or on tree-lined roads, the position information (x A ,y A ) Because its GPS module fails and cannot be accurately obtained, it is necessary to use the location information of other follower B with the help of the GPS module (x B ,y B ), and the relative distance l obtained by the UWB module of the follower A and the other follower B A , relative angle β A This information is used for position estimation and collaborative positioning to calculate the precise position information (x A ,y A ) is used as the position information of the following slave A for subsequent fitting of the predicted following path of each slave.

[0044] The formation control terminal system includes an on-board computer and a human-computer interaction interface, which is bidirectionally interconnected with the host real-time communication system to transmit control instructions to the host real-time communication system.

[0045] The host real-time communication system includes a bidirectionally connected router master station and WiFi module, while the slave real-time communication system only has a WiFi module. Information is transmitted between the host and slave real-time communication systems using the two WiFi modules. Based on TCP / IP protocol and socket technology, each follower's real-time communication system packages all received vehicle status information on a per-slave basis. This information is then wirelessly uploaded to the navigation host's router master station via the WiFi module slave node of each follower. After the terminal processes the information, the router master node distributes control information to each follower slave node. In addition to information acquired through the vehicle status collection system, the data packet includes: frame header, machine number, basic agricultural machinery information, current frame data volume, checksum frame, IP address, and frame trailer. Basic agricultural machinery information includes: license plate number, formation status, and task link. Formation status includes: manual operation, automatic operation, and crew following.

[0046] In addition, the UWB module's own wireless transceiver can realize the transmission of relative distance information between agricultural machinery.

[0047] The slave real-time communication system is connected to the slave driving control system via serial communication. The slave driving control system includes a control main board. The control main board controls specific execution components through CAN bus communication. These execution components include electronic throttle, electronic speed control device, electric steering wheel, electronic start-stop device, etc.

[0048] The formation control terminal system transmits its control instructions in sequence through the router master station in the host real-time communication system, two WiFi communications, and the slave real-time communication system to the slave driving control system. The control instructions include the rear wheel speed control value v, the front wheel angle control value δ, and the start-stop instruction. After receiving the control instructions, the slave driving control system converts them into executable quantities to control specific execution components, including: according to the rear wheel speed control value v, the gear and speed of each following slave are controlled by the electronic throttle and electronic speed control device; according to the front wheel angle control value δ, the direction of the following slave is controlled by the electric steering wheel; according to the start-stop instruction, the following stop of each following slave is controlled by the electronic start-stop device.

[0049] See also Figure 3 When the intelligent agricultural machinery automatic driving formation transfer system of the present invention is working, the formation control terminal system controls the transfer process to form an agricultural machinery formation group with a leading host machine manually driving and navigating, and multiple following slave machines automatically following; each agricultural machine obtains its own information through its own driving status acquisition system; and finally uploads the local information to the formation control terminal system through its own real-time communication system. The formation control terminal system uses the formation transfer method to form a machine adjustment algorithm to solve and obtain the follower machine adjustment decision according to the control target and its constraints, and controls each follower machine to follow the host machine's driving trajectory. Then, by setting a human-computer interaction interface, the group information is displayed in real time, and information is exchanged with the host operator to obtain human decisions. The machine adjustment is integrated with human judgment to generate a formation decision and control quantity input that can cope with time changes, and then sent to the slave machine driving control system, such as Figure 3 The specific steps are as follows:

[0050] S1. The driving status collection system of each agricultural machine obtains information, packages the information of each agricultural machine and uploads it to the formation control terminal system.

[0051] At the starting point, each agricultural machine that needs to be transferred is manually started. The driving status collection system of each agricultural machine obtains and packages the local information. The local information is uploaded to the formation control terminal system through the local real-time communication system. The packaged information flow includes:

[0052] Basic information of each agricultural machine: license plate number, formation status, and operation task link;

[0053] Environmental status information of each agricultural machine, including: distance L, angle θ, relative speed v between the machine and the transition environment obstacles, environmental vehicles, and other agricultural machines in the group within the measurement rangee , distance d from the lane line, signal light status, and images in front of each machine;

[0054] The driving status information of each agricultural machine includes: the speed v and yaw angle φ of the machine;

[0055] The positioning status information of each agricultural machine includes: the position of the machine (x, y), the distance l between the machine and other agricultural machines in the group, and the angle β between the driving direction of the machine and the position directions of other agricultural machines.

[0056] S2. Select the team number of the follower machine, set the team constraint conditions, and complete the initial creation of the team.

[0057] Set up the human-computer interaction interface, as shown in the figure As shown. The pilot host operator passes Figure 4 The "Initial Creation of Formation" window shown obtains the basic information of the agricultural machinery within the communication range, selects the license plate number of the agricultural machinery to be transferred and the "Join Formation" command, and this information is saved in the formation control terminal system.

[0058] After the formation control terminal system makes a judgment, it changes the status of the agricultural machinery joining the formation from "manual driving" to "crew following", sets the joining agricultural machinery as follower machines No. 1 to N, and displays them in the window. The follower machines determine the formation order according to the numbers, and use the follower agricultural machinery with the previous number as the following target. Follower machine No. 1 follows the lead host, follower machine No. 2 follows follower machine No. 1, and so on, forming a "one-to-one" following control in which the leading machine serves as the following target of the following machine, thereby realizing the crew formation transfer of all follower machines following the lead host.

[0059] The pilot host operator enters the initial parameters of the transfer unit through the "Initial Creation of Formation" window and saves them in the formation control terminal system. The initial parameters include:

[0060] Driving constraint: target speed V i , limit the deflection angle δ i ;

[0061] Environmental potential field constraint: lateral potential field convergence coefficient X res , safety distance S i , lane line potential field coefficient A R , lane line potential field convergence coefficient R res , formation reorganization distance D i ;

[0062] After completing all settings, you can input the start formation signal, such as Figure 5 As shown, the formation control terminal system performs transition after receiving the formation start signal.

[0063] S3, based on the curve coefficient fitting method, the trajectory point of the leading aircraft is fitted into the predicted following path f of the trailing aircraft ref .

[0064] In the “one-to-one” following control, the position (x, y) of the leading aircraft obtained by the formation control terminal system is set to (x f ,y f ), the yaw angle φ is set to φ f , store the previous machine's historical trajectory points p=[x f y f φ f ], forming a trajectory point stack. Considering that too many trajectory points will occupy storage space, and too few will affect the subsequent fitting and following path effect, the target speed V is calculated based on the initial parameter. i Determine the storage interval and number of trajectory points. Convert the trajectory point sequence to the coordinate system of the current follower, and use the curve coefficient fitting method to fit the trajectory points into a continuous smooth curve as the predicted follow path f of the follower. ref Among the curve coefficient fitting methods, the free boundary cubic spline interpolation algorithm is preferred. This algorithm fits the trajectory points into a straight line between two points as much as possible, with a continuous and smooth curve path at the inflection point, so that the generated prediction follows the path f ref More in line with the driving characteristics of agricultural machinery.

[0065] S4. Based on the artificial potential field method, the environmental potential field U of the rear engine is constructed according to the control influencing factors such as environmental obstacles and lane lines. APF Based on factors that affect the control of the rear aircraft, such as environmental obstacles and lane lines, the environmental potential field of the rear aircraft is constructed as follows:

[0066] U APF =E+R

[0067] Among them, E is the environmental obstacle potential field, which is composed of the distance L, angle θ, and relative speed v between the rear machine and the transition environmental obstacles, environmental vehicles, and other agricultural machinery in the measurement range. e It is negatively correlated with distance and positively correlated with speed. It is used to control agricultural machinery to avoid obstacles and maintain a safe distance. The specific calculation is as follows:

[0068] Calculate the longitudinal potential field coefficient Y E :

[0069]

[0070] Among them, L Y = L·cosθ is the longitudinal relative distance to environmental obstacles, other machines, and other vehicles, v e is the relative speed to the environmental obstacles and vehicles, S i This is the initial safety distance.

[0071] According to the longitudinal potential field coefficient Y E And calculate the environmental obstacle potential field E:

[0072]

[0073] Among them, L X = L·sinθ, is the lateral relative distance to environmental obstacles, other machines, and other vehicles, X res is the initially set convergence coefficient of the transverse potential field, and the convergence coefficient range can be 0-10.

[0074] R is the lane line potential field, which is determined by the distance d between the agricultural machine and the identified lane line. It is used to control the agricultural machine to drive along the lane line. The specific calculation formula is as follows:

[0075]

[0076] Among them, n is the total number of lane lines, d is the distance to each lane line, and d j is the distance to the jth lane line, A R is the initial lane line potential field coefficient, R res is the initial set lane line potential field convergence coefficient.

[0077] S5, based on the model prediction method, the rear machine follows the path f according to the prediction ref Following the trajectory of the preceding vehicle, according to the environmental potential field U APF To ensure driving safety, according to the target speed V i Maintain the following control target of the given speed, establish the kinematic model of the rear machine and the model predictive following controller, and obtain the machine adjustment decision;

[0078] In the ground coordinate system, the Ackerman steering model commonly used in agricultural machinery is used:

[0079]

[0080] Among them, (x, y) is the position of the agricultural machine in the ground coordinate system, φ is the yaw angle of the agricultural machine, v is the speed of the rear wheel of the agricultural machine, δ is the turning angle of the front wheel of the agricultural machine, and l is the wheelbase of the agricultural machine.

[0081] The system model is expressed as follows using state space equations:

[0082]

[0083] In this system, the state quantity is X = [x, y, φ] T , the control quantity is u=[v,δ] T .

[0084] With T as a discrete time step, the system is discretized as follows:

[0085]

[0086]

[0087] Where k is the discretized step size, and X(k) is the system state quantity at the kth step size.

[0088] Adding control increment constraints to the system, let u(k+1)=u(k)+Δu(k+1), the transformed system model is t is the continuous time, which is abbreviated as Further we can get:

[0089]

[0090] The control increment Δu(k)=[Δv(k),Δδ(k)] T , where Δv(k) is the speed change within the k-th step, and Δδ(k) is the wheel deflection angle change within the k-th step.

[0091] Based on the model prediction method, the rear machine follows the path f according to the prediction ref Following the trajectory of the preceding vehicle, according to the environmental potential field U APF To ensure driving safety, according to the target speed V i To maintain the following control goal of a given speed, a model-predicted tracking controller is designed, and its objective function is:

[0092]

[0093] Among them, i is the prediction step length, e x (i) e y (i) are the position of the follower in the predicted time domain (x, y) and the predicted following path f ref The lateral error and longitudinal error are used to track the trajectory of the preceding aircraft; v (i) is the speed v of the rear engine and the set driving speed V in the prediction time domain i Deviation, used to control the driving speed to maintain the set value; U APF (i) is the environmental potential field value of the rear engine at each prediction step i in the prediction time domain, which is used to ensure driving safety; △u(k+i|t) is the control increment, which is used to optimize and solve the problem of increasing the stability and economy of the rear engine driving; W, R, Q, and P are weight matrices; N p 、N c are prediction step size and control step size respectively.

[0094] In order to ensure that the optimized control input of the post-machine adapts to the working characteristics of the agricultural machinery, the speed v and the angle δ control constraints are set respectively:

[0095] 0≤v≤1.5·V i

[0096] |δ|≤δ i

[0097] Among them, V i is the initial target speed, δ i The initial limit deflection angle.

[0098] After optimizing and solving the objective function, we get the control increment sequence ΔU=[Δu(k),Δu(k+1),…,Δu(k+N c )] T According to the basic principle of model predictive control, the first element Δu(k) in the sequence is used as the control increment from time k-1 to time k to act on the system, that is, the control quantity [v,δ] T =u(k)=u(k-1)+Δu(k), and the control input of the rear machine is obtained, namely the rear wheel speed v and the front wheel steering angle δ of the agricultural machinery, so that the rear machine can meet the control objectives of avoiding environmental obstacles, restricted lane lines, maintaining a set speed, and following the driving trajectory of the leading machine.

[0099] The same method can be used to solve the control input of all follower machines to achieve safe transfer of the entire agricultural machinery group along the driving trajectory of the leading main machine.

[0100] S6. Summarize and classify high-confidence scenarios that the machine cannot adjust, incorporate human experience judgment, and use the formation control terminal system to obtain human operation decisions to intervene in the machine adjustment decisions.

[0101] During the transfer process, the fleet control terminal system processes the information stream uploaded by the real-time communication system of the master and slave aircraft and displays it on the human-computer interaction interface. The master operator monitors the transfer process of the unit in real time. The processed overall information of the unit is displayed on the following screen: Figure 4 The "Formation System Main Menu" window shown in the figure includes: the group transfer environment, the driving route of the leading host, the relative position image of each agricultural machine in the group, and the start and stop information of the slave machines in the group. The processed information of each follower machine is displayed in the following Figure 6 The "Formation Slave View" window shown includes: the environment in front of the slave aircraft, the slave aircraft's driving route image, the slave aircraft's speed, the distance to the leading aircraft, and the distance to the master aircraft.

[0102] When the data displayed in the above two windows is abnormal, and the crew encounters a high-confidence scenario where the machine cannot adjust during the transfer, the pilot host operator's experience-based judgment will be input into the formation control terminal system. High-confidence scenarios include: passing a traffic light, encountering other agricultural machines joining the formation, a slave machine in the team reaching the work point, obstacles blocking the way, slave machine failure, traffic accidents, etc. The human decision-making for each scenario can be summarized as follows:

[0103] 1) New agricultural machinery joins the fleet;

[0104] 2) Follow the follower aircraft out of the formation;

[0105] 3) Split the formation or disband the formation.

[0106] The operator controls the crew transfer process and abnormal events through the "Formation Command Adjustment" window set in the human-computer interaction interface of the formation control terminal system, and issues intervention commands to the formation control terminal system to intervene in the machine adjustment decision, such as Figure 7 As shown, the windows mentioned in this section are all "Formation Command Adjustment" windows.

[0107] During the formation process, if a new agricultural machine joins the formation, the specific operation steps are as follows:

[0108] The new agricultural machinery uploads information to the host's formation control terminal system through its own slave real-time communication system. The formation control terminal system displays the basic information of agricultural machinery that is not in the formation within the host's communication range in real time through a window. The operator determines whether the machine needs to join the formation. If so, select the license plate number of the agricultural machinery and enter the "join formation" command into the control terminal.

[0109] After the formation control terminal system obtains the input information, it changes the formation status of the machine from "manual driving" to "crew following", and assigns it to the last digit of the crew number, so that it follows the agricultural machine at the end of the original team as the leading vehicle, obtains the control inputs of rear wheel speed v and front wheel turning angle δ, and enters the formation as the new tail of the new team.

[0110] If a slave in the formation needs to exit the formation due to a fault or reaching its operating point, the specific steps are as follows:

[0111] If a slave machine fails to operate due to a fault and the formation control terminal system does not receive any slave machine information within a certain period, it will automatically determine that the machine is not in the formation, and the formation status will be changed from "crew following" to "manual control". The formation control terminal system will issue a stop command to the machine and will no longer calculate its control input. The machine will stop and wait for manual control.

[0112] If other special circumstances require stopping following, the operator selects the locomotive license plate number and the "exit formation" command through the window and inputs it into the formation control terminal system to make it exit the formation. The formation status is changed from "crew following" to "manual driving". The terminal issues a stop command to the machine and no longer calculates its control input. The machine stops and waits for manual driving.

[0113] If a follower machine arrives at the designated operating point, the operator selects the locomotive license plate number and the corresponding operating task link input into the formation control terminal system and sends it to the machine, and then selects the "exit formation" command to make it exit the formation. The formation status is changed from "crew following" to "autonomous operation". The formation control terminal system sends a stop command to the machine and no longer solves its control input. The machine starts autonomous operation according to the operating task.

[0114] When the state of any agricultural machinery in the formation is changed from "crew following" to other states, the formation control terminal system will renumber all the slaves following the leaving slave in sequence, and the slave with the next highest number will reconfirm the following target, and the slave with the highest number highest number will be used as the leading slave for following control, and the control inputs of rear wheel speed v and front wheel turning angle δ will be obtained, and so on.

[0115] If you need to split or disband a formation due to abnormal conditions such as traffic lights, the specific steps are as follows:

[0116] If the formation control terminal system detects a red signal light on a follower, or if a follower fails to calculate its control inputs due to excessive or dense environmental obstacles, the terminal assigns zero values ​​to the rear wheel speed v and front wheel angle δ for that follower and issues a stop command, halting the follower control and forcing it to wait. The operator then selects the locomotive's license plate number and enters the "Split Formation" command into the terminal. The original formation is split into two sub-groups, with that locomotive as the split point: the lead locomotive and the sub-groups before it become the original sub-group, and the locomotive and the sub-groups after it become the new sub-group.

[0117] The formation control terminal system assigns 0 to the rear wheel speed v and front wheel angle δ of all slave aircraft in the new team, and sends a stop command to the slave aircraft at the split point to stop following control and wait in place; at the same time, all slave aircraft in the original team continue to follow the navigation host and perform following control. The original team drives to the nearest safe point to wait and input the "reorganize formation" command.

[0118] The formation control terminal system records the historical trajectory points of the tail slave of the original squadron (i.e., the preceding slave at the split point) from the start of inputting the "split formation" command to the end of inputting the "reorganize formation" command, and fits it into the predicted following path of the split point slave.

[0119] After the formation control terminal system obtains the identification signal light turning green or the obstacle is removed and the machine adjustment decision can obtain the control input, the formation control terminal system stops assigning 0 value to the control input of the new team slave and issues a start command to make it continue to perform following control. After obtaining the control input of the rear wheel speed v and the front wheel angle δ, the split point slave follows the front machine according to its predicted following path, and the leading new team slave merges with the original team until the distance between the split point slave and its front machine is less than the initially set formation reorganization distance D i When the window prompts "Formation reorganization successful", the navigation host can continue to drive and guide the entire formation to transfer.

[0120] If you encounter special circumstances, you can choose to input the "disband formation" command into the terminal. The terminal will change the status of all slave aircraft formations from "crew following" to "manual driving", issue a stop command and no longer calculate the control input.

[0121] S7. Transmit the human-machine decision to the driving control system of each follower machine to realize the formation transfer.

[0122] The formation control terminal system integrates human-machine decision-making, and sends the final calculated control quantity input information of the following slave machine, namely the rear wheel speed control quantity v, the front wheel angle control quantity δ and the start-stop command, to each slave machine control system through the real-time communication system of the host and slave machines. The slave machine control system converts it into executable quantity to control the specific execution components through the control main board. According to the rear wheel speed control quantity v, the electronic throttle and electronic speed control device are used to adjust the gear and speed of each slave machine; according to the front wheel angle control quantity δ, the electric steering wheel is used to control the direction; according to the start-stop command, the electronic start-stop device is used to control each machine to follow and stop, so as to realize the overall formation transfer of the crew to adapt to the changing environment and ensure driving safety.

[0123] The embodiments described are preferred implementations of the present invention, but the present invention is not limited to the above implementations. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention are within the scope of protection of the present invention.

Claims

1. A human-machine integrated intelligent autonomous driving formation method for agricultural machinery, employing an autonomous driving formation transfer system. The autonomous driving formation transfer system is installed on a pilot host machine and several follower machines. The pilot host is a manually driven agricultural machine. The pilot host and several follower machines form a formation transfer unit. The pilot host and each follower machine are equipped with the same driving status collection system. The pilot host is also equipped with a formation control terminal system, a router master station, and a host WiFi module connected in sequence. Each follower machine is equipped with a slave WiFi module that wirelessly communicates with the host WiFi module. The driving status acquisition system collects: the distance L and relative angle θ between the machine and the transition environment obstacles, environmental vehicles, and other agricultural machinery in the measurement range; the distance d between the machine and the lane line in the measurement range, and the relative speed v between the machine and the environmental obstacles, environmental vehicles, and other agricultural machinery in the group. e , image information in front of each agricultural machine, traffic light status information; driving speed v of each agricultural machine, agricultural machine yaw angle φ; position information (x, y) of each agricultural machine, the relative distance l between each agricultural machine and other agricultural machines in the group, and the relative angle β between the driving direction of each agricultural machine and the position direction of other agricultural machines; All collected information is transmitted to the formation control terminal system via the router master station. The formation control terminal system includes an on-board computer and a human-computer interaction interface that are interconnected. The on-board computer obtains the machine adjustment decision of the follower machine based on all collected information. The pilot host operator transmits the human decision to the onboard computer through the human-computer interaction interface. The onboard computer integrates the human decision and the machine adjustment decision to generate the control instructions for the formation decision and the control quantity input. The control instructions are sequentially transmitted to the driving control system of each following slave through the router master station, the host WiFi module, and the slave WiFi module. include: S1. The driving status collection system of each agricultural machine obtains information, packages the information and uploads it to the formation control terminal system; S2. The pilot host operator sets up the human-computer interaction interface and saves the set information to the formation control terminal system; S3, the formation control terminal system fits the leading machine's trajectory point into the predicted following path f based on the position information (x, y) and the yaw angle φ of the agricultural machine ref ; S4. Construct the environmental potential field U of the rear engine based on environmental obstacles and lane line control factors. APF ; S5, follow the path f according to the prediction of the subsequent machine ref 、Environmental potential field U APF Obtain a model predictive following controller based on the model predictive following controller and the control constraint 0≤v≤1.5·V i , |δ|≤δ i , the control increment and the rear machine control input are the rear wheel speed v and the front wheel angle δ of the agricultural machinery; V i is the initial target speed, δ i is the initially set limit deflection angle; S6. The formation control terminal system displays the information collected by each agricultural machine on the human-machine interaction interface. The pilot host operator monitors the group's transfer process in real time. If the group encounters a high-confidence scenario where the machine cannot be adjusted during transfer, the pilot host operator inputs a human decision-making instruction based on his or her experience and judgment into the formation control terminal system. S7. The formation control terminal system integrates human-machine decisions and sends the rear wheel speed v, front wheel turning angle δ and start-stop instructions to each following slave machine.

2. The formation method according to claim 1, characterized in that: The environmental potential field U in step S4 APF =E+R, Longitudinal potential field coefficient L Y =L·cosθ,v e is the relative speed to the environmental obstacles and vehicles, S i is the initial safety distance, X res is the initial set lateral potential field convergence coefficient, n is the total number of identified lane lines, d is the distance to each lane line, d j is the distance to the jth lane line, A R is the initial lane line potential field coefficient, R res is the initial set lane line potential field convergence coefficient.

3. The formation method according to claim 1, characterized in that: In step S5, the objective function of the model prediction following controller is: i is the prediction step length, e x (i) e y (i) are the position of the follower in the predicted time domain (x, y) and the predicted following path f ref The lateral error and longitudinal error, e v (i) is the speed v of the rear engine and the set driving speed V in the prediction time domain i Deviation, U APF (i) is the environmental potential field of the subsequent machine at each prediction step i in the prediction time domain, W, R, Q, P are weight matrices; N p 、N c are the prediction step size and the control step size respectively; △u(k+i|t) is the control increment within the step size (k+i|t), and the control increment △u(k)=[△v(k),△δ(k)] T , Δv(k) is the speed change in the kth step, Δδ(k) is the wheel deflection angle change in the kth step; the control increment sequence ΔU=[Δu(k),Δu(k+1),…,Δu(k+N c )] T , the first element Δu(k) in the sequence is used as the control increment from time k-1 to time k, and the rear machine control input is obtained: the rear wheel speed v of the agricultural machinery and the front wheel steering angle δ of the agricultural machinery.

4. The formation method according to claim 1, characterized in that: The high-confidence scenarios include: passing through traffic lights, encountering other agricultural machinery joining the formation, a slave machine in the team reaching the work point, obstacles blocking the way, slave machine failure, and traffic accidents. The human decision-making for each scenario can be summarized as follows: 1) new agricultural machinery joins the formation; 2) following the slave machine to exit the formation; 3) splitting the formation or disbanding the formation.

5. The formation method according to claim 4, characterized in that: If a new agricultural machine joins the formation, the new agricultural machine uploads the information collected by its own driving status collection system to the formation control terminal system. The formation control terminal system displays the basic information of the agricultural machine that is not in the formation within the host communication range in real time through the window. The operator determines whether the machine needs to join the formation. If necessary, select the license plate number of the agricultural machine and input the "join formation" command into the control terminal. After receiving the input information, the formation control terminal system changes the formation status of the machine from "manual driving" to "crew following" and sets it as the last digit of the crew number. If a slave machine exits the formation, the formation control terminal system does not receive any slave machine information within a certain period. Information, the machine is automatically determined to be not in the formation, and the formation status is changed from "crew following" to "manual driving". The formation control terminal system sends a stop command to the machine and no longer solves its control quantity. The machine stops and waits for manual driving; if a following slave machine arrives at the designated operation point, the operator selects the locomotive license plate number and the corresponding operation task link input into the formation control terminal system and sends it to the machine, and then selects the "exit formation" command to make it exit the formation, and the formation status is changed from "crew following" to "autonomous operation". The formation control terminal system sends a stop command to the machine and no longer solves its control input quantity. The machine starts autonomous operation according to the operation task.

6. The formation method according to claim 5, characterized in that: When the status of any agricultural machinery in the formation is changed from "crew following" to other states, the formation control terminal system will renumber all slaves following the leaving slave in sequence. The slave with the next highest number will reconfirm the following target, and the slave with the highest number highest number will be the leading slave for following control.

7. The formation method according to claim 4, characterized in that: If the formation needs to be split or disbanded due to abnormal traffic signal conditions, the formation control terminal system will assign 0 to the rear wheel speed v and front wheel angle δ of the control input of the locomotive, and issue a stop command to stop it from following. The locomotive license plate number and the "split formation" command input terminal will be selected through the window. The original group will be divided into two teams with the locomotive as the split point. The pilot host and the slaves before the locomotive number will be the original team, and the locomotive and the slaves after the locomotive number will be the new team. The formation control terminal system will assign all the slave control inputs of the new team to the locomotive. The rear wheel speed v and the front wheel angle δ are assigned a value of 0, and a stop command is issued. The slave at the split point stops following control and waits at the spot. At the same time, all the slaves in the original team follow the lead host and continue to follow the control. The original team drives to the nearest safe point and waits and inputs the "reorganize formation" command. The formation control terminal system records the historical trajectory points of the rear slave of the original team from the start of the "split formation" command input to the end of the "reorganize formation" command input, and fits them into the predicted following path of the slave at the split point.

8. The formation method according to claim 7, characterized in that: After the formation control terminal system obtains the identification signal light turning green or the obstacle is removed and the machine adjustment decision can obtain the control input, the formation control terminal system stops assigning a value of 0 to the control input of the new team slave machine, and issues a start command to make it continue to perform following control, obtaining the control input rear wheel speed v and front wheel angle δ. The slave machine at the split point follows the machine in front of it according to its predicted following path, and the leading slave machine of the new team merges with the original team until the distance between the slave machine at the split point and the machine in front of it is less than the initially set formation reorganization distance, and the window prompts "Formation reorganization successful". The leading host machine can continue to drive and navigate the entire formation transfer.

9. The formation method according to claim 1, characterized in that: The position information (x, y) of each agricultural machine is obtained by the GPS module, and the distance l and the angle β are obtained by the UWB module. If the position information of the follower slave A in the group cannot be accurately obtained due to its GPS module, the position information of the follower slave A is calculated based on the position information of other follower slaves with valid GPS modules in the group and the relative distance and relative angle with other follower slaves obtained by the UWB module of the follower slave A.

Images