An automatic deviation correction method and system for a rodless aircraft tractor based on machine vision

Abstract

The present invention relates to a method and system for automatic deviation correction of a rodless aircraft tractor based on machine vision, including: collecting environmental information of the ground area in the airport; obtaining the planned driving path of the target vehicle in the airport; collecting the current position of the target vehicle in real time; and determining whether to start a deviation correction control program according to the environmental information, the planned driving path, and the current position. Thereby, the working efficiency of the tractor is improved and the accident risk is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle driving, and particularly to an automatic deviation correction method, system, electronic device and non-transitory computer-readable storage medium for a rodless aircraft tractor based on machine vision. Background Art

[0002] In airport ground operations, rodless aircraft tractors are used to move aircraft from parking positions to positions between runways or parking positions. Traditional aircraft tractors are usually manually controlled by operators, which may pose some challenges and risks. For example, factors such as operator fatigue, human error, and environmental changes may cause the tractor to deviate from the planned driving path, resulting in accidents. Summary of the Invention

[0003] The present invention addresses the technical problems existing in the prior art and provides an automatic deviation correction method, system, electronic device and non-transitory computer-readable storage medium for a rodless aircraft tractor based on machine vision.

[0004] The technical solution of the present invention for solving the above technical problems is as follows:

[0005] The present invention provides an automatic deviation correction method for a rodless aircraft tractor based on machine vision, the method comprising:

[0006] Collecting environmental information of the ground area in the airport;

[0007] Obtaining the planned driving path of the target vehicle in the airport;

[0008] Real-time collecting the current position of the target vehicle;

[0009] Determining whether to start a deviation correction control program according to the environmental information, the planned driving path and the current position.

[0010] Optionally, the real-time collecting the current position of the target vehicle includes:

[0011] Obtaining a real-time image during the driving of the target vehicle, and analyzing the current position through an image processor.

[0012] Optionally, the determining whether to start a deviation correction control program according to the environmental information, the planned driving path and the current position includes:

[0013] Based on a path deviation algorithm, determining whether the target vehicle deviates;

[0014] If so, determining an emergency area;

[0015] Judging whether the current position is within the emergency area, if so, re-planning the path;

[0016] Conversely, automatically start the deviation correction control program.

[0017] Optionally, determining whether the target vehicle deviates based on the path deviation algorithm includes:

[0018] Dividing the ground area into multiple grids;

[0019] Obtaining the planned driving grids according to the planned driving path;

[0020] Obtaining the current grid where the target vehicle is located according to the current position;

[0021] Calculating the minimum distance between the planned driving grid and the current grid;

[0022] Comparing the minimum distance with a preset threshold. If it is greater than the threshold, the target vehicle deviates; otherwise, the target vehicle does not deviate.

[0023] Optionally, determining the emergency area includes:

[0024] Calculating the safety index of each grid according to the environmental information;

[0025] Determining the emergency area based on the safety index.

[0026] Optionally, calculating the safety index of each grid according to the environmental information includes:

[0027] Determining the road surface flatness P, road obstacles X, and road congestion degree Q based on the environmental information;

[0028] Calculating the safety index of each grid according to the following formula:

[0029] ;

[0030] where X = 1 indicates that there are road obstacles, X = 0 indicates that there are no road obstacles, and k represents the weight coefficient.

[0031] Optionally, determining the emergency area based on the safety index includes:

[0032] Regarding multiple grids within a preset distance from the planned driving grid as candidate driving areas;

[0033] Selecting the grid with the highest safety index from the candidate driving areas as the emergency area.

[0034] Optionally, the re-planning of the path includes:

[0035] Re-planning the path based on the path planning algorithm.

[0036] Optionally, the path planning algorithm is an ant colony algorithm.

[0037] The present invention also provides an automatic deviation correction system for a rodless aircraft tractor based on machine vision, and the system includes:

[0038] A first information acquisition device, configured to collect environmental information of a ground area in an airport;

[0039] A second information acquisition device, configured to acquire a planned driving path of a target vehicle in the airport;

[0040] An image acquisition and analysis device, configured to collect the current position of the target vehicle in real time;

[0041] A deviation analysis device, configured to determine whether to start a deviation correction control program according to the environmental information, the planned driving path, and the current position.

[0042] Optionally, the image acquisition and analysis device is further configured to:

[0043] Acquire a real-time image during the driving of the target vehicle, and analyze the current position through an image processor.

[0044] Optionally, the deviation analysis device is further configured to:

[0045] Determine whether the target vehicle deviates based on a path deviation algorithm;

[0046] If so, determine an emergency area;

[0047] Judge whether the current position is within the emergency area. If so, re-plan the path;

[0048] On the contrary, automatically start the deviation correction control program.

[0049] Optionally, determining whether the target vehicle deviates based on the path deviation algorithm includes:

[0050] Divide the ground area into a plurality of grids;

[0051] Acquire a planned driving grid according to the planned driving path;

[0052] Acquire the current grid where the target vehicle is located according to the current position;

[0053] Calculate the minimum distance between the planned driving grid and the current grid;

[0054] Compare the minimum distance with a preset threshold. If it is greater than the threshold, the target vehicle deviates; otherwise, the target vehicle does not deviate.

[0055] Optionally, the determining of the emergency area includes:

[0056] Calculating the safety index of each grid according to the environmental information;

[0057] Determining the emergency area based on the safety index.

[0058] Optionally, the calculating of the safety index of each grid according to the environmental information includes:

[0059] Determining the road flatness P, road obstacles X, and road congestion degree Q based on the environmental information;

[0060] Calculating the safety index of each grid according to the following formula:

[0061] ;

[0062] where X = 1 indicates that there are road obstacles, X = 0 indicates that there are no road obstacles, and k represents the weight coefficient.

[0063] Optionally, the determining of the emergency area based on the safety index includes:

[0064] Taking multiple grids within a preset range of the distance from the planned driving grid as candidate driving areas;

[0065] Selecting the grid with the highest safety index from the candidate driving areas as the emergency area.

[0066] Optionally, the re-planning of the path includes:

[0067] Re-planning the path based on the path planning algorithm.

[0068] Optionally, the path planning algorithm is the ant colony algorithm.

[0069] In addition, to achieve the above object, the present invention also provides an electronic device, including: a memory for storing computer software programs; a processor for reading and executing the computer software programs, thereby implementing a method for automatic deviation correction of a rodless aircraft tractor based on machine vision as described above.

[0070] In addition, to achieve the above object, the present invention also provides a non-transitory computer-readable storage medium, in which a computer software program is stored, and when the computer software program is executed by a processor, a method for automatic deviation correction of a rodless aircraft tractor based on machine vision as described above is implemented.

[0071] The beneficial effects of the present invention are:

[0072] (1) The present invention collects environmental information of the ground area in the airport; obtains the planned driving path of the target vehicle in the airport; collects the current position of the target vehicle in real time; and determines whether to start the deviation correction control program according to the environmental information, the planned driving path, and the current position. Thereby, the working efficiency of the tractor is improved and the accident risk is reduced.

[0073] (2) The present invention divides the ground area into multiple grids, calculates the minimum distance between the planned driving grid and the current grid where the target vehicle is located, and judges that the target vehicle deviates based on this. At the same time, an emergency area is determined. If the vehicle is within the emergency area, it is regarded as a normal deviation, and the system can re-plan the path to ensure the safe passage of the target vehicle through the emergency area. For abnormal deviations, the system can automatically correct the deviation and redirect the target vehicle back to the planned driving path. By this means, accurate detection of the deviation situation of the target vehicle and corresponding measures are taken, thereby improving the safety and efficiency of airport ground operations.

[0074] (3) The present invention also determines the road flatness, road obstacles, and road congestion degree according to the environmental information of the ground area, proposes a safety index calculation method, takes multiple grids whose distance from the planned driving grid is within a preset range as candidate driving areas, and screens the grid with the highest safety index in the candidate driving areas as the emergency area. Thereby further improving the accuracy of deviation detection. Description of the Drawings

[0075] Figure 1 It is a scene diagram of an automatic deviation correction method for a rodless aircraft tractor based on machine vision provided by the present invention;

[0076] Figure 2 It is a flowchart of an automatic deviation correction method for a rodless aircraft tractor based on machine vision provided by the present invention;

[0077] Figure 3 It is a structural schematic diagram of an automatic deviation correction system for a rodless aircraft tractor based on machine vision provided by the present invention;

[0078] Figure 4 It is a hardware structural schematic diagram of a possible electronic device provided by the present invention;

[0079] Figure 5 It is a hardware structural schematic diagram of a possible computer-readable storage medium provided by the present invention. Detailed Embodiments

[0080] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0081] In the description of the present invention, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality of" means two or more unless otherwise specifically defined.

[0082] In the description of the present invention, the term "for example" is used to mean "serving as an example, illustration, or explanation". Any embodiment described as "for example" in the present invention is not necessarily construed as being more preferred or having more advantages than other embodiments. In order for any person skilled in the art to implement and use the present invention, the following description is given. In the following description, details are set forth for purposes of explanation. It should be understood that those of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope that conforms to the principles and features disclosed in the present invention.

[0083] Please refer to Figure 1 , Figure 1 which is a scene diagram of an automatic deviation correction method for a rodless aircraft tractor based on machine vision provided by the present invention. As Figure 1 shown, the terminal and the server are connected through a network, for example, through a wired or wireless network connection, etc. Among them, the terminal may include, but is not limited to, portable terminals such as mobile phones and tablets installed with various network platform applications, as well as fixed terminals such as computers, inquiry machines, and advertising machines. Among them, the server provides various business services for users, including service push servers, user recommendation servers, etc.

[0084] It should be noted that Figure 1The scene diagram of an automatic deviation correction method for a rodless aircraft tractor based on machine vision shown is merely an example. The terminals, servers, and application scenarios described in the embodiments of the present invention are for more clearly illustrating the technical solutions of the embodiments of the present invention, and do not impose limitations on the technical solutions provided by the embodiments of the present invention. Those of ordinary skill in the art can understand that with the evolution of the system and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present invention are equally applicable to similar technical problems.

[0085] Among them, the terminal can be used for:

[0086] Collect the environmental information of the ground area in the airport;

[0087] Obtain the planned driving path of the target vehicle in the airport;

[0088] Collect the current position of the target vehicle in real time;

[0089] Determine whether to start the deviation correction control program according to the environmental information, the planned driving path, and the current position.

[0090] Please refer to Figure 2 , which provides a flowchart of an automatic deviation correction method for a rodless aircraft tractor based on machine vision of the present invention, including the following steps:

[0091] Step 201: Collect the environmental information of the ground area in the airport.

[0092] Among them, the environmental information of the ground area can be collected through machine vision technology, and the environmental information includes the road image of the ground area. For example, image data of the ground area can be obtained using a camera or other image acquisition devices, and a single or multiple cameras can be used for acquisition to obtain a comprehensive field of view.

[0093] Step 202: Obtain the planned driving path of the target vehicle in the airport.

[0094] Among them, the historical driving path of the target vehicle in the airport can be obtained, and one of the historical driving paths can be selected as the planned driving path of the target vehicle.

[0095] Step 203: Collect the current position of the target vehicle in real time.

[0096] Among them, a high-definition camera can be used to obtain real-time images during the driving process of the target vehicle, and the current position can be obtained through analysis by an image processor; the current position of the target vehicle can also be obtained through GPS positioning.

[0097] Step 204: Determine whether to start the deviation correction control program according to the environmental information, the planned driving path, and the current position.

[0098] In one embodiment, step 204 may further include the following steps:

[0099] Step 211: Based on the path deviation algorithm, determine whether the target vehicle deviates; if so, proceed to step 212; otherwise, continue to collect the current position of the target vehicle.

[0100] In yet another embodiment, step 211 may further include the following steps:

[0101] Step 2111: Divide the ground area into multiple grids.

[0102] Among them, the system divides the grids according to the size of the airport ground. The grids can be continuous squares with equal side lengths.

[0103] Step 2112: According to the planned driving path, obtain the planned driving grids.

[0104] Among them, the planned driving path is a curve or a straight line segment. The grid where any point on the curve or straight line segment is located is used as the planned driving grid.

[0105] Step 2113: According to the current position, obtain the current grid where the target vehicle is located.

[0106] Among them, the current position is regarded as a point, and the grid where this point is located is used as the current grid.

[0107] Step 2114: Calculate the minimum distance between the planned driving grid and the current grid.

[0108] Among them, obtain the center point A0 of the current grid, obtain the center points {A1, A2, …, A n} of all grids in the planned driving grid, calculate the straight-line distance between A0 and any point in {A1, A2, …, A n}, and determine the minimum distance.

[0109] Step 2115: Compare the minimum distance with a preset threshold. If it is greater than the threshold, the target vehicle deviates; otherwise, the target vehicle does not deviate.

[0110] Among them, the preset threshold is less than the side length of the grid.

[0111] Thus, by calculating the minimum distance, the auxiliary system can judge whether the target vehicle deviates from the planned driving path, improving the accuracy and efficiency of deviation correction.

[0112] Step 212: Determine the emergency area.

[0113] In yet another embodiment, step 212 may further include the following steps:

[0114] Step 2121: Calculate the safety index of each grid according to the environmental information.

[0115] Optionally, based on the environmental information, determine the road flatness P, road obstacles X, and road congestion level Q. Among them, image processing technology can be used to extract the contour of the road image, and the point cloud method can be used to calculate the unevenness of the road surface, and the unevenness is used as the road flatness P. Use object detection and tracking algorithms to detect and track obstacles X in the ground area, such as vehicles, pedestrians, buildings, signboards, etc. Obtain congestion indicators such as traffic flow, vehicle speed, and congestion in the road image, and combine machine learning algorithms to calculate the road congestion level Q.

[0116] Furthermore, calculate the safety index of each grid according to the following formula:

[0117] ;

[0118] where X = 1 indicates that there are obstacles on the road, X = 0 indicates that there are no obstacles on the road, and k represents the weight coefficient.

[0119] It should be noted that the weight coefficient k can be set manually according to actual needs, or obtained by the system through statistical analysis.

[0120] Step 2122: Determine the emergency area based on the safety index.

[0121] Optionally, take multiple grids within a preset distance from the planned driving grid as candidate driving areas; screen the grid with the highest safety index in the candidate driving areas as the emergency area.

[0122] In this way, the accuracy of deviation detection is further improved.

[0123] Step 213: Determine whether the current position is within the emergency area. If so, go to step 214; otherwise, go to step 215.

[0124] Step 214: Re-plan the path.

[0125] Optionally, re-plan the path based on the path planning algorithm.

[0126] Among them, the path planning algorithm is the Ant Colony Optimization (ACO). The ant colony algorithm is a heuristic optimization algorithm that solves combinatorial optimization problems by simulating the information exchange and path selection behaviors of ants when searching for food. This algorithm is a commonly used algorithm in the prior art and will not be elaborated here.

[0127] Step 215, automatically start the deviation correction control program.

[0128] Among them, in step 213, it is judged that the target is not in the emergency area, triggering the computer to generate a deviation correction instruction. The deviation correction instruction is transmitted to the PLC controller through the data line. Based on the current position of the target vehicle, the PLC controller outputs a signal to control the steering motor to perform a deviation correction action, so that the tractor travels to the planned driving path. During this process, machine vision technology is used to detect the deviation between the tractor and the planned driving path, and the deviation correction angle is calculated in real time to achieve automatic deviation correction.

[0129] Furthermore, the tractor is equipped with an emergency stop button. When an emergency occurs, the operator can immediately stop the tractor by pressing the emergency stop button.

[0130] Through deviation detection and emergency avoidance identification, the system can timely detect the deviation of the target vehicle and take corresponding measures according to the specific situation. In particular, regarding the emergency avoidance as a normal deviation, the system can re-plan the path to ensure the safe passage of the target vehicle through the emergency area. For abnormal deviations, the system can automatically correct the deviation and re-guide the target vehicle back to the planned driving path, realizing the accurate detection of the deviation of the target vehicle and the taking of corresponding measures, thereby improving the safety and efficiency of airport ground operations. At the same time, it improves the working efficiency of the tractor and reduces the accident risk.

[0131] Please refer to Figure 3 , Figure 3 , which is a schematic structural diagram of an automatic deviation correction system for a rodless aircraft tractor based on machine vision provided by the present invention.

[0132] As Figure 3 shown, an automatic deviation correction system for a rodless aircraft tractor based on machine vision proposed in an embodiment of the present invention includes:

[0133] A first information acquisition device for collecting environmental information of the ground area in the airport;

[0134] A second information acquisition device for obtaining the planned driving path of the target vehicle in the airport;

[0135] An image acquisition and analysis device for real-time collecting the current position of the target vehicle;

[0136] A deviation analysis device for determining whether to start the deviation correction control program according to the environmental information, the planned driving path and the current position.

[0137] Please refer to Figure 4 , Figure 4 , which is a schematic diagram of an embodiment of an electronic device provided in an embodiment of the present invention. As Figure 4As shown in the figure, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored on the memory 410 and operable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented:

[0138] Collect the environmental information of the ground area in the airport;

[0139] Obtain the planned driving path of the target vehicle in the airport;

[0140] Collect the current position of the target vehicle in real time;

[0141] Determine whether to start the deviation correction control program according to the environmental information, the planned driving path, and the current position.

[0142] Please refer to Figure 5 , Figure 5 which is a schematic diagram of an embodiment of a computer-readable storage medium provided by an embodiment of the present invention. As Figure 5 shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 411 is stored. When the computer program 411 is executed by a processor, the following steps are implemented:

[0143] Collect the environmental information of the ground area in the airport;

[0144] Obtain the planned driving path of the target vehicle in the airport;

[0145] Collect the current position of the target vehicle in real time;

[0146] Determine whether to start the deviation correction control program according to the environmental information, the planned driving path, and the current position.

[0147] It should be noted that in the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0148] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0149] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and combinations of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing devices to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices produce a system for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0150] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction system that implements the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0151] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0152] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concept. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0153] Obviously, those skilled in the art can make various changes and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A method for automatically correcting the deviation of a rodless aircraft tractor based on machine vision, characterized in that: The method comprises: Collect environmental information of the ground area in the airport; Obtaining a planned driving path of a target vehicle in the airport; Collecting the current position of the target vehicle in real time; Determining whether to start a deviation correction control program according to the environmental information, the planned driving path and the current position includes: Determining whether the target vehicle deviates based on a path deviation algorithm; comprising: Dividing the ground area into a plurality of grids; According to the planned driving path, obtaining a planned driving grid; According to the current position, obtaining the current grid where the target vehicle is located; Calculating the minimum distance between the planned travel grid and the current grid; The minimum distance is compared with a preset threshold value, if it is greater than the threshold value, the target vehicle deviates and an emergency area is determined; otherwise, the target vehicle does not deviate; Determine whether the current location is within the emergency area, and if so, re-plan the path; Otherwise, the deviation correction control program will be automatically started; Determining the emergency area includes: Based on the environmental information, determine the road flatness P, road obstacles X and road congestion level Q; The safety index of each grid is calculated according to the following formula: ； Among them, X=1 means there are obstacles on the road, X=0 means there are no obstacles on the road, and k represents the weight coefficient; taking a plurality of grids whose distances from the planned driving grid are within a preset range as candidate driving areas; The grid with the highest safety index in the candidate driving area is selected as the emergency area.

2. The automatic deviation correction method for a rodless aircraft tractor based on machine vision according to claim 1 is characterized in that: The real-time acquisition of the current position of the target vehicle includes: A real-time image of the target vehicle during its travel is acquired, and the current position is obtained through analysis by an image processor.

3. The automatic deviation correction method for a rodless aircraft tractor based on machine vision according to claim 1 is characterized in that: The re-planning of the path includes: Replan the path based on the path planning algorithm.

4. The automatic deviation correction method for a rodless aircraft tractor based on machine vision according to claim 3 is characterized in that: The path planning algorithm is an ant colony algorithm.

5. An automatic deviation correction system for a rodless aircraft tractor based on machine vision, characterized in that: The system comprises: The first information acquisition device is used to collect environmental information of the ground area in the airport; A second information acquisition device is used to acquire a planned driving path of the target vehicle in the airport; An image acquisition and analysis device, used to acquire the current position of the target vehicle in real time; A deviation analysis device, used to determine whether to start a deviation correction control program based on the environmental information, the planned driving path and the current position; The deviation analysis device is further used for: Determining whether the target vehicle deviates based on a path deviation algorithm; comprising: Dividing the ground area into a plurality of grids; According to the planned driving path, obtaining a planned driving grid; According to the current position, obtaining the current grid where the target vehicle is located; Calculating the minimum distance between the planned travel grid and the current grid; The minimum distance is compared with a preset threshold value, if it is greater than the threshold value, the target vehicle deviates and an emergency area is determined; otherwise, the target vehicle does not deviate; Determine whether the current location is within the emergency area, and if so, re-plan the path; Otherwise, the deviation correction control program will be automatically started; Determining the emergency area includes: Based on the environmental information, determine the road flatness P, road obstacles X and road congestion level Q; The safety index of each grid is calculated according to the following formula: ； Among them, X=1 means there are obstacles on the road, X=0 means there are no obstacles on the road, and k represents the weight coefficient; taking a plurality of grids whose distances from the planned driving grid are within a preset range as candidate driving areas; The grid with the highest safety index in the candidate driving area is selected as the emergency area.

Images