Rodless aircraft tractor automatic deviation correction method and system based on machine vision

Abstract

The invention relates to a rodless aircraft tractor automatic deviation correction method and system based on machine vision. The method comprises the steps that environment information of a ground area in an airport is collected; acquiring a planned driving path of a target vehicle in the airport; collecting the current position of the target vehicle in real time; and according to the environment information, the planned driving path and the current position, whether a deviation rectification control program is started or not is determined. Therefore, the working efficiency of the tractor is improved, and accident risks are reduced.

Description

Technical Field

[0001] The present invention relates to the field of vehicle driving technology, and in particular to a machine vision-based automatic deviation correction method, system, electronic equipment and non-transient computer-readable storage medium for a poleless aircraft tractor. Background Art

[0002] In airport ground operations, towbarless aircraft tugs are used to move aircraft from parking positions to runways or between parking positions. Traditional aircraft tugs are usually manually operated by operators, which may present some challenges and risks. For example, factors such as operator fatigue, human error, and environmental changes may cause the tug to deviate from the planned driving path, resulting in an accident. Summary of the invention

[0003] In view of the technical problems existing in the prior art, the present invention provides a method, system, electronic device and non-transient computer-readable storage medium for automatic deviation correction of a rodless aircraft tractor based on machine vision.

[0004] The technical solution of the present invention to solve the above technical problems is as follows: The present invention provides a method for automatically correcting the deviation of a rodless aircraft tractor based on machine vision, the method comprising: Collect environmental information of the ground area in the airport; Obtaining a planned driving path of a target vehicle in the airport; Collecting the current position of the target vehicle in real time; Determine whether to start a deviation correction control program based on the environmental information, the planned driving path and the current position.

[0005] Optionally, the real-time acquisition of the current position of the target vehicle includes: A real-time image of the target vehicle during its travel is acquired, and the current position is obtained through analysis by an image processor.

[0006] Optionally, determining whether to start a deviation correction control program according to the environmental information, the planned driving path and the current position includes: Based on a path deviation algorithm, determining whether the target vehicle deviates; If yes, then determine the emergency area; Determine whether the current location is within the emergency area, and if so, re-plan the path; Otherwise, the deviation correction control program will be started automatically.

[0007] Optionally, the determining whether the target vehicle deviates based on a path deviation algorithm includes: Dividing the ground area into a plurality of grids; According to the planned driving path, obtaining a planned driving grid; According to the current position, obtaining the current grid where the target vehicle is located; Calculating the minimum distance between the planned travel grid and the current grid; The minimum distance is compared with a preset threshold value. If the minimum distance is greater than the threshold value, the target vehicle deviates; otherwise, the target vehicle does not deviate.

[0008] Optionally, determining the emergency area includes: Calculating a safety index of each grid according to the environmental information; Based on the safety index, an emergency area is determined.

[0009] Optionally, calculating the safety index of each grid according to the environmental information includes: Based on the environmental information, determine the road flatness P, road obstacles X and road congestion level Q; The safety index of each grid is calculated according to the following formula: ; Among them, X=1 means there are obstacles on the road, X=0 means there are no obstacles on the road, and k represents the weight coefficient.

[0010] Optionally, determining the emergency area based on the safety index includes: taking a plurality of grids whose distances from the planned driving grid are within a preset range as candidate driving areas; The grid with the highest safety index in the candidate driving area is selected as the emergency area.

[0011] Optionally, the re-planning the path includes: Replan the path based on the path planning algorithm.

[0012] Optionally, the path planning algorithm is an ant colony algorithm.

[0013] The present invention also provides a machine vision-based automatic deviation correction system for a rodless aircraft tractor, the system comprising: The first information acquisition device is used to collect environmental information of the ground area in the airport; A second information acquisition device is used to acquire a planned driving path of the target vehicle in the airport; An image acquisition and analysis device, used to acquire the current position of the target vehicle in real time; The deviation analysis device is used to determine whether to start a deviation correction control program based on the environmental information, the planned driving path and the current position.

[0014] Optionally, the image acquisition and analysis device is further used for: A real-time image of the target vehicle during its travel is acquired, and the current position is obtained through analysis by an image processor.

[0015] Optionally, the deviation analysis device is further used for: Based on a path deviation algorithm, determining whether the target vehicle deviates; If yes, then determine the emergency area; Determine whether the current location is within the emergency area, and if so, re-plan the path; Otherwise, the deviation correction control program will be started automatically.

[0016] Optionally, the determining whether the target vehicle deviates based on a path deviation algorithm includes: Dividing the ground area into a plurality of grids; According to the planned driving path, obtaining a planned driving grid; According to the current position, obtaining the current grid where the target vehicle is located; Calculating the minimum distance between the planned travel grid and the current grid; The minimum distance is compared with a preset threshold value. If the minimum distance is greater than the threshold value, the target vehicle deviates; otherwise, the target vehicle does not deviate.

[0017] Optionally, determining the emergency area includes: Calculating a safety index of each grid according to the environmental information; Based on the safety index, an emergency area is determined.

[0018] Optionally, calculating the safety index of each grid according to the environmental information includes: Based on the environmental information, determine the road flatness P, road obstacles X and road congestion level Q; The safety index of each grid is calculated according to the following formula: ; Among them, X=1 means there are obstacles on the road, X=0 means there are no obstacles on the road, and k represents the weight coefficient.

[0019] Optionally, determining the emergency area based on the safety index includes: taking a plurality of grids whose distances from the planned driving grid are within a preset range as candidate driving areas; The grid with the highest safety index in the candidate driving area is selected as the emergency area.

[0020] Optionally, the re-planning the path includes: Replan the path based on the path planning algorithm.

[0021] Optionally, the path planning algorithm is an ant colony algorithm.

[0022] In addition, to achieve the above-mentioned purpose, the present invention also proposes an electronic device, comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby realizing a method for automatic deviation correction of a rodless aircraft tractor based on machine vision as described above.

[0023] In addition, to achieve the above-mentioned purpose, the present invention also proposes a non-transitory computer-readable storage medium, in which a computer software program is stored. When the computer software program is executed by a processor, the automatic deviation correction method of a rodless aircraft tractor based on machine vision as described above is implemented.

[0024] The beneficial effects of the present invention are: (1) The present invention collects environmental information of the ground area in the airport; obtains the planned driving path of the target vehicle in the airport; collects the current position of the target vehicle in real time; and determines whether to start the deviation correction control program based on the environmental information, the planned driving path and the current position. This improves the working efficiency of the tractor and reduces the risk of accidents.

[0025] (2) The present invention divides the ground area into multiple grids, calculates the minimum distance between the planned driving grid and the current grid where the target vehicle is located, and judges whether the target vehicle has deviated based on this. At the same time, the emergency area is determined. If the vehicle is in the emergency area, it is considered a normal deviation. The system can re-plan the path to ensure that the target vehicle safely passes through the emergency area. For abnormal deviations, the system can automatically correct the deviation and redirect the target vehicle back to the planned driving path. In this way, accurate detection of the deviation of the target vehicle and the adoption of corresponding measures can be achieved, thereby improving the safety and efficiency of airport ground operations.

[0026] (3) The present invention also determines the road flatness, road obstacles and road congestion level based on the environmental information of the ground area, proposes a safety index calculation method, takes multiple grids whose distances from the planned driving grid are within a preset range as candidate driving areas, and selects the grid with the highest safety index in the candidate driving area as the emergency area. This further improves the accuracy of deviation detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A scene diagram of an automatic deviation correction method for a rodless aircraft tractor based on machine vision provided by the present invention; Figure 2A flowchart of an automatic deviation correction method for a rodless aircraft tractor based on machine vision provided by the present invention; Figure 3 A schematic structural diagram of an automatic deviation correction system for a rodless aircraft tractor based on machine vision provided by the present invention; Figure 4 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention; Figure 5 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0029] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0030] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0031] See also Figure 1 , Figure 1 A scene diagram of an automatic deviation correction method for a rodless aircraft tractor based on machine vision provided by the present invention. Figure 1As shown, the terminal and the server are connected via a network, such as a wired or wireless network connection. The terminal may include but is not limited to portable terminals such as mobile phones and tablets installed with various network platform applications, as well as fixed terminals such as computers, query machines, and advertising machines. The server provides users with various business services, including service push servers, user recommendation servers, etc.

[0032] It should be noted that Figure 1 The scene diagram of the automatic deviation correction method of a rodless aircraft tug based on machine vision is only an example. The terminal, server and application scenario described in the embodiment of the present invention are for more clearly illustrating the technical solution of the embodiment of the present invention, and do not generate limitations on the technical solution provided by the embodiment of the present invention. A person of ordinary skill in the art can know that with the evolution of the system and the emergence of new business scenarios, the technical solution provided by the embodiment of the present invention is also applicable to similar technical problems.

[0033] Among them, the terminal can be used for: Collect environmental information of the ground area in the airport; Obtaining a planned driving path of a target vehicle in the airport; Collecting the current position of the target vehicle in real time; Determine whether to start a deviation correction control program based on the environmental information, the planned driving path and the current position.

[0034] See also Figure 2 , a flowchart of a method for automatically correcting a rodless aircraft tractor based on machine vision of the present invention is provided, comprising the following steps: Step 201: Collect environmental information of the ground area in the airport.

[0035] The environmental information of the ground area can be collected by machine vision technology, and the environmental information includes the road image of the ground area. For example, a camera or other image acquisition device is used to obtain image data of the ground area, and a single or multiple cameras can be used for acquisition to obtain a comprehensive field of view.

[0036] Step 202: Obtain the planned driving path of the target vehicle in the airport.

[0037] The historical driving paths of the target vehicle in the airport may be obtained, and one of the historical driving paths may be selected as the planned driving path of the target vehicle.

[0038] Step 203: Collect the current position of the target vehicle in real time.

[0039] Among them, a high-definition camera can be used to obtain real-time images of the target vehicle during its travel, and the current position can be obtained through image processor analysis; the current position of the target vehicle can also be obtained through GPS positioning.

[0040] Step 204: Determine whether to start a deviation correction control program based on the environmental information, the planned driving path and the current position.

[0041] In one implementation, step 204 may further include the following steps: Step 211: Based on the path deviation algorithm, determine whether the target vehicle deviates; if so, proceed to step 212; otherwise, continue to collect the current position of the target vehicle.

[0042] In yet another embodiment, the step 211 may further include the following steps: Step 2111: Divide the ground area into multiple grids.

[0043] The system divides the grid into grids according to the size of the airport ground, and the grids can be continuous squares with equal side lengths.

[0044] Step 2112: Obtain a planned driving grid according to the planned driving path.

[0045] The planned driving path is a curve or a straight line segment, and the grid where any point on the curve or the straight line segment is located is used as the planned driving grid.

[0046] Step 2113: According to the current position, obtain the current grid where the target vehicle is located.

[0047] The current position is regarded as a point, and the grid where the point is located is regarded as the current grid.

[0048] Step 2114: Calculate the minimum distance between the planned driving grid and the current grid.

[0049] Among them, obtain the center point A0 of the current grid, obtain the center points of all grids in the planned driving grid {A1, A2, ..., A n}, calculate A0 and {A1,A2,…,A n}, determine the minimum distance.

[0050] Step 2115: compare the minimum distance with a preset threshold value. If the minimum distance is greater than the threshold value, the target vehicle deviates; otherwise, the target vehicle does not deviate.

[0051] The preset threshold is smaller than the side length of the grid.

[0052] Therefore, by calculating the minimum distance, the auxiliary system determines whether the target vehicle deviates from the planned driving path, thereby improving the accuracy and efficiency of correction.

[0053] Step 212: Determine the emergency area.

[0054] In yet another embodiment, the step 212 may further include the following steps: Step 2121: Calculate the safety index of each grid based on the environmental information.

[0055] Optionally, based on the environmental information, the road flatness P, road obstacles X and road congestion level Q are determined. The image processing technology can be used to extract the contour of the road image, and the point cloud method can be used to calculate the concavity of the road surface, and the concavity is used as the road flatness P. The target detection and tracking algorithm is used to detect and track obstacles X in the ground area, such as vehicles, pedestrians, buildings, signs, etc. Congestion indicators such as traffic volume, speed, and congestion in the road image are obtained, and the road congestion level Q is calculated in combination with the machine learning algorithm.

[0056] Furthermore, the safety index of each grid is calculated according to the following formula: ; Among them, X=1 means there are obstacles on the road, X=0 means there are no obstacles on the road, and k represents the weight coefficient.

[0057] It should be noted that the weight coefficient k can be set manually according to actual needs, or can be obtained by the system through statistical analysis.

[0058] Step 2122: Determine an emergency area based on the safety index.

[0059] Optionally, a plurality of grids whose distances from the planned driving grid are within a preset range are taken as candidate driving areas; and a grid with the highest safety index in the candidate driving areas is selected as an emergency area.

[0060] This method further improves the accuracy of deviation detection.

[0061] Step 213, determine whether the current position is within the emergency area, if so, proceed to step 214; otherwise, proceed to step 215.

[0062] Step 214: re-plan the path.

[0063] Optionally, the path is replanned based on a path planning algorithm.

[0064] The path planning algorithm is Ant Colony Optimization (ACO). The ACO is a heuristic optimization algorithm that solves combinatorial optimization problems by simulating the information exchange and path selection behavior of ants when searching for food. This algorithm is a commonly used algorithm in the prior art and will not be described in detail here.

[0065] Step 215: automatically start the deviation correction control program.

[0066] Among them, step 213 determines that the target is not in the emergency area, triggering the computer to generate a correction instruction, which is transmitted to the PLC controller through the data line. Based on the current position of the target vehicle, the PLC controller outputs a signal to control the steering motor to perform the correction action, so that the tractor drives to the planned driving path. In this process, machine vision technology is used to detect the deviation between the tractor and the planned driving path, and the correction angle is calculated in real time to achieve automatic correction.

[0067] Furthermore, the tractor is equipped with an emergency stop button, and when an emergency occurs, the operator can immediately stop the tractor from running by pressing the emergency stop button.

[0068] Through deviation detection and emergency avoidance identification, the system can promptly detect the deviation of the target vehicle and take appropriate measures according to the specific situation. In particular, emergency avoidance is regarded as a normal deviation, and the system can re-plan the path to ensure that the target vehicle passes the emergency area safely. For abnormal deviations, the system can automatically correct the deviation and redirect the target vehicle back to the planned driving path, achieving accurate detection of the target vehicle's deviation and taking appropriate measures, thereby improving the safety and efficiency of airport ground operations. At the same time, it improves the work efficiency of tractors and reduces the risk of accidents.

[0069] See also Figure 3 , Figure 3 A structural schematic diagram of an automatic deviation correction system for a rodless aircraft tractor based on machine vision provided by the present invention.

[0070] like Figure 3 As shown, an automatic deviation correction system for a rodless aircraft tractor based on machine vision proposed in an embodiment of the present invention includes: The first information acquisition device is used to collect environmental information of the ground area in the airport; A second information acquisition device is used to acquire a planned driving path of the target vehicle in the airport; An image acquisition and analysis device, used to acquire the current position of the target vehicle in real time; The deviation analysis device is used to determine whether to start a deviation correction control program based on the environmental information, the planned driving path and the current position.

[0071] See also Figure 4 , Figure 4 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented: Collect environmental information of the ground area in the airport; Obtaining a planned driving path of a target vehicle in the airport; Collecting the current position of the target vehicle in real time; Determine whether to start a deviation correction control program based on the environmental information, the planned driving path and the current position.

[0072] See also Figure 5 , Figure 5 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 5 As shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 411 is stored. When the computer program 411 is executed by a processor, the following steps are implemented: Collect environmental information of the ground area in the airport; Obtaining a planned driving path of a target vehicle in the airport; Collecting the current position of the target vehicle in real time; Determine whether to start a deviation correction control program based on the environmental information, the planned driving path and the current position.

[0073] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0074] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0076] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0078] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0079] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for automatically correcting the deviation of a rodless aircraft tractor based on machine vision, characterized in that: The method comprises: Collect environmental information of the ground area in the airport; Obtaining a planned driving path of a target vehicle in the airport; Collecting the current position of the target vehicle in real time; Determine whether to start a deviation correction control program based on the environmental information, the planned driving path and the current position.

2. The automatic deviation correction method for a rodless aircraft tractor based on machine vision according to claim 1 is characterized in that: The real-time acquisition of the current position of the target vehicle includes: A real-time image of the target vehicle during its travel is acquired, and the current position is obtained through analysis by an image processor.

3. The automatic deviation correction method for a rodless aircraft tractor based on machine vision according to claim 1 is characterized in that: The step of determining whether to start a deviation correction control program according to the environmental information, the planned driving path and the current position includes: Based on a path deviation algorithm, determining whether the target vehicle deviates; If yes, then determine the emergency area; Determine whether the current location is within the emergency area, and if so, re-plan the path; Otherwise, the deviation correction control program will be started automatically.

4. The automatic deviation correction method for a rodless aircraft tractor based on machine vision according to claim 3 is characterized in that: The determining whether the target vehicle deviates based on the path deviation algorithm includes: Dividing the ground area into a plurality of grids; According to the planned driving path, obtaining a planned driving grid; According to the current position, obtaining the current grid where the target vehicle is located; Calculating the minimum distance between the planned travel grid and the current grid; The minimum distance is compared with a preset threshold value. If the minimum distance is greater than the threshold value, the target vehicle deviates; otherwise, the target vehicle does not deviate.

5. The automatic deviation correction method for a rodless aircraft tractor based on machine vision according to claim 4 is characterized in that: Determining the emergency area includes: Calculating a safety index of each grid according to the environmental information; Based on the safety index, an emergency area is determined.

6. The automatic deviation correction method for a rodless aircraft tractor based on machine vision according to claim 5 is characterized in that: The calculating the safety index of each grid according to the environmental information includes: Based on the environmental information, determine the road flatness P, road obstacles X and road congestion level Q; The safety index of each grid is calculated according to the following formula: ； Among them, X=1 means there are obstacles on the road, X=0 means there are no obstacles on the road, and k represents the weight coefficient.

7. The automatic deviation correction method for a rodless aircraft tractor based on machine vision according to claim 6 is characterized in that: The step of determining an emergency area based on the safety index includes: taking a plurality of grids whose distances from the planned driving grid are within a preset range as candidate driving areas; The grid with the highest safety index in the candidate driving area is selected as the emergency area.

8. The automatic deviation correction method for a rodless aircraft tractor based on machine vision according to claim 3 is characterized in that: The re-planning of the path includes: Replan the path based on the path planning algorithm.

9. The automatic deviation correction method for a rodless aircraft tractor based on machine vision according to claim 8, characterized in that: The path planning algorithm is an ant colony algorithm.

10. An automatic deviation correction system for a rodless aircraft tractor based on machine vision, characterized in that: The system comprises: The first information acquisition device is used to collect environmental information of the ground area in the airport; A second information acquisition device is used to acquire a planned driving path of the target vehicle in the airport; An image acquisition and analysis device, used to acquire the current position of the target vehicle in real time; The deviation analysis device is used to determine whether to start a deviation correction control program based on the environmental information, the planned driving path and the current position.

Images