Road and bridge inspection data processing method and server based on drone

Abstract

The present invention relates to the field of data processing technology, and discloses a method and server for processing road and bridge inspection data based on drones. By obtaining the target inspection task distributed in the drone road and bridge inspection task list, the maximum flight height information of each road and bridge trajectory point in the target inspection task is determined by using the inspection execution association information and the inspection task execution requirements. Then, based on the maximum flight height information, the target inspection period and the constraint condition set determined by the image overlap rate and the optimization goal of minimizing the inspection flight distance, the inspection trajectory execution strategy of the target inspection drone for the target inspection task is optimized and solved, which can optimize the rationality and superiority of the drone flight path and balance the contradiction between the drone road and bridge inspection efficiency and the inspection shooting image quality. At the same time, by judging the execution delay of the target inspection task, the inspection trajectory execution strategy for the remaining inspection tasks is updated in real time, and the environmental adaptability of the drone in different environments and conditions is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and server for processing road and bridge inspection data based on an unmanned aerial vehicle. Background Art

[0002] Road and bridge inspection refers to the work process of regular or irregular inspection, monitoring and evaluation of roads and bridges. It is usually used to identify road surface defects that have a greater impact on vehicle driving comfort and safety (for example, deep potholes, potholes, and rutting may cause vehicle tire blowouts and loss of control). Through long-term inspection records, the development law of road and bridge defects can be analyzed, and then a reasonable maintenance plan can be formulated (for example, if the rutting disease of a certain section of road develops rapidly, milling and re-paving maintenance projects can be arranged in advance. For example, timely grouting of road cracks can prevent rainwater from penetrating into the road base, avoid damage to the base, and thus extend the service life of the road). At present, road and bridge inspections are mostly carried out by manual inspection, which has the problems of low inspection efficiency and high risk.

[0003] With the advancement of drone technology, drone road and bridge inspection technology has gradually become popular. Drone road and bridge inspection refers to a modern inspection method that uses drone technology to inspect and monitor roads and bridges. However, in the existing drone road and bridge inspection implementation scheme, the role of drones is only to replace manual flight and image acquisition along the road and bridge path. Since the drone flies along the road and bridge path, one image shooting action still only contains one path trajectory point, and the number of inspection images has not been significantly reduced. The workload of image preprocessing and splicing in the subsequent road and bridge disease analysis is still heavy. At the same time, although high-altitude inspection can obtain road and bridge images with multiple path trajectory points through one image shooting, which can reduce the inspection images to a certain extent, in some scenarios (such as road and bridge sections with winding paths in mountainous areas), high-altitude flight control and path planning of drones are very difficult. The high-altitude wind in mountainous areas is usually stronger than the low-altitude wind, which will cause the drone flight to be unstable and affect the image quality. Although low-altitude inspection is more stable, it will affect the inspection efficiency and increase the workload.

[0004] Therefore, how to reasonably plan the routes for UAVs to perform road and bridge inspections in mountainous areas, balance the contradiction between the efficiency of UAV road and bridge inspections and the quality of inspection images, and enhance the environmental adaptability of UAVs in different environments and conditions is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a road and bridge inspection data processing method and server based on unmanned aerial vehicle, aiming to solve at least one of the above-mentioned technical problems.

[0006] To achieve the above object, the present invention provides a method for processing road and bridge inspection data based on an unmanned aerial vehicle, comprising:

[0007] Obtain the target inspection task distributed by the UAV road and bridge inspection task list; wherein the target inspection task includes inspection execution association information and inspection task execution requirements;

[0008] Extract the identification information of the target inspection area and the target inspection drone in the inspection execution association information and the target inspection period, image resolution requirement and image overlap rate in the inspection task execution requirements, use the target inspection area and the pre-trained road and bridge damage grade prediction model to predict the road and bridge damage grade of each road and bridge trajectory point in the target inspection area, and generate the maximum shooting height information of each road and bridge trajectory point based on the road and bridge damage grade and the identification information of the target inspection drone;

[0009] According to the maximum shooting height information and image resolution requirements, the maximum flight height information of each road and bridge trajectory point is determined. Based on the maximum flight height information, the target inspection period and the constraint condition set determined by the image overlap rate and the optimization goal of minimizing the inspection flight distance, the inspection trajectory execution strategy of the target inspection UAV for the target inspection task is optimized and solved; wherein the inspection trajectory execution strategy includes the shooting heights and theoretical inspection shooting times of several inspection image shooting points;

[0010] The inspection trajectory execution strategy is sent to the UAV road and bridge inspection control terminal, so that the target inspection UAV performs the inspection flight and image shooting actions of the target inspection task;

[0011] The actual inspection shooting time of the target inspection drone at each inspection image shooting point is obtained, and based on the actual inspection shooting time and the theoretical inspection shooting time, it is determined whether the target inspection task is delayed. If so, the inspection trajectory execution strategy is regenerated.

[0012] Optionally, before obtaining the target inspection task step for the distribution of the UAV road and bridge inspection task list, it also includes:

[0013] Obtaining the road and bridge disease image demand information sent by the demand terminal; wherein the road and bridge disease image demand information includes the area to which the road and bridge disease image belongs, the image resolution requirement and the image overlap rate;

[0014] The road and bridge disease image demand information is published, and the road and bridge inspection execution information for the road and bridge disease image demand information sent by the inspection terminal is received; wherein the road and bridge inspection execution information includes the target inspection period and the identification information of the target inspection drone;

[0015] Feeding back the road and bridge inspection execution information to the demand terminal, and upon receiving confirmation information from the demand terminal, determining the target inspection area according to the area to which the road and bridge disease image belongs;

[0016] The identification information of the target inspection area and the target inspection drone is used as the inspection execution association information, the target inspection period, image resolution requirement and image overlap rate are used as the inspection task execution requirements, and the target inspection task is generated in the drone road and bridge inspection task list.

[0017] Optionally, the step of predicting the road and bridge disease grade of each road and bridge trajectory point in the target inspection area by using the target inspection area and the pre-trained road and bridge disease grade prediction model specifically includes:

[0018] Based on the target inspection area, a number of road and bridge trajectory points of each road and bridge path are extracted from the road and bridge trajectory electronic map, and a road and bridge disease impact feature set of each road and bridge trajectory point is matched in a road and bridge disease impact feature database according to the location information of each road and bridge trajectory point;

[0019] Accessing a road and bridge disease case database, extracting the road and bridge disease grade and road and bridge disease impact characteristics of each road and bridge disease case record stored in the road and bridge disease case database, using the road and bridge disease grade and the road and bridge disease impact characteristics to construct road and bridge disease training samples and train a neural network model, and obtaining a trained road and bridge disease recognition model;

[0020] The road and bridge disease impact feature set is input into the trained road and bridge disease grade prediction model to obtain the road and bridge disease grade of each road and bridge trajectory point output by the road and bridge disease grade prediction model.

[0021] Optionally, based on the road and bridge damage level and the identification information of the target inspection drone, the step of generating the maximum shooting height information of each road and bridge trajectory point specifically includes:

[0022] Based on the road and bridge damage level of each road and bridge trajectory point and the image resolution requirements of UAV road and bridge inspection for different road and bridge damage levels, the image resolution requirement information of each road and bridge trajectory point is determined;

[0023] The maximum shooting height information of each road and bridge trajectory point is calculated according to the camera parameters determined in the camera parameter library based on the image resolution requirement information and the identification information of the target inspection drone.

[0024] Optionally, according to the maximum shooting height information and the image resolution requirements, the maximum flight height information of each road and bridge trajectory point is determined, and based on the maximum flight height information, the constraint condition set determined by the target inspection period and the image overlap rate and the optimization goal of minimizing the inspection flight distance, the inspection trajectory execution strategy steps of the target inspection UAV for the target inspection task are optimized, specifically including:

[0025] Using the shooting height information and the elevation information of each road and bridge trajectory point in the geographic elevation database, determine the maximum flight altitude information of the target inspection drone at each road and bridge trajectory point;

[0026] The sum of the spatial areas of each road and bridge trajectory point within the preset spatial distance range constitutes the inspection image shooting spatial area, and the inspection image shooting points are selected in the inspection image shooting spatial area, with the flight altitude of the target inspection UAV at each inspection image shooting point being less than the maximum flight altitude of each road and bridge trajectory point contained in the shot image as the first constraint condition, and the overlap rate of the road and bridge trajectory points of the target cruise UAV in the shot images of two adjacent inspection image shooting points being higher than the image overlap rate as the second constraint condition, and the completion time of the target inspection task determined by the cumulative sum of the standard shooting time of the target inspection UAV at each inspection image shooting point and the flight time of two adjacent inspection image shooting points being shorter than the target inspection period as the third constraint condition, and the minimization of the sum of the inspection flight distances of two adjacent inspection image shooting points executed by the target inspection UAV as the optimization target, and the genetic optimization algorithm is used to optimize and solve the optimal number of inspection image shooting points of the target inspection UAV for the target inspection task;

[0027] Based on several inspection image shooting points and the standard shooting time of the target inspection drone at each inspection image shooting point and the flight time of two adjacent inspection image shooting points, the shooting height and theoretical inspection shooting time of each inspection image shooting point are determined, and the inspection trajectory execution strategy is generated;

[0028] Among them, the steps of using genetic optimization algorithm to optimize and solve the optimal several inspection image shooting points of the target inspection drone for the target inspection task specifically include:

[0029] The inspection image shooting points selected by the target inspection UAV in the inspection image shooting space area are binary-coded in sequence to obtain a number of path sequence codes; wherein each spatial area point selected as the inspection image shooting point is assigned 1, and each spatial area point not selected is assigned 0;

[0030] Based on the plurality of path sequence codes, a number M of chromosomes satisfying the first constraint condition, the second constraint condition and the third constraint condition are randomly generated, and a fitness value is calculated for each chromosome; wherein the fitness function for calculating the fitness value is configured as a preset operation function for converting the inspection flight distance in the optimization target into the fitness value;

[0031] A roulette wheel selection method is adopted to select the corresponding chromosomes into the next generation population according to the fitness value ratio of each chromosome, and a crossover operation is performed on the selected chromosomes and the path sequence encoding corresponding to the chromosome is mutated with a preset probability; wherein the mutation includes the value change and position exchange of the binary characters in the binary encoding;

[0032] Repeat the above steps of calculating the fitness value until the preset number of iterations is reached or the fitness value of the optimal chromosome in the population meets the threshold requirement. At this time, the path sequence code currently corresponding to the optimal chromosome is output, and the path sequence code is decoded to obtain the optimal several inspection image shooting points for the target inspection UAV for the target inspection task.

[0033] Optionally, the inspection trajectory execution strategy is sent to the UAV road and bridge inspection control terminal to enable the target inspection UAV to perform the inspection flight and image shooting action steps of the target inspection task, specifically including:

[0034] The inspection trajectory execution strategy is sent to the UAV road and bridge inspection control terminal, so that the UAV road and bridge inspection control terminal converts a number of inspection image shooting points in the inspection trajectory execution strategy into UAV flight control instructions;

[0035] The UAV flight control instruction is sent to the target inspection UAV, so that the target inspection UAV performs an inspection flight and image shooting action including a plurality of inspection image shooting points.

[0036] Optionally, the actual inspection shooting time of the target inspection drone at each inspection image shooting point is obtained, and based on the actual inspection shooting time and the theoretical inspection shooting time, it is determined whether the target inspection task is delayed. If so, the inspection trajectory execution strategy steps are regenerated, specifically including:

[0037] Obtain the actual inspection shooting time of the target inspection drone after completing the image shooting action at each inspection image shooting point, and determine whether the time difference between the actual inspection shooting time and the theoretical inspection shooting time of the inspection image shooting point exceeds a preset difference value;

[0038] If so, take the remaining cruise period as the target cruise period, take the remaining road and bridge trajectory points as the road and bridge trajectory points of the target inspection task, return to execute the optimization solution of the inspection trajectory execution strategy step of the target inspection UAV for the target inspection task, and regenerate the inspection trajectory execution strategy.

[0039] In addition, in order to achieve the above-mentioned purpose, the present invention also provides a road and bridge inspection data processing server based on a drone, comprising:

[0040] An acquisition module is used to acquire a target inspection task distributed by a UAV road and bridge inspection task list; wherein the target inspection task includes inspection execution association information and inspection task execution requirements;

[0041] A generation module is used to extract the identification information of the target inspection area and the target inspection drone in the inspection execution association information and the target inspection period, image resolution requirement and image overlap rate in the inspection task execution requirements, and use the target inspection area and the pre-trained road and bridge disease grade prediction model to predict the road and bridge disease grade of each road and bridge trajectory point in the target inspection area, and generate the maximum shooting height information of each road and bridge trajectory point based on the road and bridge disease grade and the identification information of the target inspection drone;

[0042] A solution module is used to determine the maximum flight height information of each road and bridge trajectory point according to the maximum shooting height information and the image resolution requirement, and optimize the inspection trajectory execution strategy of the target inspection UAV for the target inspection task based on the maximum flight height information, the target inspection period and the image overlap rate. The inspection trajectory execution strategy includes the shooting heights and theoretical inspection shooting times of several inspection image shooting points;

[0043] An execution module is used to send the inspection trajectory execution strategy to the UAV road and bridge inspection control terminal, so that the target inspection UAV performs the inspection flight and image shooting actions of the target inspection task;

[0044] The judgment module is used to obtain the actual inspection shooting time of the target inspection drone at each inspection image shooting point, and based on the actual inspection shooting time and the theoretical inspection shooting time, judge whether the target inspection task is delayed. If so, regenerate the inspection trajectory execution strategy.

[0045] The beneficial effects of the present invention are as follows: a method and server for processing road and bridge inspection data based on drones are proposed. By extending each inspection image shooting point of the target inspection drone to the path trajectory point, it is not limited to flying along the path trajectory of the road and bridge. On the one hand, it can obtain road images of multiple path trajectory points through one image shooting action, adapt to the inspection image acquisition needs of winding road and bridge paths in mountainous areas, and improve the efficiency of drone road and bridge inspection. On the other hand, it is not limited to flying along the path trajectory of the road and bridge, which produces a wider path planning area, and can further optimize the rationality and superiority of the drone flight path to balance the contradiction between the efficiency of drone road and bridge inspection and the quality of inspection shooting images. At the same time, by comparing the actual inspection shooting time of each inspection image shooting point with the theoretical inspection shooting time to realize task delay judgment, and updating the inspection trajectory execution strategy for the remaining inspection tasks in real time, it can solve the delay caused by emergencies during the drone inspection process and enhance the environmental adaptability of drones in different environments and conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of the method for processing road and bridge inspection data based on unmanned aerial vehicle of the present invention;

[0047] Figure 2 The figure is a schematic diagram of the structure of the road and bridge inspection data processing server based on drone of the present invention.

[0048] The implementation, functional features and advantages of the present invention will be further described in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0050] The embodiment of the present invention provides a method for processing road and bridge inspection data based on an unmanned aerial vehicle. Figure 1 , Figure 1 The figure is a flow chart of a method for processing road and bridge inspection data based on an unmanned aerial vehicle according to an embodiment of the present invention.

[0051] In this embodiment, a method for processing road and bridge inspection data based on a drone includes:

[0052] S100: Obtain the target inspection task distributed by the UAV road and bridge inspection task list; wherein the target inspection task includes inspection execution association information and inspection task execution requirements;

[0053] S200: extracting the identification information of the target inspection area and the target inspection drone in the inspection execution association information and the target inspection period, image resolution requirement and image overlap rate in the inspection task execution requirements, using the target inspection area and the pre-trained road and bridge disease grade prediction model to predict the road and bridge disease grade of each road and bridge trajectory point in the target inspection area, and generating the maximum shooting height information of each road and bridge trajectory point based on the road and bridge disease grade and the identification information of the target inspection drone;

[0054] S300: According to the maximum shooting height information and the image resolution requirements, the maximum flight height information of each road and bridge trajectory point is determined, and based on the maximum flight height information, the constraint condition set determined by the target inspection period and the image overlap rate and the optimization goal of minimizing the inspection flight distance, the inspection trajectory execution strategy of the target inspection UAV for the target inspection task is optimized; wherein the inspection trajectory execution strategy includes the shooting heights and theoretical inspection shooting times of several inspection image shooting points;

[0055] S400: sending the inspection trajectory execution strategy to the UAV road and bridge inspection control terminal, so that the target inspection UAV performs the inspection flight and image shooting actions of the target inspection task;

[0056] S500: Obtain the actual inspection shooting time of the target inspection drone at each inspection image shooting point, and determine whether the target inspection task is delayed based on the actual inspection shooting time and the theoretical inspection shooting time. If so, regenerate the inspection trajectory execution strategy.

[0057] It should be noted that in the existing implementation scheme of UAV road and bridge inspection, the role of UAV is only to replace manual execution of flight and image collection along the road and bridge path. Since the UAV flies along the road and bridge path, one image capture action still only contains one path trajectory point, and the number of inspection images has not been significantly reduced. The workload of image preprocessing and splicing in the subsequent road and bridge disease analysis is still heavy. At the same time, although high-altitude inspection can obtain road and bridge images with multiple path trajectory points through one image capture, which can reduce the number of inspection images to a certain extent, in some scenarios (such as road and bridge sections with winding paths in mountainous areas), high-altitude flight control and path planning of UAVs are quite difficult. The high-altitude wind in mountainous areas is usually stronger than the low-altitude wind, which will cause the UAV flight to be unstable and affect the image quality. Although low-altitude inspection is more stable, it will affect the inspection efficiency and increase the workload.

[0058] In order to solve the above problems, this embodiment obtains the target inspection task distributed in the UAV road and bridge inspection task list, uses the inspection execution association information and inspection task execution requirements, determines the maximum flight altitude information of each road and bridge trajectory point in the target inspection task, and then optimizes and solves the inspection trajectory execution strategy of the target inspection UAV for the target inspection task based on the maximum flight altitude information, the target inspection period and the image overlap rate, and the optimization goal of minimizing the inspection flight distance. It can optimize the rationality and superiority of the UAV flight path and balance the contradiction between the UAV road and bridge inspection efficiency and the inspection shooting image quality. At the same time, by judging the execution delay of the target inspection task, the inspection trajectory execution strategy for the remaining inspection tasks is updated in real time, enhancing the environmental adaptability of the UAV in different environments and conditions.

[0059] In a preferred embodiment, before obtaining the target inspection task step distributed by the UAV road and bridge inspection task list, it also includes:

[0060] S01: Obtaining road and bridge disease image demand information sent by a demand terminal; wherein the road and bridge disease image demand information includes the area to which the road and bridge disease image belongs, the image resolution requirement, and the image overlap rate;

[0061] S02: publishing the road and bridge disease image demand information, and receiving road and bridge inspection execution information for the road and bridge disease image demand information sent by the inspection terminal; wherein the road and bridge inspection execution information includes the target inspection period and the identification information of the target inspection drone;

[0062] S03: Feedback the road and bridge inspection execution information to the demand terminal, and upon receiving confirmation information from the demand terminal, determine the target inspection area according to the area to which the road and bridge disease image belongs;

[0063] S04: Use the identification information of the target inspection area and the target inspection drone as inspection execution association information, use the target inspection period, image resolution requirement and image overlap rate as inspection task execution requirements, and generate the target inspection task in the drone road and bridge inspection task list.

[0064] In this embodiment, the road and bridge disease image demand information sent by the demand terminal is obtained in advance, the road and bridge disease image demand information is published, and the road and bridge inspection execution information for the road and bridge disease image demand information sent by the inspection terminal is accepted. The inspection task execution requirements including the target inspection time period, image resolution requirements and image overlap rate and the inspection execution association information including the identification information of the target inspection area and the target inspection drone are extracted from the road and bridge disease image demand information and the road and bridge inspection execution information, so as to generate the target inspection task and store it in the drone road and bridge inspection task list.

[0065] In a preferred embodiment, the step of predicting the road and bridge damage level of each road and bridge trajectory point in the target inspection area by using the target inspection area and the pre-trained road and bridge damage level prediction model specifically includes:

[0066] S210: Based on the target inspection area, extract a number of road and bridge trajectory points of each road and bridge path in the road and bridge trajectory electronic map, and match the road and bridge disease impact feature set of each road and bridge trajectory point in the road and bridge disease impact feature database according to the position information of each road and bridge trajectory point;

[0067] S220: Accessing a road and bridge disease case database, extracting the road and bridge disease grade and road and bridge disease impact characteristics of each road and bridge disease case record stored in the road and bridge disease case database, using the road and bridge disease grade and the road and bridge disease impact characteristics to construct road and bridge disease training samples and train a neural network model, and obtaining a trained road and bridge disease recognition model;

[0068] S230: Inputting the road and bridge disease impact feature set into the trained road and bridge disease grade prediction model to obtain the road and bridge disease grade of each road and bridge trajectory point output by the road and bridge disease grade prediction model.

[0069] Furthermore, based on the road and bridge damage level and the identification information of the target inspection drone, the maximum shooting height information of each road and bridge trajectory point is generated, specifically including:

[0070] S240: Determine image resolution requirement information for each road and bridge trajectory point based on the road and bridge disease level of each road and bridge trajectory point and image resolution requirements of the UAV road and bridge inspection for different road and bridge disease levels;

[0071] S250: Calculate the maximum shooting height information of each road and bridge trajectory point according to the camera parameters determined in the camera parameter library based on the image resolution requirement information and the identification information of the target inspection drone.

[0072] In this embodiment, the image resolution requirements in the inspection task execution requirements and the identification information of the target inspection area and the target inspection UAV in the inspection execution associated information are utilized, and the historical damage conditions recorded in the road and bridge damage case database are taken into consideration to predict the road and bridge damage level of each road and bridge trajectory point in the target inspection area. Taking into account the image resolution requirements, the maximum shooting height information of each road and bridge trajectory point is determined, and this is used as a reference factor for limiting the flight height of the UAV in the target inspection area.

[0073] In a preferred embodiment, the maximum flight height information of each road and bridge trajectory point is determined according to the maximum shooting height information and the image resolution requirement. Based on the maximum flight height information, the constraint condition set determined by the target inspection period and the image overlap rate and the optimization goal of minimizing the inspection flight distance, the inspection trajectory execution strategy steps of the target inspection drone for the target inspection task are optimized and solved, specifically including:

[0074] S310: Determine the maximum flight altitude information of the target inspection drone at each road and bridge trajectory point by using the shooting altitude information and the altitude information of each road and bridge trajectory point in the geographic altitude database;

[0075] S320: The sum of the spatial areas of each road and bridge trajectory point within the preset spatial distance range constitutes the inspection image shooting spatial area, and the inspection image shooting points are selected in the inspection image shooting spatial area, with the flight altitude of the target inspection UAV at each inspection image shooting point being less than the maximum flight altitude of each road and bridge trajectory point contained in the shooting image as the first constraint condition, with the overlap rate of the road and bridge trajectory points of the target cruise UAV in the shooting images of two adjacent inspection image shooting points being higher than the image overlap rate as the second constraint condition, with the completion time of the target inspection task determined by the cumulative sum of the standard shooting time of the target inspection UAV at each inspection image shooting point and the flight time of two adjacent inspection image shooting points being shorter than the target inspection period as the third constraint condition, with the minimum sum of the inspection flight distances of the target inspection UAV executing two adjacent inspection image shooting points as the optimization target, and using the genetic optimization algorithm to optimize and solve the optimal number of inspection image shooting points of the target inspection UAV for the target inspection task;

[0076] S330: Based on a number of inspection image shooting points and the standard shooting time of the target inspection drone at each inspection image shooting point and the flight time of two adjacent inspection image shooting points, determine the shooting height and theoretical inspection shooting time of each inspection image shooting point, and generate an inspection trajectory execution strategy;

[0077] Among them, the steps of using genetic optimization algorithm to optimize and solve the optimal several inspection image shooting points of the target inspection drone for the target inspection task specifically include:

[0078] S321: Binary-code the inspection image shooting points selected by the target inspection drone in the inspection image shooting space area in sequence to obtain a plurality of path sequence codes; wherein each spatial area point selected as the inspection image shooting point is assigned 1, and each spatial area point not selected is assigned 0;

[0079] S322: Based on the plurality of path sequence codes, randomly generate M chromosomes that satisfy the first constraint, the second constraint, and the third constraint, and calculate a fitness value for each chromosome; wherein the fitness function for calculating the fitness value is configured as a preset operation function for converting the inspection flight distance in the optimization target into the fitness value;

[0080] S323: adopting a roulette wheel selection method, selecting a corresponding chromosome to enter the next generation population according to the fitness value ratio of each chromosome, performing a crossover operation on the selected chromosome and mutating the path sequence code corresponding to the chromosome with a preset probability; wherein the mutation includes a numerical change and a position exchange of binary characters in the binary code;

[0081] S324: Repeat the above steps of calculating the fitness value until the preset number of iterations is reached or the fitness value of the optimal chromosome in the population meets the threshold requirement. At this time, the path sequence code currently corresponding to the optimal chromosome is output, and the path sequence code is decoded to obtain the optimal number of inspection image shooting points for the target inspection drone for the target inspection task.

[0082] In this embodiment, after obtaining the target inspection task distributed by the UAV road and bridge inspection task list, and using the inspection execution association information and inspection task execution requirements in the target inspection task to determine the maximum flight altitude information of each road and bridge trajectory point in the target inspection task, the inspection trajectory execution strategy of the target inspection UAV for the target inspection task is optimized and solved based on the maximum flight altitude information, the target inspection period and the constraint condition set determined by the image overlap rate and the optimization goal of minimizing the inspection flight distance. The inspection flight and image shooting action of the target inspection UAV are executed by using the inspection trajectory execution strategy. By extending each inspection image shooting point of the target inspection UAV to the path trajectory point, it is not limited to flying along the road and bridge path trajectory. On the one hand, it is possible to obtain road surface images of multiple path trajectory points through one image shooting action, adapt to the inspection image acquisition requirements of the winding road and bridge paths in mountainous areas, and improve the efficiency of UAV road and bridge inspection. On the other hand, it is not limited to flying along the road and bridge path trajectory, which creates a wider path planning area, which can further optimize the rationality and superiority of the UAV flight path and balance the contradiction between the efficiency of UAV road and bridge inspection and the quality of inspection shooting images.

[0083] In a preferred embodiment, the inspection trajectory execution strategy is sent to the UAV road and bridge inspection control terminal to enable the target inspection UAV to perform the inspection flight and image shooting action steps of the target inspection task, specifically including:

[0084] S410: Sending the inspection trajectory execution strategy to the UAV road and bridge inspection control terminal, so that the UAV road and bridge inspection control terminal converts a number of inspection image shooting points in the inspection trajectory execution strategy into UAV flight control instructions;

[0085] S420: Sending the UAV flight control instruction to the target inspection UAV, so that the target inspection UAV performs an inspection flight and image shooting action including a plurality of inspection image shooting points.

[0086] On this basis, the actual inspection shooting time of the target inspection drone at each inspection image shooting point is obtained. Based on the actual inspection shooting time and the theoretical inspection shooting time, it is determined whether the target inspection task is delayed. If so, the inspection trajectory execution strategy steps are regenerated, specifically including:

[0087] S510: Obtaining the actual inspection shooting time of the target inspection drone after completing the image shooting action of each inspection image shooting point, and determining whether the time difference between the actual inspection shooting time and the theoretical inspection shooting time of the inspection image shooting point exceeds a preset difference value;

[0088] S520: If yes, take the remaining cruise period as the target cruise period, take the remaining road and bridge trajectory points as the road and bridge trajectory points of the target inspection task, return to execute the optimization solution of the inspection trajectory execution strategy step of the target inspection UAV for the target inspection task, and regenerate the inspection trajectory execution strategy.

[0089] In this embodiment, the task delay judgment is realized by comparing the actual inspection shooting time of each inspection image shooting point with the theoretical inspection shooting time, and the inspection trajectory execution strategy for the remaining inspection tasks is updated in real time. This can solve the delays caused by emergencies during the drone inspection process and enhance the environmental adaptability of the drone in different environments and conditions.

[0090] Reference Figure 2 , Figure 2 FIG. 1 is a schematic diagram of the structure of a road and bridge inspection data processing server based on a drone according to an embodiment of the present invention. Figure 2 As shown, the road and bridge inspection data processing server based on drones includes:

[0091] The acquisition module 10 is used to acquire the target inspection task distributed by the UAV road and bridge inspection task list; wherein the target inspection task includes inspection execution association information and inspection task execution requirements;

[0092] A generation module 20 is used to extract the identification information of the target inspection area and the target inspection drone in the inspection execution association information and the target inspection period, image resolution requirement and image overlap rate in the inspection task execution requirements, and use the target inspection area and the pre-trained road and bridge disease grade prediction model to predict the road and bridge disease grade of each road and bridge trajectory point in the target inspection area, and generate the maximum shooting height information of each road and bridge trajectory point based on the road and bridge disease grade and the identification information of the target inspection drone;

[0093] The solution module 30 is used to determine the maximum flight height information of each road and bridge trajectory point according to the maximum shooting height information and the image resolution requirement, and optimize and solve the inspection trajectory execution strategy of the target inspection UAV for the target inspection task based on the maximum flight height information, the constraint condition set determined by the target inspection period and the image overlap rate and the optimization goal of minimizing the inspection flight distance; wherein the inspection trajectory execution strategy includes the shooting heights and theoretical inspection shooting times of several inspection image shooting points;

[0094] An execution module 40 is used to send the inspection trajectory execution strategy to the UAV road and bridge inspection control terminal, so that the target inspection UAV performs the inspection flight and image shooting actions of the target inspection task;

[0095] The judgment module 50 is used to obtain the actual inspection shooting time of the target inspection drone at each inspection image shooting point, and based on the actual inspection shooting time and the theoretical inspection shooting time, judge whether the target inspection task is delayed. If so, regenerate the inspection trajectory execution strategy.

[0096] Other embodiments or specific implementation methods of the drone-based road and bridge inspection data processing server of the present invention can refer to the above-mentioned method embodiments and will not be repeated here.

[0097] It is understood that, in the description of this specification, the description with reference to the terms "one embodiment", "another embodiment", "other embodiments", or "first to Nth embodiments" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0098] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0099] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for processing road and bridge inspection data based on drones, characterized in that: include: Obtain the target inspection task distributed by the UAV road and bridge inspection task list; wherein the target inspection task includes inspection execution association information and inspection task execution requirements; Extract the identification information of the target inspection area and the target inspection drone in the inspection execution association information and the target inspection period, image resolution requirement and image overlap rate in the inspection task execution requirements; based on the target inspection area, extract a number of road and bridge trajectory points of each road and bridge path in the road and bridge trajectory electronic map; match the road and bridge disease impact feature set of each road and bridge trajectory point in the road and bridge disease impact feature database according to the location information of each road and bridge trajectory point; access the road and bridge disease case database, extract the road and bridge disease grade and road and bridge disease impact feature of each road and bridge disease case record stored in the road and bridge disease case database, use the road and bridge disease grade and the road and bridge disease impact feature to construct road and bridge disease training samples and train the neural network model, and obtain a trained road and bridge disease recognition model; input the road and bridge disease impact feature set into the trained road and bridge disease grade prediction model, obtain the road and bridge disease grade of each road and bridge trajectory point output by the road and bridge disease grade prediction model, and generate the maximum shooting height information of each road and bridge trajectory point based on the road and bridge disease grade and the identification information of the target inspection drone; According to the maximum shooting height information and elevation information, the maximum flight height information of each road and bridge trajectory point is determined. Based on the maximum flight height information, the constraint condition set determined by the target inspection period and the image overlap rate and the optimization goal of minimizing the inspection flight distance, the inspection trajectory execution strategy of the target inspection UAV for the target inspection task is optimized and solved; wherein the inspection trajectory execution strategy includes the shooting heights and theoretical inspection shooting times of several inspection image shooting points; Specifically, it includes: using the maximum shooting height information and the elevation information of each road and bridge trajectory point in the geographic elevation database to determine the maximum flight height information of the target patrol drone at each road and bridge trajectory point; the sum of the spatial areas of each road and bridge trajectory point within the preset spatial distance range constitutes the patrol image shooting spatial area, and patrol image shooting points are selected within the patrol image shooting spatial area, with the flight height of the target patrol drone at each patrol image shooting point being less than the maximum flight height of each road and bridge trajectory point contained in the shooting image as the first constraint condition, with the overlap rate of the road and bridge trajectory points of the target cruise drone in the shooting images of two adjacent patrol image shooting points being higher than the image overlap rate as the second constraint condition, and with the target patrol drone at each patrol image shooting point being less than the maximum flight height of each road and bridge trajectory point contained in the shooting image as the second constraint condition. The third constraint condition is that the completion time of the target inspection task determined by the cumulative sum of the standard shooting time of the inspection drone at each inspection image shooting point and the flight time of two adjacent inspection image shooting points is shorter than the target inspection period. The optimization goal is to minimize the sum of the inspection flight distances of the target inspection drone at two adjacent inspection image shooting points. A genetic optimization algorithm is used to optimize and solve the optimal number of inspection image shooting points for the target inspection drone for the target inspection task; based on several inspection image shooting points and the standard shooting time of the target inspection drone at each inspection image shooting point and the flight time of two adjacent inspection image shooting points, the shooting height and theoretical inspection shooting time of each inspection image shooting point are determined to generate an inspection trajectory execution strategy; The inspection trajectory execution strategy is sent to the UAV road and bridge inspection control terminal, so that the target inspection UAV performs the inspection flight and image shooting actions of the target inspection task; The actual inspection shooting time of the target inspection drone at each inspection image shooting point is obtained, and based on the actual inspection shooting time and the theoretical inspection shooting time, it is determined whether the target inspection task is delayed. If so, the inspection trajectory execution strategy is regenerated.

2. The method for processing road and bridge inspection data based on an unmanned aerial vehicle according to claim 1, characterized in that: Before obtaining the target inspection task steps for the UAV road and bridge inspection task list distribution, it also includes: Obtaining the road and bridge disease image demand information sent by the demand terminal; wherein the road and bridge disease image demand information includes the area to which the road and bridge disease image belongs, the image resolution requirement and the image overlap rate; The road and bridge disease image demand information is published, and the road and bridge inspection execution information for the road and bridge disease image demand information sent by the inspection terminal is received; wherein the road and bridge inspection execution information includes the target inspection period and the identification information of the target inspection drone; Feeding back the road and bridge inspection execution information to the demand terminal, and upon receiving confirmation information from the demand terminal, determining the target inspection area according to the area to which the road and bridge disease image belongs; The identification information of the target inspection area and the target inspection drone is used as the inspection execution association information, the target inspection period, image resolution requirement and image overlap rate are used as the inspection task execution requirements, and the target inspection task is generated in the drone road and bridge inspection task list.

3. The method for processing road and bridge inspection data based on an unmanned aerial vehicle according to claim 1, characterized in that: Based on the road and bridge disease level and the identification information of the target inspection drone, the steps of generating the maximum shooting height information of each road and bridge trajectory point include: Based on the road and bridge damage level of each road and bridge trajectory point and the image resolution requirements of UAV road and bridge inspection for different road and bridge damage levels, the image resolution requirement information of each road and bridge trajectory point is determined; The maximum shooting height information of each road and bridge trajectory point is calculated according to the camera parameters determined in the camera parameter library based on the image resolution requirement information and the identification information of the target inspection drone.

4. The method for processing road and bridge inspection data based on an unmanned aerial vehicle according to claim 1, characterized in that: The steps of optimizing and solving the optimal several inspection image shooting points of the target inspection UAV for the target inspection task by using the genetic optimization algorithm include: The inspection image shooting points selected by the target inspection UAV in the inspection image shooting space area are binary-coded in sequence to obtain a number of path sequence codes; wherein each spatial area point selected as the inspection image shooting point is assigned 1, and each spatial area point not selected is assigned 0; Based on the plurality of path sequence codes, a number M of chromosomes satisfying the first constraint condition, the second constraint condition and the third constraint condition are randomly generated, and a fitness value is calculated for each chromosome; wherein the fitness function for calculating the fitness value is configured as a preset operation function for converting the inspection flight distance in the optimization target into the fitness value; A roulette wheel selection method is adopted to select the corresponding chromosomes into the next generation population according to the fitness value ratio of each chromosome, and a crossover operation is performed on the selected chromosomes and the path sequence encoding corresponding to the chromosome is mutated with a preset probability; wherein the mutation includes the value change and position exchange of the binary characters in the binary encoding; Repeat the above steps of calculating the fitness value until the preset number of iterations is reached or the fitness value of the optimal chromosome in the population meets the threshold requirement. At this time, the path sequence code currently corresponding to the optimal chromosome is output, and the path sequence code is decoded to obtain the optimal several inspection image shooting points for the target inspection UAV for the target inspection task.

5. The method for processing road and bridge inspection data based on an unmanned aerial vehicle according to claim 1, characterized in that: The inspection trajectory execution strategy is sent to the UAV road and bridge inspection control terminal to enable the target inspection UAV to perform the inspection flight and image shooting action steps of the target inspection task, specifically including: The inspection trajectory execution strategy is sent to the UAV road and bridge inspection control terminal, so that the UAV road and bridge inspection control terminal converts a number of inspection image shooting points in the inspection trajectory execution strategy into UAV flight control instructions; The UAV flight control instruction is sent to the target inspection UAV, so that the target inspection UAV performs an inspection flight and image shooting action including a plurality of inspection image shooting points.

6. The method for processing road and bridge inspection data based on an unmanned aerial vehicle according to claim 1, characterized in that: The actual inspection shooting time of the target inspection drone at each inspection image shooting point is obtained, and based on the actual inspection shooting time and the theoretical inspection shooting time, it is determined whether the target inspection task is delayed. If so, the inspection trajectory execution strategy steps are regenerated, specifically including: Obtain the actual inspection shooting time of the target inspection drone after completing the image shooting action at each inspection image shooting point, and determine whether the time difference between the actual inspection shooting time and the theoretical inspection shooting time of the inspection image shooting point exceeds a preset difference value; If so, take the remaining cruise period as the target cruise period, take the remaining road and bridge trajectory points as the road and bridge trajectory points of the target inspection task, return to execute the optimization solution of the inspection trajectory execution strategy step of the target inspection UAV for the target inspection task, and regenerate the inspection trajectory execution strategy.

7. A road and bridge inspection data processing server based on drones, characterized in that: include: An acquisition module is used to acquire a target inspection task distributed by a UAV road and bridge inspection task list; wherein the target inspection task includes inspection execution association information and inspection task execution requirements; A generation module is used to extract the identification information of the target inspection area and the target inspection drone in the inspection execution association information and the target inspection period, image resolution requirement and image overlap rate in the inspection task execution requirements, and based on the target inspection area, extract a number of road and bridge trajectory points of each road and bridge path in the road and bridge trajectory electronic map, and match the road and bridge disease impact feature set of each road and bridge trajectory point in the road and bridge disease impact feature database according to the location information of each road and bridge trajectory point; access the road and bridge disease case database, extract the road and bridge disease case points stored in the road and bridge disease case database, and generate a generation module, which is used to extract the road and bridge disease impact feature set of each road and bridge trajectory point in the road and bridge disease case database, and generate a generation module, which is used to extract the road and bridge disease impact feature set of each road and bridge trajectory point in the road and bridge disease case database, and extract the road and bridge disease case points stored in the road and bridge disease case database. The road and bridge disease grade and road and bridge disease impact characteristics of each road and bridge disease case record are stored, and the road and bridge disease training samples and neural network model training are constructed using the road and bridge disease grade and the road and bridge disease impact characteristics to obtain a trained road and bridge disease recognition model; the road and bridge disease impact feature set is input into the trained road and bridge disease grade prediction model to obtain the road and bridge disease grade of each road and bridge trajectory point output by the road and bridge disease grade prediction model, and the maximum shooting height information of each road and bridge trajectory point is generated based on the road and bridge disease grade and the identification information of the target inspection drone; A solution module is used to determine the maximum flight height information of each road and bridge trajectory point based on the maximum shooting height information and the elevation information, and optimize the inspection trajectory execution strategy of the target inspection UAV for the target inspection task based on the maximum flight height information, the constraint condition set determined by the target inspection period and the image overlap rate and the optimization goal of minimizing the inspection flight distance; wherein the inspection trajectory execution strategy includes the shooting heights and theoretical inspection shooting times of several inspection image shooting points; Specifically, it includes: using the maximum shooting height information and the elevation information of each road and bridge trajectory point in the geographic elevation database to determine the maximum flight height information of the target patrol drone at each road and bridge trajectory point; the sum of the spatial areas of each road and bridge trajectory point within the preset spatial distance range constitutes the patrol image shooting spatial area, and patrol image shooting points are selected within the patrol image shooting spatial area, with the flight height of the target patrol drone at each patrol image shooting point being less than the maximum flight height of each road and bridge trajectory point contained in the shooting image as the first constraint condition, with the overlap rate of the road and bridge trajectory points of the target cruise drone in the shooting images of two adjacent patrol image shooting points being higher than the image overlap rate as the second constraint condition, and with the target patrol drone at each patrol image shooting point being less than the maximum flight height of each road and bridge trajectory point contained in the shooting image as the second constraint condition. The third constraint condition is that the completion time of the target inspection task determined by the cumulative sum of the standard shooting time of the inspection drone at each inspection image shooting point and the flight time of two adjacent inspection image shooting points is shorter than the target inspection period. The optimization goal is to minimize the sum of the inspection flight distances of the target inspection drone at two adjacent inspection image shooting points. A genetic optimization algorithm is used to optimize and solve the optimal number of inspection image shooting points for the target inspection drone for the target inspection task; based on several inspection image shooting points and the standard shooting time of the target inspection drone at each inspection image shooting point and the flight time of two adjacent inspection image shooting points, the shooting height and theoretical inspection shooting time of each inspection image shooting point are determined to generate an inspection trajectory execution strategy; An execution module is used to send the inspection trajectory execution strategy to the UAV road and bridge inspection control terminal, so that the target inspection UAV performs the inspection flight and image shooting actions of the target inspection task; The judgment module is used to obtain the actual inspection shooting time of the target inspection drone at each inspection image shooting point, and based on the actual inspection shooting time and the theoretical inspection shooting time, judge whether the target inspection task is delayed. If so, regenerate the inspection trajectory execution strategy.

Images