Route data processing method and device, storage medium and unmanned aerial vehicle

Abstract

The application discloses a route data processing method and device, a storage medium and a UAV. The UAV comprises an image acquisition device. The method comprises the following steps: during flight of the UAV along a preset task route, the image acquisition device is used to shoot multiple images at multiple observation points; each image is identified to determine whether a target object exists in the image shot at each observation point; for each observation point, if the target object does not exist in all the images shot at the observation point, the observation point is deleted or the shooting field of view parameter of the observation point is adjusted in the task route, so as to update the task route. The above scheme can exclude some useless areas, and improve the efficiency of material collection and the quality of materials.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) technology, specifically to a method, apparatus, storage medium, and UAV for processing flight route data. Background Technology

[0002] AI recognition technology plays a crucial role in practical production applications. For example, in the construction industry, AI recognition can identify personnel in dangerous areas of construction sites in real time, effectively preventing safety accidents. Furthermore, it can automate defect monitoring of building components, significantly improving the efficiency of engineering quality inspection. However, common drone data collection routes often inadvertently collect data from areas without target objects, and manually planning routes to exclude these useless areas is difficult and subjective. Collecting data from these areas adds a lot of useless training material, and manual screening wastes a significant amount of time and effort. Summary of the Invention

[0003] The purpose of this application is to provide a method, apparatus, storage medium, and drone for processing flight route data, in order to solve the technical problem of low quality and efficiency of drones in collecting training materials in the prior art.

[0004] To achieve the above objectives, the first aspect of this application provides a flight path data processing method, which is applied to an unmanned aerial vehicle (UAV). The UAV includes an image acquisition device, and the method includes: As the drone flies along a pre-set mission route, it captures multiple images at multiple observation points using image acquisition equipment. Each image is identified to determine whether a target object exists in the image taken at each observation point; For each observation point, if no target object is found in any of the images taken at that observation point, the observation point is deleted from the mission flight path or the shooting field of view parameters of the observation point are adjusted to update the mission flight path.

[0005] In embodiments of this application, the method further includes: acquiring shooting information for each image, the shooting information including shooting location and shooting time; sorting all images according to shooting time; for any first image and second image with adjacent shooting time order, if the distance between the shooting locations of the first image and the second image is less than a preset distance threshold, fusing the observation point corresponding to the first image and the observation point corresponding to the second image to obtain a fused observation point; and replacing the observation points of the first image and the second image in the mission flight path with the fused observation point to update the mission flight path.

[0006] In the embodiments of this application, identifying each image to determine whether a target object exists in the image captured at each observation point includes: storing all images and the capturing information of all images in a database; pushing the storage address of all images to an identification queue; and detecting each image in the identification queue based on an image detection model to obtain the identification result of each image, the identification result including whether a target object exists in the image, the object type of the target object, and the confidence level.

[0007] In embodiments of this application, the method further includes: pushing the recognition results of images with a confidence level greater than a first preset threshold to a result queue; storing each recognition result in the result queue to a database and associating it with the image corresponding to the recognition result; updating the recognition quantity of the preset route inspection task using a distributed lock; and updating the recognition status of the preset route inspection task to "completed" when the recognition quantity is greater than or equal to the total number of images.

[0008] In embodiments of this application, the method further includes: determining a first target image selected by the user on a display interface, and a first target region of the first target image selected by the user based on a annotation tool on the display interface; acquiring input data from the user for the first target region; updating the recognition result of the first target image based on the input data; determining a second target image selected by the user on a display interface, and a second target region of the second target image selected by the user based on a blur tool on the display interface; blurring the second target region to obtain a processed second target image; and performing iterative training based on the processed second target image to update the recognition result of the second target image.

[0009] In embodiments of this application, the method further includes: determining the number of images in the first target image and the second target image; and adjusting the model parameters of the image detection model when the number of images is greater than or equal to a second preset threshold, so as to identify the captured images through the adjusted image detection model.

[0010] In embodiments of this application, the method further includes: acquiring route information of the mission route, including the take-off point, route altitude, and round-trip speed of the UAV; marking the mission route based on the images captured by each observation point in the updated mission route; and displaying the marked mission route, the route information of the mission route, and the images captured by each observation point through a display interface.

[0011] A second aspect of this application provides a route data processing apparatus, comprising: The memory is configured to store instructions; The processor is configured to retrieve instructions from memory and, when executing instructions, to implement the aforementioned route data processing method.

[0012] A third aspect of this application provides an unmanned aerial vehicle (UAV) including an image acquisition device and a flight path data processing device according to the above description.

[0013] A fourth aspect of this application provides a machine-readable storage medium storing instructions that cause a machine to perform the above-described route data processing method.

[0014] The above technical solution involves capturing multiple images at various observation points while the UAV flies along a preset mission route. Each image is then analyzed to determine if a target object exists within the images captured at each observation point. If no target object is found in any of the images captured at that observation point, the observation point is either deleted from the mission route or its field-of-view parameters are adjusted to update the mission route. This solution eliminates useless areas, improving both the efficiency and quality of data acquisition.

[0015] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings: Figure 1 The illustration shows a flowchart of a route data processing method according to an embodiment of this application; Figure 2 This illustration schematically shows a flowchart of the task execution process for media image recognition according to an embodiment of this application; Figure 3 A schematic diagram illustrating the initial mission path according to an embodiment of this application is shown; Figure 4 This illustration shows a schematic diagram of an AI training data acquisition route according to an embodiment of this application; Figure 5 The schematic diagram illustrates a flowchart of a route data processing method according to a specific embodiment of this application; Figure 6 This schematic diagram illustrates a structural block diagram of a flight route data processing apparatus according to an embodiment of the present application; Figure 7 The illustration shows a schematic diagram of the structure of a computer device according to an embodiment of the present application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0018] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0019] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0020] Figure 1 The illustration shows a schematic flowchart of a flight route data processing method according to an embodiment of this application. Figure 1 As shown in the figure, this application provides a route data processing method, which is applied to an unmanned aerial vehicle (UAV). The method may include the following steps.

[0021] S102, during the flight of the UAV along the preset mission route, takes multiple images at multiple observation points through image acquisition equipment.

[0022] Specifically, a preset mission route refers to the flight path along which the drone needs to complete a specific task. Drones are typically equipped with image acquisition devices. As the drone flies along the designated route, these devices collect image data along that route. This image data can be incorporated into AI training materials for training AI models. Users can select the desired route type based on their needs, creating routes such as linear patrols, strip routes, elevation patrols, and area patrols. Each route possesses unique spatial coverage characteristics and data acquisition advantages, enabling the acquisition of rich visual information from different angles and heights to meet diverse AI material acquisition needs. In essence, technicians can pre-set or pre-calculate the points along the mission route as observation points for capturing image data. The image acquisition device can capture multiple images at each observation point.

[0023] S104, identify each image to determine whether a target object exists in the image taken at each observation point.

[0024] Specifically, the target object can be a designated person, open flame, vehicle, smoke, crack, pipe, etc. Image recognition can select the required functional modules from a pre-set AI algorithm library. These algorithms cover multiple fields such as personnel recognition, open flame recognition, vehicle recognition, smoke recognition, crack recognition, and pipe leak recognition, to adapt to different application scenarios such as security monitoring, industrial inspection, and fire early warning. At each observation point, the image acquisition device can acquire multiple images. Based on the recognition results of each image, it can be determined whether a target object exists in the image captured at that observation point.

[0025] In the embodiments of this application, identifying each image to determine whether a target object exists in the image captured at each observation point includes: storing all images and the capturing information of all images in a database; pushing the storage address of all images to an identification queue; and detecting each image in the identification queue based on an image detection model to obtain the identification result of each image, the identification result including whether a target object exists in the image, the object type of the target object, and the confidence level.

[0026] Specifically, all images collected after the mission's flight along the designated route, along with their capture information, can be stored in a database. When media photos are transferred to the storage server, key parameters such as gimbal pitch angle, flight yaw angle, capture time, latitude and longitude, capture altitude, and focal length are extracted by parsing the XMP information and other metadata stored by the UAV. These parameters are then stored in the database. Once all mission media photos have been transferred, they are automatically sent in batches to a neural network model for analysis. Specifically, the storage addresses of all images are pushed to a Kafka recognition queue. Each image in the queue is then detected using an image detection model to obtain a recognition result. The recognition result includes whether a target object exists in the image, the object type, and the confidence level. The system consumes the Kafka result queue and automatically stores the recognition results (including the object type and location data) into the system database, linking them to the original media files for subsequent data querying, analysis, and model optimization.

[0027] S106, for each observation point, if no target object is found in any of the images taken at the observation point, delete the observation point or adjust the shooting field of view parameters of the observation point in order to update the mission flight path.

[0028] Understandably, after completing a material collection task, the system automatically generates an AI training data collection route, i.e., the task route. In the initial stage, due to a lack of experience, users often define a large area for material collection. After actual scanning, it will be found that many media photos taken at observation points do not contain the target object. Therefore, when executing the task route again, this observation point can be ignored, thus allowing for adjustments to the task route. The system will analyze the recognition results of each observation point. If it finds that the media photos taken at a certain observation point do not contain the target object, that observation point can be deleted from the generated task route, directly eliminating some useless areas. On the other hand, task routes and route actions can also be added to locations that may have potential problems, thereby improving the efficiency and quality of material collection.

[0029] The above scheme involves capturing multiple images from various observation points while the UAV flies along a pre-set mission route. Each image is then analyzed to determine if a target object exists within the images captured at each observation point. If no target object is found in any of the images captured at that observation point, the observation point is either removed from the mission route or its field-of-view parameters are adjusted to update the mission route. This scheme eliminates useless areas, improving the efficiency and quality of data acquisition.

[0030] In embodiments of this application, the method further includes: acquiring shooting information for each image, the shooting information including shooting location and shooting time; sorting all images according to shooting time; for any first image and second image with adjacent shooting time order, if the distance between the shooting locations of the first image and the second image is less than a preset distance threshold, fusing the observation point corresponding to the first image and the observation point corresponding to the second image to obtain a fused observation point; and replacing the observation points of the first image and the second image in the mission flight path with the fused observation point to update the mission flight path.

[0031] Specifically, the shooting location can be defined by the shooting latitude, longitude, and altitude. All photo records taken for this mission are retrieved from the database, and the media photos are sorted by shooting time. Adjacent photos are matched, comparing whether the shooting latitude and longitude of two consecutive media photos are equal and whether the difference in shooting altitude is within a preset distance threshold. If the distance between shooting locations is less than the preset distance threshold, the observation points corresponding to the two images are merged to obtain a merged observation point. Specifically, the merging method can be to retain any one of the observation points or to take the midpoint between the shooting locations of the two observation points. This updates the observation points in the mission flight path, thereby updating the mission flight path. Specifically, while updating the mission flight path, the unupdated mission flight path can be retained. The updated mission flight path is used in the next flight mission. For example, if the interval between two shooting locations is within 0.05m (during processing, the shooting latitude and longitude must be retained to five decimal places, as there may be media photos from the same shooting point, but the latitude and longitude may deviate after five decimal places), they are classified as the same observation point, and all media photos are traversed according to this rule. Based on the matching results, a fusion observation point is generated based on each shooting latitude, longitude and shooting height, and the first and second images before fusion are bound to the corresponding fusion observation point.

[0032] In embodiments of this application, the method further includes: pushing the recognition results of images with a confidence level greater than a first preset threshold to a result queue; storing each recognition result in the result queue to a database and associating it with the image corresponding to the recognition result; updating the recognition quantity of the preset route inspection task using a distributed lock; and updating the recognition status of the preset route inspection task to "completed" when the recognition quantity is greater than or equal to the total number of images.

[0033] Specifically, the storage addresses of all images are pushed to the Kafka recognition queue. Each image in the recognition queue is detected using an image detection model to obtain a recognition result for each image. The recognition result includes whether a target object exists in the image, the object type, and the confidence level. To reduce the probability of false recognition, the recognized information is only pushed to the result queue when the confidence level is greater than a first preset threshold. The system consumes the Kafka result queue, automatically storing the recognition results (including the object type and location data) into the system database and associating them with the original media files for subsequent data querying, analysis, and model optimization. For each parsed recognition result message, a distributed lock increments the AI ​​recognition count. The preset flight path inspection task refers to the image acquisition and recognition tasks required for each flight path. Once all media photos acquired during the flight have been processed, the recognition status of the preset flight path inspection task is updated to "completed."

[0034] In one specific embodiment, when creating a flight mission for AI training, the system defaults to enabling the recognition algorithm, and this setting cannot be changed. Based on the actual application scenario, the required functional modules are selected from a preset AI algorithm library. These algorithms cover multiple fields such as personnel recognition, open flame recognition, vehicle recognition, smoke recognition, crack recognition, and pipe leak recognition, to adapt to different application scenarios such as security monitoring, industrial inspection, and fire early warning. After all media photos for the mission are dumped, they are automatically sent in batches to the neural network model for analysis and processing. For media images that identify target objects, the model accurately outputs the category of the target object and the location information of the object's area, using a standardized annotation format: (target category, x-coordinate of the area center point, y-coordinate of the area center point, area width, area height). In the embodiments of this application, a two-stage training strategy can be used to construct the target detection system. Specifically, firstly, the neural network recognition model is pre-trained based on a public dataset and a small amount of privately labeled data to obtain an initial model with basic detection capabilities. Then, the pre-trained model is deployed to the actual business scenario to perform preliminary target detection on the mission-related media images, and the recognition results serve as the basis for subsequent optimization. Figure 2The flowchart shown illustrates the initial recognition process for media images in a task, using a pre-trained model. Information such as the storage address of the media photos is pushed to a Kafka recognition queue. The AI ​​model consumes the media photos to be recognized from the queue and pushes the recognition results of the completed photos to the Kafka recognition result queue. To reduce the probability of false recognition, recognized information is only pushed to the result queue when the confidence level is greater than a system preset value. The system consumes the Kafka result queue and automatically stores the recognition results (including target type and location data) into the system database, associating them with the original media files for subsequent data querying, analysis, and model optimization. For each recognition result message parsed, a distributed lock increments the task's AI recognition count. When all media photos in the task have been processed, the task's AI recognition status is updated to "completed." Preprocessing using the AI ​​model compensates for the time-consuming and inefficient nature of manual annotation. For simple target annotation, a lot of manpower is often required. AI models can annotate most target objects, thereby avoiding the decline in annotation accuracy caused by long-term, high-intensity manual annotation work, and reducing the negative impact on image analysis, model training and other work based on the dataset.

[0035] In embodiments of this application, the method further includes: determining a first target image selected by the user on a display interface, and a first target region of the first target image selected by the user based on a annotation tool on the display interface; acquiring input data from the user for the first target region; updating the recognition result of the first target image based on the input data; determining a second target image selected by the user on a display interface, and a second target region of the second target image selected by the user based on a blur tool on the display interface; blurring the second target region to obtain a processed second target image; and performing iterative training based on the processed second target image to update the recognition result of the second target image.

[0036] It is understood that both the first and second target images are images selected by the user on the display interface. The first target image can be selected using the annotation tool, and the second target image can be selected using the blur tool. The first target region is the area selected by the user for the first target image, and the second target region is the area selected by the user for the second target image. The user can input data for the first target region using the annotation tool. This input data can include the object type, object position, region width, region height, etc., as perceived by the user. After receiving the input data, the processor can replace the AI ​​recognition result of the first target image with the user's input data for manual correction. The user can use the blur tool to select the second target region of the second target image and blur that region to obtain a processed second target image. The processed second target image is equivalent to losing the image data of the second target region, thus allowing for iterative training based on the processed second target image to update the recognition result of the second target image.

[0037] In the embodiments of this application, the display interface can display the mission flight path. After the UAV completes a full mission flight path, the observation points along the mission flight path will change. The processor can then mark the mission flight path based on the images captured by each observation point in the updated mission flight path, and display the marked mission flight path, the mission flight path information, and the images captured by each observation point through the display interface. Specifically, after media photos are pre-identified by the AI ​​model, the image recognition information is displayed intuitively. This includes the type corresponding to each marked area (e.g., people, vehicles), and the statistics of the number of marks for each type (e.g., people: 15, vehicles: 8). Due to the differences in the AI ​​model's ability to recognize each type, there will be a certain number of incorrect marks (misidentifying type A as type B) and missing marks (not recognizing the actual target). Therefore, the display interface provides manual correction tools, annotation tools, and blurring tools. The manual correction tool allows manual adjustment of incorrect mark types, changing the classification label of the target object, and removing misidentified marked areas. Key targets that are difficult for the AI ​​model to recognize are often key samples for improving model accuracy. The annotation tool supports manually defining the target area and specifying the correct category for the defined area. Furthermore, for small areas, the image supports zooming in, zooming out, and dragging, making it easy to circle and label these areas. After submission, the system stores the coordinates and label categories of the manually labeled areas in the database and links them to the media photo. Additionally, media photos often contain areas that are difficult to label or have densely packed objects. If these areas are not handled correctly, they are often treated as negative samples by the model, affecting its accuracy. To address this, a blurring tool is provided, allowing selective blurring of these problematic areas and the option to delete blurred areas via right-click. After manual labeling is completed, the image is submitted to the algorithm model. The system converts the location and category information of all labels on the media image into a TXT file, and then submits the media image and the TXT file containing the labeling information to the AI ​​model training queue.

[0038] In embodiments of this application, the method further includes: determining the number of images in the first target image and the second target image; and adjusting the model parameters of the image detection model when the number of images is greater than or equal to a second preset threshold, so as to identify the captured images through the adjusted image detection model.

[0039] It's understandable that annotation and blurring tools are used to correct the recognition results of images. The preset threshold is a set value for the model parameters adjusted by technicians to trigger the AI ​​recognition model. When a certain number of annotated or blurred images are submitted, model fine-tuning is automatically triggered. Through continuous iterative training, the AI ​​model's recognition ability in complex scenes can be significantly improved, and the model's recognition accuracy for specific categories can be increased.

[0040] In embodiments of this application, the method further includes: acquiring route information of the mission route, including the take-off point, route altitude, and round-trip speed of the UAV; marking the mission route based on the images captured by each observation point in the updated mission route; and displaying the marked mission route, the route information of the mission route, and the images captured by each observation point through a display interface.

[0041] Specifically, the display interface can show the mission flight path. After the drone completes a full mission flight path, the observation points along the path will change. The processor can then mark the mission flight path based on the images captured by each observation point in the updated path, and display the marked path, its information, and the images captured by each observation point on the interface, thus generating the AI ​​training data acquisition path. As shown in the figure, Figure 3 For the initial mission trajectory, Figure 4 The flight path is used to collect training data for the generated AI.

[0042] In the embodiments of this application, the high flexibility of UAVs in material collection (the ability to adjust flight paths as needed to adapt to complex terrain and environments), high data collection efficiency (large-area rapid collection and real-time data transmission), and rich data diversity (equipped with multiple sensors) are utilized. Combined with AI recognition models and manual processing, high-quality AI training materials are obtained for model training, generating an efficient AI training material collection flight path. This method achieves a closed-loop workflow of "collection-annotation-optimization." Figure 5As shown, a data collection route for AI training can be established based on different route types according to needs. Different AI recognition algorithms can be selected, such as personnel recognition, open flame recognition, vehicle recognition, smoke recognition, crack recognition, and pipe water leakage recognition, completing a basic first-round data collection function. After the drone completes a mission according to the planned route, it collects some data. Based on the selected AI recognition algorithm, the images collected by the drone undergo a basic first-round recognition, and the existing targets are labeled using the AI ​​model. After the model automatically identifies potential hazards, the hazards identified by the AI ​​model can be corrected or deleted manually. Unidentified areas are labeled, and areas interfering with recognition are blurred. After manual labeling, these are submitted as correct AI training material to the AI ​​model. When a certain amount of training material is collected, the AI ​​recognition model is fine-tuned. Based on the completed AI training route task, observation points are merged according to the shooting location of media photos. Media photos with the same shooting location are grouped into the same observation point, and the AI ​​training data collection route is updated. Users can exclude some useless areas and adjust the AI ​​training data collection route to retain observation points containing target objects. After adjusting the mission route, a new mission can be issued, generating new sorties based on the optimized route. The media photos captured in this new sortie are then processed by the AI ​​model for identification, resulting in a batch of higher-quality AI training material. Through this iterative mechanism, the system can significantly improve material acquisition efficiency, reduce invalid shots, substantially improve the quality of training material, focus on the target area, and achieve continuous improvement of the AI ​​model.

[0043] In the embodiments of this application, the generation process of the AI ​​training material acquisition route (hereinafter referred to as the AI ​​training route) is as follows: Figure 5 As shown, when media photos are transferred to the storage server, the XMP information and other metadata stored by the UAV are parsed to extract key parameters such as gimbal pitch angle, flight yaw angle, shooting time, shooting location (latitude and longitude), shooting altitude, and focal length of the photos. These parameters are then stored in the database. Once all media photos for the mission have been transferred, all photo records taken during the mission are retrieved from the database and sorted by shooting time. Observation point fusion processing is performed on adjacent photos. The latitude and longitude of consecutive media photos are compared to ensure they are equal and the difference in shooting altitude is within 0.05m. During processing, the latitude and longitude are retained to five decimal places. Even if media photos are taken from the same point, discrepancies may exist in the latitude and longitude after the fifth decimal place. These are grouped into the same observation point, and this rule is applied to all media photos. Based on the matching results, a waypoint is generated for each shooting latitude, longitude, and altitude, and the relevant media photos are bound to the corresponding fused observation point.

[0044] Each media photo corresponds to a shooting action. If several media photos are associated with a waypoint, then several shooting actions are added. The system acquires parameters such as gimbal pitch angle, aircraft yaw angle, and focal length corresponding to the media photos. Before each shooting action, the shooting parameters (gimbal pitch angle, aircraft yaw angle, focal length) are set. If the parameters of media photos at adjacent waypoints have not changed, the parameter settings of the previous shooting action are automatically adopted. After updating the observation points, combined with information such as the original flight path's takeoff point, flight path altitude, and round-trip speed, a complete AI training data collection flight path is finally formed.

[0045] This technical solution leverages the highly diverse nature of drone operations, overcoming limitations imposed by geographical environment, safety risks, or cost factors. It acquires multi-dimensional data that is difficult to collect using traditional methods and allows for dynamic incremental data collection based on environmental changes, continuously updating the training dataset. Users first select the flight path type and matching AI recognition algorithm based on their needs. After completing the aerial photography mission, the drone automatically performs AI preprocessing on the captured media footage, initially labeling target object areas. Subsequently, the recognition results are corrected and blurred through a user-interactive interface, and the optimized data is fed back to the AI ​​model for iterative training. Specifically, the system automatically generates optimized flight paths based on the shooting parameters of the media photos collected in each mission. Users can edit and adjust the flight paths based on the target object occurrence rate at observation points, significantly improving the success rate of target object acquisition and sample quality in subsequent missions. This method achieves a closed-loop workflow of "acquisition-labeling-optimization," improving acquisition efficiency by more than 40% compared to traditional methods.

[0046] Figure 1 This is a flowchart illustrating a route data processing method in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0047] Figure 6 A schematic block diagram of a flight route data processing apparatus according to an embodiment of this application is shown. Figure 6 As shown in the figure, this application provides a route data processing device, which may include: The memory is configured to store instructions; and The processor is configured to retrieve instructions from memory and, when executing instructions, to implement the aforementioned route data processing method.

[0048] Specifically, in this embodiment of the application, the processor can be configured as follows: As the UAV flies along the preset mission route, it takes multiple images at multiple observation points using image acquisition equipment. Each image is identified to determine whether a target object exists in the image taken at each observation point. If no target object is found in any of the images taken at each observation point, the observation point is deleted from the mission route or the shooting field of view parameters of the observation point are adjusted to update the mission route.

[0049] In embodiments of this application, the processor may also be configured to: Acquire the shooting information for each image, including the shooting location and shooting time; sort all images according to the shooting time; for any first and second images with adjacent shooting times, if the distance between the shooting locations of the first and second images is less than a preset distance threshold, fuse the observation points corresponding to the first image and the observation points corresponding to the second image to obtain fused observation points; replace the observation points of the first and second images in the mission flight path with the fused observation points to update the mission flight path.

[0050] In embodiments of this application, the processor may also be configured to: All images and their capture information are stored in a database; the storage addresses of all images are pushed to a recognition queue; each image in the recognition queue is detected based on an image detection model to obtain the recognition result for each image. The recognition result includes whether a target object exists in the image, the object type of the target object, and the confidence level.

[0051] In embodiments of this application, the processor may also be configured to: The recognition results of images with a confidence level greater than the first preset threshold are pushed to the result queue; each recognition result in the result queue is stored in the database and associated with the image corresponding to the recognition result; a distributed lock is used to update the recognition quantity of the preset route inspection task; if the recognition quantity is greater than or equal to the total number of images, the recognition status of the preset route inspection task is updated to completed.

[0052] In embodiments of this application, the processor may also be configured to: The system identifies the first target image selected by the user on the display interface, and the first target region of the first target image selected by the user using the annotation tool on the display interface; it acquires the user's input data for the first target region; it updates the recognition result of the first target image based on the input data; it identifies the second target image selected by the user on the display interface, and the second target region of the second target image selected by the user using the blur tool on the display interface; it blurs the second target region to obtain the processed second target image; and iteratively trains based on the processed second target image to update the recognition result of the second target image.

[0053] In embodiments of this application, the processor may also be configured to: Determine the number of images in the first target image and the second target image; if the number of images is greater than or equal to a second preset threshold, adjust the model parameters of the image detection model so that the captured images can be identified by the adjusted image detection model.

[0054] In embodiments of this application, the processor may also be configured to: Acquire the mission flight path information, including the UAV's takeoff point, flight path altitude, and round-trip speed; mark the mission flight path based on the images taken by each observation point in the updated mission flight path, and display the marked mission flight path, mission flight path information, and images taken by each observation point through the display interface.

[0055] This application also provides an unmanned aerial vehicle (UAV), which may include an image acquisition device and a flight path data processing device as described above.

[0056] This application also provides a machine-readable storage medium storing instructions that cause a machine to perform the above-described route data processing method.

[0057] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor A01, a network interface A02, a memory (not shown), and a database (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A04. The database stores flight route data processing data. The network interface A02 communicates with external terminals via a network connection. When the computer program B02 is executed by the processor A01, it implements a flight route data processing method.

[0058] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0059] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0060] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0062] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0063] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0064] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0065] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0066] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0067] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for processing flight route data, characterized in that, The flight path data processing method is applied to a drone, the drone including an image acquisition device, and the method includes: During the flight of the UAV along the preset mission route, the image acquisition device captures multiple images at multiple observation points; Each image is identified to determine whether a target object exists in the image taken at each observation point; If the target object is not present in any of the images captured at each observation point, the observation point is deleted from the mission flight path or the field of view parameters of the observation point are adjusted to update the mission flight path. The step of identifying each image to determine whether a target object exists in the image captured at each observation point includes: Store all images and their capture information in a database; Push the storage addresses of all images to the recognition queue; The recognition queue is detected based on an image detection model to obtain the recognition result of each image. The recognition result includes whether the target object exists in the image, the object type of the target object, and the confidence level.

2. The route data processing method according to claim 1, characterized in that, The method further includes: Acquire the shooting information for each image, including the shooting location and shooting time; Sort all images according to the shooting time; For any first image and second image that are adjacent in shooting time sequence, if the distance between the shooting positions of the first image and the second image is less than a preset distance threshold, the observation point corresponding to the first image and the observation point corresponding to the second image are fused to obtain a fused observation point. The observation points of the first and second images in the mission flight path are replaced with the fused observation points to update the mission flight path.

3. The route data processing method according to claim 1, characterized in that, The method further includes: The recognition results of images with a confidence level greater than the first preset threshold are pushed to the result queue; Each recognition result in the result queue is stored in the database and associated with the image corresponding to that recognition result; Distributed locks are used to update the number of identified items for the preset route inspection tasks; If the number of identified images is greater than or equal to the number of all images, the identification status of the preset route inspection task will be updated to "completed".

4. The route data processing method according to claim 1, characterized in that, The method further includes: Determine the first target image selected by the user on the display interface, and the first target region of the first target image selected by the user based on the annotation tool on the display interface; Obtain the user's input data for the first target area; The recognition result of the first target image is updated based on the input data; Determine the second target image selected by the user on the display interface, and the second target region of the second target image selected by the user based on the blur tool of the display interface; The second target region is blurred to obtain the processed second target image; Iterative training is performed based on the processed second target image to update the recognition result of the second target image.

5. The route data processing method according to claim 4, characterized in that, The method further includes: Determine the number of images in the first target image and the second target image; If the number of images is greater than or equal to a second preset threshold, the model parameters of the image detection model are adjusted so that the captured images can be identified by the adjusted image detection model.

6. The route data processing method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain the flight path information of the mission route, which includes the takeoff point, flight path altitude, and round-trip speed of the UAV; The mission route is marked based on the images taken by each observation point in the updated mission route, and the marked mission route, the route information of the mission route, and the images taken by each observation point are displayed on the display interface.

7. A route data processing device, characterized in that, include: The memory is configured to store instructions; The processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement the route data processing method according to any one of claims 1 to 6.

8. A drone, characterized in that, include: Image acquisition equipment; The route data processing device according to claim 7.

9. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions for causing the machine to perform the route data processing method according to any one of claims 1 to 6.

Images