An intelligent cruise system and method based on real scene image feedback

Abstract

The present application belongs to the technical field of unmanned aerial vehicle cruise, and particularly relates to an intelligent cruise system and method based on real scene image feedback. The present application can dynamically adapt to the needs of different cruise environments through real-time feedback and verification of real scene images collected during the cruise of the unmanned aerial vehicle, and when unknown cruise areas appear in the cruise path, the unknown areas can be deduced and completed according to known waypoints, the target area can be determined whether it is completely covered by combining the coverage completeness and feature density evaluation of the target type, and at the same time, when it is not completely covered, the image to be calibrated will be used to adaptively supplement, and when the image to be calibrated still cannot meet the coverage requirements, the supplementary cruise mechanism will be triggered, thereby effectively reducing the energy consumption and time cost caused by redundant flight, and achieving the purpose of improving the cruise efficiency of the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) cruise technology, specifically relating to an intelligent cruise system and method based on real-scene image feedback. Background Technology

[0002] With the widespread application of drone patrol technology, its use in fields such as power line inspection, environmental monitoring, and traffic management is increasing. It not only makes patrols more convenient and significantly improves efficiency, but also effectively reduces labor costs, thereby reducing safety risks in manual inspections and improving the accuracy and reliability of operations. However, in order to ensure the integrity of the patrol mission and the accuracy of the data, it is obviously necessary to perform real-time verification of the real-scene images collected during the drone patrol.

[0003] While some image recognition-based cruise verification methods exist in existing technologies, most rely on static image libraries or preset templates for comparison, lacking adaptability to dynamic environmental changes. When conducting cruise operations in unknown areas, drones are often dispatched for exploratory flights to collect preliminary geographic and visual data and construct a temporary feature library. The system then dynamically updates the cruise strategy, adjusting flight altitude and shooting angle to ensure multi-dimensional coverage of key target areas. However, this approach suffers from excessive reliance on exploratory flights, which not only increases the time and energy consumption in the early stages of the mission but also leads to blind spots in subsequent cruise coverage when initial data collection is insufficient, affecting the integrity of the overall mission. Therefore, this solution proposes an intelligent cruise method based on real-scene image feedback to address the aforementioned problems. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent cruise system and method based on real-scene image feedback, which can realize real-time feedback and verification of real-scene images during the cruise, timely supplement missing target type areas, and improve the integrity of cruise coverage and data accuracy.

[0005] The specific technical solution adopted by this invention is as follows:

[0006] A smart cruise method based on real-scene image feedback includes:

[0007] Receive a cruise mission request initiated by the user terminal, wherein the cruise mission request includes the target area, target type and demand feedback information;

[0008] Based on the geographical location information of the target area and the historical cruise records of the target area, a corresponding cruise path is generated, and cruise instructions are sent to the drone.

[0009] The system collects real-time images of the drone during its cruise and transmits them back in real time. It also matches the real-time images with the feedback information and marks the successfully matched real-time images as valid images, while recording the unmatched real-time images as images to be calibrated.

[0010] The geographic location information of the effective images is correlated with the spatial distribution of the target area to verify whether the effective images cover all target types. If not all target types are covered, feature extraction is performed on the images to be calibrated to supplement the images of the missing target types corresponding to the areas in the effective images.

[0011] If the image to be calibrated fails to fully fill in the missing target types, a supplementary patrol mechanism is triggered to conduct a second patrol of the uncovered area until all target types are covered and then the patrol stops.

[0012] In a preferred embodiment, receiving a cruise mission request initiated by a user terminal includes the following steps:

[0013] Obtain the target area input by the user and verify whether the target area matches the user's permission range;

[0014] If a target area exists within the user's permissions, allow the user to continue inputting the target type and feedback information.

[0015] If the target area does not exist within the user's authorized scope, return an error message to the user and terminate the current patrol mission request;

[0016] Based on the center coordinates of the target area, retrieve the mission execution cycle from the historical cruise records and determine whether the real-time cruise mission request is within the mission execution cycle.

[0017] If it is within the task execution period, the real-time patrol task is determined to be a patrol task within the period, and a historical path recommendation is sent for inquiry. After confirmation by the user, the historical patrol path is used as the patrol path for this time.

[0018] If it is not within the task execution cycle, it is determined to be a new patrol task, and a reminder to establish a new patrol task is sent simultaneously.

[0019] In a preferred embodiment, the step of generating a corresponding cruise route based on the geographic location information of the target area and the historical cruise records of the target area includes:

[0020] Collect the geographic coordinates of the target patrol points in the target area and perform a topological comparison with the historical waypoints in the historical patrol records;

[0021] When the spatial topology of the target patrol point is completely consistent with that of the historical waypoints, a comprehensive efficiency rating is performed on each historical cruise path, and the historical cruise path with the highest comprehensive efficiency rating is selected as the reuse path.

[0022] When there is a difference in the spatial topology between the target patrol point and the historical waypoint, the difference point is output synchronously, and an unknown patrol area is generated by expanding outward from the difference point as the center.

[0023] Obtain known waypoints adjacent to the unknown cruise area, and construct the minimum coverage path to connect the unknown cruise area based on the spatial distance between the known waypoints and the point of difference;

[0024] Based on the connection points between the minimum coverage path and the reused path, multiple complete cruise paths containing unknown areas are generated. The multiple complete cruise paths are then sorted according to the comprehensive efficiency rating of adjacent known waypoints to determine the optimal cruise path.

[0025] In a preferred embodiment, the step of performing a comprehensive efficiency rating on each historical cruise path and selecting the historical cruise path with the highest comprehensive efficiency rating as the reuse path includes:

[0026] Extract the cruise duration, energy consumption, and image acquisition completeness of each historical cruise path;

[0027] The cruise duration, energy consumption, and image acquisition completeness of historical cruise paths are normalized and recorded as the first efficiency factor, the second efficiency factor, and the third efficiency factor, respectively.

[0028] The first efficiency factor, the second efficiency factor, and the third efficiency factor are weighted to obtain the comprehensive efficiency score of each historical cruise path. Each comprehensive efficiency score corresponds to a comprehensive efficiency rating, and the comprehensive efficiency score and the comprehensive efficiency rating are positively correlated.

[0029] The overall efficiency scores are sorted in descending order, and the historical cruise route with the highest overall efficiency score is selected as the reuse route.

[0030] In a preferred embodiment, the step of constructing a minimum coverage path connecting unknown cruise areas based on the spatial distance between known waypoints and points of difference includes:

[0031] Obtain all known waypoints adjacent to the unknown cruise area, as well as the difference points that need to be covered in the target area;

[0032] This is used to calculate the straight-line distance between each known waypoint and the point of difference, generating a distance set that includes all straight-line distances;

[0033] Select the shortest straight-line distance between known waypoints and difference points from the distance set, and connect the known waypoints and difference points corresponding to the shortest straight-line distances until all difference points corresponding to the unknown cruise area are connected and form a continuous path, thus obtaining the minimum coverage path.

[0034] In a preferred embodiment, the step of matching the real-scene images with the demand feedback information, marking successfully matched real-scene images as valid images, and recording unmatched real-scene images as images to be calibrated includes:

[0035] Denoising is performed on real-scene images to eliminate image noise.

[0036] Based on the image feature requirements in the feedback information, image features are extracted from the denoised real-scene image to obtain the target features corresponding to the feedback information.

[0037] The target features and the demand features in the demand feedback information are converted into vector representations in the same feature space to obtain the first feature vector corresponding to the target features and the second feature vector corresponding to the demand features.

[0038] The similarity between the first feature vector and the second feature vector is calculated, and the matching score between the target feature and the feedback information is output.

[0039] The matching score is compared with a preset matching threshold. When the matching score is higher than or equal to the matching threshold, the real-world image corresponding to the matching score is determined to be a valid image. When the matching score is lower than the matching threshold, the corresponding real-world image is recorded as an image to be calibrated.

[0040] In a preferred embodiment, the step of performing correlation analysis between the geographic location information of the valid images and the spatial distribution of the target area to verify whether the valid images cover all target types includes:

[0041] The target area is divided into several grid cells according to a preset grid density, and each grid cell is labeled with the set of target types that should be covered.

[0042] Obtain the geographic coordinates of each valid image, map the geographic coordinates to the corresponding grid cells, and determine the target types covered within the grid cells;

[0043] Compare the set of target types that should be covered within the grid cell with the target types that are already covered to determine if there are any missing target types;

[0044] If there are missing cells, mark the corresponding grid cell as the area to be supplemented for data collection;

[0045] If there are no missing features, the target feature coverage density of each target type is calculated, and when the target feature coverage density is lower than the preset coverage threshold, the corresponding grid cell is marked as an enhanced acquisition area.

[0046] After the supplementary acquisition area and the enhanced acquisition area are output, the geographic coordinates of the image to be calibrated are matched to identify the image to be calibrated that is geographically consistent with the supplementary acquisition area or the enhanced acquisition area, and then included in the supplementary acquisition task queue.

[0047] The geographic locations of the areas to be supplemented or enhanced within the acquisition area are extracted to cut the image to be calibrated. Based on the cut image blocks, feature matching and geographic coordinate association are re-performed, and the integrity and spatial distribution consistency of the target features are verified item by item.

[0048] After verification, the cut image blocks are added to the valid image set, and the coverage density of the corresponding grid cells is updated.

[0049] In a preferred embodiment, the step of triggering the supplementary patrol mechanism to conduct a secondary patrol of the uncovered area includes:

[0050] Extract all grid cells from the areas to be supplemented and enhanced, and generate a set of boundary coordinates for the uncovered areas;

[0051] Collect the geometric center point of the uncovered area, and expand outward from the geometric center point to form a supplementary patrol sub-region;

[0052] The total range requirement of the supplementary patrol sub-area is obtained and compared with the remaining allowable range of the UAV. When the remaining allowable range is greater than or equal to the total range requirement, a supplementary patrol path is generated and the UAV is triggered to perform a second patrol mission. The supplementary patrol path starts from the real-time position of the UAV, passes through the center point of each supplementary patrol sub-area in sequence, and finally returns to the patrol path for return.

[0053] When the remaining allowable range is less than the total range requirement, the target type missing amount and target feature coverage density in each supplementary patrol sub-region are collected, and the target type missing amount and target feature coverage density are weighted and calculated to generate a priority score.

[0054] The priority scores are sorted from high to low, and the sub-areas are selected in order of priority score according to the coverage of the remaining allowable range. The supplementary patrol routes are then sent to the UAVs to perform a second patrol mission.

[0055] The present invention also provides an intelligent cruise system based on real-scene image feedback, which uses the above-described intelligent cruise method based on real-scene image feedback, including:

[0056] An initialization module is used to receive a cruise mission request initiated by a user terminal, wherein the cruise mission request includes the target area, target type, and demand feedback information.

[0057] The path planning module is used to generate a corresponding cruise path based on the geographical location information of the target area and the historical cruise records of the target area, and to push cruise instructions to the drone.

[0058] The image calibration module is used to acquire real-time images during the drone's cruise process and transmit the real-time images back in real time. At the same time, it matches the real-time images with the required feedback information, marks the successfully matched real-time images as valid images, and records the unmatched real-time images as images to be calibrated.

[0059] The image supplementation module is used to perform correlation analysis between the geographic location information of the effective image and the spatial distribution of the target area, verify whether the effective image covers all target types, and when it does not cover all target types, it extracts features from the image to be calibrated and supplements the image of the missing target type corresponding area into the effective image;

[0060] The supplementary cruise module is used to trigger a supplementary cruise mechanism to conduct a second cruise on the uncovered area when the image to be calibrated fails to fully cover the missing target types, until all target types are covered and then the cruise stops.

[0061] And, an electronic device, the electronic device comprising:

[0062] At least one processor;

[0063] and a memory communicatively connected to the at least one processor;

[0064] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the above-described intelligent cruise method based on real-scene image feedback.

[0065] The technical effects achieved by this invention are as follows:

[0066] This invention provides real-time feedback and verification of real-world images collected during UAV cruise, enabling dynamic adaptation to different cruise environments. When unknown cruise areas appear along the cruise path, it can extrapolate and complete the unknown areas based on known waypoints. By combining the coverage completeness and feature density assessment of target types, it can determine whether the target area is fully covered. If the coverage is incomplete, it will first use images to be calibrated for adaptive supplementation. Only when the images to be calibrated still cannot meet the coverage requirements will the supplementary cruise mechanism be triggered. This effectively reduces the energy consumption and time costs caused by redundant flights, thereby improving the efficiency of UAV cruise. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0068] Figure 2 This is a schematic diagram of the system modules of the present invention;

[0069] Figure 3 This is a schematic diagram of the electronic device structure of the present invention. Detailed Implementation

[0070] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0071] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0072] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that mutually excludes other embodiments.

[0073] With the rapid development of drone technology, its application in various fields is becoming more and more widespread. It has shown its advantages of high efficiency and flexibility in scenarios such as security patrol, urban management, environmental monitoring, and traffic control. Especially in complex urban environments, drones have significantly improved the accuracy of data collection and operational efficiency by combining autonomous navigation with real-time image feedback.

[0074] Please see Figure 1 As shown, the present invention provides an intelligent navigation method based on real-scene image feedback, comprising:

[0075] S1. Receive a cruise mission request initiated by the user terminal, wherein the cruise mission request includes the target area, target type and feedback information.

[0076] In step S1, before the drone performs a patrol mission, the user terminal needs to pre-set the patrol parameters, including the geographical boundaries of the target area, the types of targets to be identified (such as vehicles, pedestrians, fire sources, etc.), and the information to be fed back, so as to ensure that the drone has a clear operational direction when performing the mission. Receiving a patrol mission request initiated by the user terminal includes the following steps:

[0077] Obtain the target area input by the user and verify whether the target area matches the user's permission range;

[0078] If a target area exists within the user's permissions, allow the user to continue inputting the target type and feedback information.

[0079] If the target area does not exist within the user's authorized scope, return an error message to the user and terminate the current patrol mission request;

[0080] Based on the center coordinates of the target area, retrieve the mission execution cycle from the historical cruise records and determine whether the real-time cruise mission request is within the mission execution cycle.

[0081] If it is within the task execution period, the real-time patrol task is determined to be a patrol task within the period, and a historical path recommendation is sent for inquiry. After confirmation by the user, the historical patrol path is used as the patrol path for this time.

[0082] If it is not within the task execution cycle, it is determined to be a new patrol task, and a reminder to establish a new patrol task is sent simultaneously;

[0083] Specifically, when a user sends a cruise request, the system first performs dual verification of the user's identity and device status. Only after confirming that the user's permission level and device authentication information are correct can the user enter the task parameter configuration interface to prevent data leakage or flight accidents caused by unauthorized operations. The user then needs to input the specific geographical range of the target area on the user's end. This geographical range is then compared with preset areas in the user's permission database. If a match is found, the user is allowed to continue setting the target type, such as whether to identify specific vehicle models or pedestrians exhibiting specific behaviors. The user must also specify the level of detail required in the feedback information, such as whether high-definition images, real-time video streams, or shooting from specific angles are needed. If the target area exceeds the permission range, the input interface will be immediately locked, and a pop-up window will display a message indicating the specific reason for insufficient permissions. Simultaneously, an abnormal operation log will be recorded for subsequent updates. Upon successful access verification, the system automatically retrieves historical mission data based on the center coordinates of the target area. If the current request time overlaps with the historical mission execution cycle, a historical path comparison window pops up, displaying the flight trajectory, energy consumption data, and image acquisition quality from the last mission in that area, providing users with reference for decision-making. When a new mission is identified, a mission number is generated and pushed to the device management module. The system checks whether the drone's battery level, signal strength, and current location meet the takeoff conditions. If not, the user is prompted to adjust the mission time or change the equipment. After the user finally confirms all parameters, the cruise mission request is encapsulated into a standard data packet, containing the geographic coordinates of the target area, a target type classification list, the level of request feedback information, and the expected mission completion time, thus providing sufficient data support for the drone's cruise.

[0084] S2. Based on the geographical location information of the target area and the historical cruise records of the target area, generate the corresponding cruise path and push cruise instructions to the drone.

[0085] In step S2, after the user's cruise mission request is confirmed, the drone's cruise path is planned based on the geographical location information of the target area and historical cruise records. The historical cruise records include flight trajectories, obstacle distribution, and energy consumption information. Then, a safe cruise path is searched using methods such as genetic algorithms to achieve the goals of minimizing flight distance, avoiding known obstacles, and balancing energy consumption distribution. The step of generating the corresponding cruise path based on the geographical location information of the target area and its historical cruise records includes:

[0086] Collect the geographic coordinates of the target patrol points in the target area and perform a topological comparison with the historical waypoints in the historical patrol records;

[0087] When the spatial topology of the target patrol point is completely consistent with that of the historical waypoints, a comprehensive efficiency rating is performed on each historical cruise path, and the historical cruise path with the highest comprehensive efficiency rating is selected as the reuse path.

[0088] When there is a difference in the spatial topology between the target patrol point and the historical waypoint, the difference point is output synchronously, and an unknown patrol area is generated by expanding outward from the difference point as the center.

[0089] Obtain known waypoints adjacent to the unknown cruise area, and construct the minimum coverage path to connect the unknown cruise area based on the spatial distance between the known waypoints and the point of difference;

[0090] Based on the connection points between the minimum coverage path and the reused path, multiple complete cruise paths containing unknown areas are generated. The multiple complete cruise paths are then sorted according to the comprehensive efficiency rating of adjacent known waypoints to determine the optimal cruise path.

[0091] Specifically, when planning the drone's patrol path, the first step is to collect the exact geographic coordinates of all patrol points in the target area. Then, the geographic coordinates of the patrol points are compared topologically with waypoints stored in the historical database. If the current patrol point layout is completely consistent with historical records, the path with the best overall performance is selected as the reuse plan based on factors such as flight efficiency and energy consumption levels of each historical path. When a difference in spatial topology is detected, the coordinates of the difference point are immediately marked, and an unknown patrol area is formed by expanding outward from the difference point. The boundary of the unknown patrol area is determined by the minimum proximity distance of adjacent known waypoints, ensuring coverage of all newly added patrol points. Subsequently, the geographic locations of surrounding known waypoints are retrieved, and a minimum coverage path is constructed through spatial distance calculations, ensuring the connection between the unknown area and existing paths. Finally, the generated complete paths are sorted according to the comprehensive efficiency rating of adjacent waypoints, with the path with the highest comprehensive efficiency rating being prioritized as the final patrol path. During the path planning process, the drone continuously monitors the dynamic changes of obstacles, fine-tunes local paths based on real-time sensor data, and also assesses flight risks based on real-time weather data. When a sudden obstacle or severe weather is detected, emergency obstacle avoidance or... Emergency landing is implemented to ensure the safety of the drone during its cruise. For obstacle avoidance, multi-source sensing data from lidar, visual recognition, and millimeter-wave radar can be integrated to identify obstacles and predict their distance. In severe weather, corresponding correction coefficients can be applied based on wind speed and visibility, and incorporated into the overall efficiency rating process. Higher wind speeds or lower visibility result in higher weightings for the correction coefficients and a lower overall efficiency rating, ensuring that safer and more stable cruise paths are prioritized under high wind speeds or low visibility conditions. The specific values ​​of the correction coefficients are determined based on expert experience and historical data. The system is configured based on flight experience. For example, when the wind speed exceeds 12 m / s, the correction factor is set to 0.6; when the visibility is less than 1 km, the correction factor drops to 0.5; if both occur simultaneously, the overall correction factor is set to 0.3, significantly reducing the priority of the path. In addition, when the wind speed is too high or the visibility is low, an emergency landing command will be sent to the drone. For example, if the wind speed exceeds 15 m / s or the visibility is less than 500 meters, continuing to perform the cruise mission will obviously increase the flight risk and may cause loss of control or collision. The emergency landing command will be triggered immediately to avoid the drone from continuing to operate under extreme weather conditions.

[0092] Secondly, the process of comprehensively evaluating the efficiency of each historical cruise route and selecting the historical cruise route with the highest comprehensive efficiency rating as the reuse route includes:

[0093] Extract the cruise duration, energy consumption, and image acquisition completeness of each historical cruise path;

[0094] The cruise duration, energy consumption, and image acquisition completeness of historical cruise paths are normalized and recorded as the first efficiency factor, the second efficiency factor, and the third efficiency factor, respectively.

[0095] The first efficiency factor, the second efficiency factor, and the third efficiency factor are weighted to obtain the comprehensive efficiency score of each historical cruise path. Each comprehensive efficiency score corresponds to a comprehensive efficiency rating, and the comprehensive efficiency score and the comprehensive efficiency rating are positively correlated.

[0096] The overall efficiency scores are sorted in descending order, and the historical cruise route with the highest overall efficiency score is selected as the reuse route.

[0097] In the above process, when selecting a reuse path, the cruise duration, energy consumption, and image acquisition completeness of each historical cruise path are first extracted. These parameters reflect the path's efficiency, economy, and mission completion quality, respectively. Then, these parameters are normalized and converted into dimensionless first, second, and third efficiency factors for subsequent weighted calculations. By assigning appropriate weights to these factors, the impact of different factors on the cruise path's efficiency can be comprehensively considered, resulting in a comprehensive efficiency score for each historical cruise path. The weights of the first, second, and third efficiency factors are determined by expert experience combined with actual operational data. The efficiency score is dynamically adjusted based on the mission type. For example, in emergency patrols, cruise speed is prioritized, and the weight of the first efficiency factor can be set to 0.5. In routine monitoring missions, energy economy and image quality are emphasized, and the weights of the second and third efficiency factors are increased to 0.4 and 0.4, respectively. This ensures that the comprehensive efficiency score accurately reflects the merits of the path in different mission scenarios. Furthermore, the comprehensive efficiency score and comprehensive efficiency rating are positively correlated. The higher the comprehensive efficiency score, the higher the comprehensive efficiency rating, indicating that the corresponding cruise path has better overall performance. The comprehensive efficiency scores are then sorted in descending order, and the historical cruise path with the highest comprehensive efficiency score is selected as the reuse path. This ensures that the UAV can use the optimal cruise path scheme when performing cruise missions, thereby improving cruise efficiency and mission completion quality.

[0098] In addition, the steps for constructing the minimum coverage path connecting unknown cruise areas based on the spatial distance between known waypoints and points of difference include:

[0099] Obtain all known waypoints adjacent to the unknown cruise area, as well as the difference points that need to be covered in the target area;

[0100] This is used to calculate the straight-line distance between each known waypoint and the point of difference, generating a distance set that includes all straight-line distances;

[0101] Select the shortest straight-line distance between known waypoints and difference points from the distance set, and connect the known waypoints and difference points corresponding to the shortest straight-line distances until all difference points corresponding to the unknown cruise area are connected and form a continuous path, thus obtaining the minimum coverage path;

[0102] In the above process, after the difference points are determined, all known waypoints adjacent to the unknown cruise area and the difference points that need to be covered in the target area are first obtained. Based on this, the straight-line distance between each known waypoint and the difference point is calculated, generating a distance set including all straight-line distances to provide a basis for subsequent path selection. Then, the shortest straight-line distance between the known waypoint and the difference point is selected from the distance set to ensure the optimization of the path, that is, to connect the unknown area and the known waypoints with the shortest possible path, and to connect the known waypoints and the difference points corresponding to the shortest straight-line distance, until all the difference points corresponding to the unknown cruise area are connected and form a continuous path. Finally, the minimum coverage path is obtained. The setting of the minimum coverage path not only considers the minimization of spatial distance, but also ensures the effective connection between the unknown cruise area and the existing path, which can provide corresponding guarantees for the efficient cruise of the UAV.

[0103] S3. Real-time acquisition of real-scene images during the drone's cruise process, and real-time transmission of the real-scene images. At the same time, the real-scene images are matched with the required feedback information, and the real-scene images that are successfully matched are marked as valid images, while the real-scene images that are not successfully matched are recorded as images to be calibrated.

[0104] In step S3, during the drone's cruise, real-world images of the target area are acquired via a high-definition camera or specific sensors. These images are collected at an initially set frequency to ensure the capture of feedback information during the cruise. Simultaneously, the real-world images are matched with the feedback information. This process is based on pre-defined target types and recognition rules. For example, if the feedback information specifies the need to identify a particular vehicle model, the vehicle in the real-world image is compared with a pre-defined vehicle model database. When the content in the real-world image matches the target type in the feedback information, the corresponding real-world image is marked as a valid image. Unmatched real-world images are recorded as images awaiting calibration. These images may fail to be calibrated due to factors such as shooting angle, lighting conditions, or obstructions. They are not immediately deleted but stored for subsequent manual review or supplementary calibration, thus ensuring the completeness of the feedback information and the accuracy of the recognition. The steps of matching real-world images with the feedback information, marking successfully matched real-world images as valid images, and recording unmatched real-world images as images awaiting calibration include:

[0105] Denoising is performed on real-scene images to eliminate image noise.

[0106] Based on the image feature requirements in the feedback information, image features are extracted from the denoised real-scene image to obtain the target features corresponding to the feedback information.

[0107] The target features and the demand features in the demand feedback information are converted into vector representations in the same feature space to obtain the first feature vector corresponding to the target features and the second feature vector corresponding to the demand features.

[0108] The similarity between the first feature vector and the second feature vector is calculated, and the matching score between the target feature and the feedback information is output.

[0109] The matching score is compared with a preset matching threshold. When the matching score is higher than or equal to the matching threshold, the real-world image corresponding to the matching score is determined to be a valid image. When the matching score is lower than the matching threshold, the corresponding real-world image is recorded as an image to be calibrated.

[0110] Specifically, when classifying valid images and images to be calibrated, the real-scene images are first denoised to eliminate various noise interferences, providing a clear and clean real-scene image foundation for subsequent matching. Alternatively, preprocessing operations such as image enhancement and color correction can be performed to improve image quality and ensure the accuracy of subsequent feature extraction. Then, based on the image feature requirements specified in the feedback information, corresponding target features are extracted from the denoised real-scene images. These target features reflect information related to the feedback information in the real-scene images. Subsequently, the target features and the requirement features from the feedback information are uniformly transformed into the same feature space and presented in vector representation, yielding a first feature vector corresponding to the target features and a second feature vector corresponding to the requirement features. Vector transformation can employ common feature embedding methods such as principal component analysis, linear discriminant analysis, or deep neural network models. Dimensionality reduction and mapping are performed to ensure the comparability of features from different sources in a unified space. Then, a pre-defined similarity calculation method is used to calculate the similarity between the first and second feature vectors, determining the similarity between them and outputting a matching score between the target feature and the demand feedback information. Similarity calculation can use metrics such as cosine similarity, Euclidean distance, or Manhattan distance. The matching score reflects the degree of fit between the real-world image and the demand feedback information. Finally, the matching score is compared with a pre-defined matching threshold. When the matching score is higher than or equal to the matching threshold, it indicates that the real-world image and the demand feedback information are highly matched, and the real-world image corresponding to the matching score is determined to be a valid image. When the matching score is lower than the matching threshold, it indicates that there is a large gap between the corresponding real-world image and the demand feedback information, and the corresponding real-world image is recorded as an image to be calibrated for further processing and analysis.

[0111] S4. Perform correlation analysis between the geographic location information of the effective image and the spatial distribution of the target area to verify whether the effective image covers all target types. If it does not cover all target types, perform feature extraction on the image to be calibrated and supplement the image of the area corresponding to the missing target type into the effective image.

[0112] In step S4, after the effective images are output, they are correlated with the spatial distribution information of the target area to confirm whether the effective images fully cover all preset target types. If all target types are covered and the effective image distribution for each target type meets the preset coverage threshold, the UAV can return to base, ending the flight mission. Conversely, if not all target types are covered or the image coverage for some target types is below the preset threshold, a supplementary acquisition mechanism will be triggered to extract the missing target type features contained in the image to be calibrated, thereby supplementing the effective images. The step of correlating the geographical location information of the effective images with the spatial distribution of the target area to verify whether the effective images cover all target types includes:

[0113] The target area is divided into several grid cells according to a preset grid density, and each grid cell is labeled with the set of target types that should be covered.

[0114] Obtain the geographic coordinates of each valid image, map the geographic coordinates to the corresponding grid cells, and determine the target types covered within the grid cells;

[0115] Compare the set of target types that should be covered within the grid cell with the target types that are already covered to determine if there are any missing target types;

[0116] If there are missing cells, mark the corresponding grid cell as the area to be supplemented for data collection;

[0117] If there are no missing features, the target feature coverage density of each target type is calculated, and when the target feature coverage density is lower than the preset coverage threshold, the corresponding grid cell is marked as an enhanced acquisition area.

[0118] After the supplementary acquisition area and the enhanced acquisition area are output, the geographic coordinates of the image to be calibrated are matched to identify the image to be calibrated that is geographically consistent with the supplementary acquisition area or the enhanced acquisition area, and then included in the supplementary acquisition task queue.

[0119] The geographic locations of the areas to be supplemented or enhanced within the acquisition area are extracted to cut the image to be calibrated. Based on the cut image blocks, feature matching and geographic coordinate association are re-performed, and the integrity and spatial distribution consistency of the target features are verified item by item.

[0120] After verification, the cut image blocks are added to the valid image set, and the coverage density of the corresponding grid cells is updated.

[0121] Specifically, in verifying whether valid images cover all target types, the target area is first divided into several grid cells according to a preset grid density, and each grid cell is labeled with the set of target types that should be covered, thus clarifying the target type information that needs to be collected for each grid cell. Then, the geographic coordinates of each valid image are obtained, reflecting the specific location of the valid image in the target area. Mapping the geographic coordinates to the corresponding grid cell determines the grid cell to which each valid image belongs, and further determines the target types already covered within the grid cell. By comparing the set of target types that should be covered within the grid cell with the already covered target types, it can be determined whether there are any missing target types. If there are missing types, the corresponding grid cell is marked as an area to be supplemented for subsequent targeted supplementary collection. If there are no missing types, the target feature coverage density for each target type is calculated. The target feature coverage density reflects the distribution of each target type within the grid cell. When the target feature coverage density is lower than a preset coverage threshold, it indicates that the collection amount of that target type within that grid cell is insufficient. The corresponding grid cell is marked as an enhanced acquisition area for subsequent enhanced acquisition. After the supplementary acquisition area and the enhanced acquisition area are output, the geographic coordinates of the image to be calibrated are matched to identify the image to be calibrated that has the same geographic location as the supplementary acquisition area or the enhanced acquisition area. The image to be calibrated may contain missing target type information or target type information that needs to be enhanced. It is then included in the supplementary acquisition task queue for subsequent processing. Specifically, the geographic location of the image to be calibrated is extracted from the supplementary acquisition area or the enhanced acquisition area and then cut. This cutting method can ensure that only the image part related to the supplementary acquisition area or the enhanced acquisition area is processed, which improves processing efficiency. Based on the cut image blocks, feature matching and geographic coordinate association are re-performed, and the integrity of its target features and the consistency of its spatial distribution are verified item by item to ensure that the supplementary acquisition image is consistent with the original effective image in terms of features and spatial distribution. This verification process is the same as the above verification process to ensure that the cut image blocks can be supplemented into the effective image set and the coverage density of the corresponding grid cell is updated, thereby completing the supplementation and enhancement of the effective image.

[0122] S5. When the image to be calibrated fails to fully fill the missing target types, the supplementary patrol mechanism is triggered to conduct a second patrol of the uncovered area until all target types are covered and then stops.

[0123] In step S5, when the image to be calibrated has not completed the supplementation and enhancement of the effective image, a supplementary patrol mechanism is triggered to start a secondary patrol mission. A supplementary patrol path is planned based on the geographic coordinates of the uncovered area, thereby guiding the UAV to conduct a secondary patrol and perform targeted image acquisition on the uncovered or insufficiently covered grid cells, ensuring that missing target types are effectively supplemented. The step of triggering the supplementary patrol mechanism to conduct a secondary patrol of the uncovered area includes:

[0124] Extract all grid cells from the areas to be supplemented and enhanced, and generate a set of boundary coordinates for the uncovered areas;

[0125] Collect the geometric center point of the uncovered area, and expand outward from the geometric center point to form a supplementary patrol sub-region;

[0126] The total range requirement of the supplementary patrol sub-area is obtained and compared with the remaining allowable range of the UAV. When the remaining allowable range is greater than or equal to the total range requirement, a supplementary patrol path is generated and the UAV is triggered to perform a second patrol mission. The supplementary patrol path starts from the real-time position of the UAV, passes through the center point of each supplementary patrol sub-area in sequence, and finally returns to the patrol path for return.

[0127] When the remaining allowable range is less than the total range requirement, the target type missing amount and target feature coverage density in each supplementary patrol sub-region are collected, and the target type missing amount and target feature coverage density are weighted and calculated to generate a priority score.

[0128] The priority scores are sorted from high to low, and the sub-areas are selected in order of priority score according to the coverage of the remaining allowable range. The supplementary patrol path is then sent to the UAV to perform a second patrol mission.

[0129] Specifically, after the supplementary patrol mechanism is triggered, the first step is to extract the set of grid cells in the currently uncovered area. This set contains information about all areas that require supplementary or enhanced data collection. By obtaining the boundary coordinates of the grid cells, a set of boundary coordinates for the uncovered area can be generated, providing basic data for subsequent path planning and area processing. Next, the geometric center point of the uncovered area is collected, and then the area is expanded outward from the geometric center point to form a supplementary patrol sub-region. The expansion range can be set according to actual needs and the drone's flight capabilities, ensuring that the supplementary patrol sub-region covers the entire area requiring supplementary or enhanced data collection. Subsequently, the total range requirement for the supplementary patrol sub-region is obtained and compared with the drone's remaining permissible range. The remaining permissible range is the maximum distance the drone can fly in the current state. When the remaining permissible range is greater than or equal to the total range requirement, it indicates that the drone is capable of completing the patrol of all supplementary patrol sub-regions. At this point, a supplementary patrol path is directly generated, and the drone is triggered to perform a secondary patrol mission. The supplementary patrol path starts from the drone's real-time position and passes through the center points of each supplementary patrol sub-region sequentially, ensuring that the drone can cover all areas requiring supplementary or enhanced data collection. The drone will eventually return to its cruise path to return to its starting point or the next mission point. When the remaining allowable range is less than the total range requirement, it means that the drone cannot complete the cruise of all supplementary sub-regions in one go. At this time, the drone will collect the target type missing amount and target feature coverage density in each supplementary sub-region. The target type missing amount reflects which target types are missing in the area, and the target feature coverage density reflects the distribution of existing target type images in the corresponding area. The target type missing amount and target feature coverage density are weighted to reflect the importance of different factors in priority assessment. A priority score is generated through weighted calculation. The priority score can comprehensively reflect the degree to which each supplementary sub-region needs to be cruised first. The priority scores are sorted from high to low. According to the coverage range of the remaining allowable range, the supplementary sub-regions are selected in turn and included in the supplementary cruise path. This ensures that the drone prioritizes the areas that most need supplementary or enhanced data collection within the limited range. Finally, a supplementary cruise command is sent to the drone to perform a second cruise mission, so that the drone flies according to the generated supplementary cruise path to complete the targeted image collection of the uncovered or insufficiently covered areas.

[0130] Please see Figure 2 A smart cruise system based on real-scene image feedback, using the aforementioned smart cruise method based on real-scene image feedback, includes:

[0131] The initialization module is used to receive cruise mission requests initiated by the user terminal. The cruise mission request includes the target area, target type, and feedback information.

[0132] The path planning module is used to generate corresponding cruise paths based on the geographic location information of the target area and the historical cruise records of the target area, and to push cruise commands to the drone.

[0133] The image calibration module is used to acquire real-time images during the drone's cruise process and transmit the real-time images back in real time. At the same time, it matches the real-time images with the required feedback information, marks the successfully matched real-time images as valid images, and records the unmatched real-time images as images to be calibrated.

[0134] The image supplementation module is used to perform correlation analysis between the geographic location information of the effective image and the spatial distribution of the target area, verify whether the effective image covers all target types, and when it does not cover all target types, it extracts features from the image to be calibrated and supplements the image of the missing target type corresponding area into the effective image;

[0135] The supplementary cruise module is used to trigger a supplementary cruise mechanism to conduct a second cruise on the uncovered area when the image to be calibrated fails to fully cover the missing target types, until all target types are covered and then the cruise stops.

[0136] The execution process of the above-mentioned cruise system corresponds to the process of the intelligent cruise method based on real-scene image feedback, so it will not be repeated here.

[0137] Please see Figure 3 An electronic device, comprising:

[0138] At least one processor;

[0139] and memory that is communicatively connected to at least one processor;

[0140] The memory stores a computer program that can be executed by at least one processor, which enables the at least one processor to execute the aforementioned intelligent cruise method based on real-scene image feedback.

[0141] The processor of the aforementioned electronic device can be a chip with data processing capabilities, such as a central processing unit (CPU), a graphics processing unit (GPU), or a neural network processor (NPU). The memory can be different types of storage media, such as random access memory (RAM), read-only memory (ROM), or flash memory, to meet different data storage needs. The electronic device may also include an arithmetic logic unit, such as an arithmetic logic unit (ALU) or a floating-point unit (FPU), as well as input devices and output devices. The input devices can be touch screens, keyboards, or mice, etc., used to receive user operation commands. The output devices can be displays or printers, etc., used to display or output processing results.

[0142] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0143] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.

Claims

1. An intelligent cruise method based on real-scene image feedback, characterized in that: include: Receive a cruise mission request initiated by the user terminal, wherein the cruise mission request includes the target area, target type and demand feedback information; Based on the geographical location information of the target area and the historical cruise records of the target area, a corresponding cruise path is generated, and cruise instructions are sent to the drone. Real-time acquisition of real-scene images during the drone's cruise process, real-time transmission of real-scene images, matching of real-scene images with required feedback information, marking successfully matched real-scene images as valid images, and recording unmatched real-scene images as images to be calibrated. The geographic location information of the effective images is correlated with the spatial distribution of the target area to verify whether the effective images cover all target types. If not all target types are covered, feature extraction is performed on the images to be calibrated to supplement the images of the missing target types corresponding to the areas into the effective images. If the image to be calibrated fails to fully fill in the missing target types, a supplementary patrol mechanism is triggered to conduct a second patrol of the uncovered area until all target types are covered and then the patrol stops.

2. The intelligent cruise method based on real-scene image feedback according to claim 1, characterized in that: When receiving a cruise mission request initiated by a user terminal, the process includes the following steps: Obtain the target area input by the user and verify whether the target area matches the user's permission range; If a target area exists within the user's permissions, allow the user to continue inputting the target type and feedback information. If the target area does not exist within the user's authorized scope, return an error message to the user and terminate the current patrol mission request; Based on the center coordinates of the target area, retrieve the mission execution cycle from the historical cruise records and determine whether the real-time cruise mission request is within the mission execution cycle. If it is within the task execution period, the real-time patrol task is determined to be a patrol task within the period, and a historical path recommendation is sent for inquiry. After confirmation by the user, the historical patrol path is used as the patrol path for this time. If it is not within the task execution cycle, it is determined to be a new patrol task, and a reminder to establish a new patrol task is sent simultaneously.

3. The intelligent cruise method based on real-scene image feedback according to claim 1, characterized in that: The step of generating a corresponding cruise route based on the geographic location information of the target area and the historical cruise records of the target area includes: Collect the geographic coordinates of the target patrol points in the target area and perform a topological comparison with the historical waypoints in the historical patrol records; When the spatial topology of the target patrol point is completely consistent with that of the historical waypoints, a comprehensive efficiency rating is performed on each historical cruise path, and the historical cruise path with the highest comprehensive efficiency rating is selected as the reuse path. When there is a difference in the spatial topology between the target patrol point and the historical waypoint, the difference point is output synchronously, and an unknown patrol area is generated by expanding outward from the difference point as the center. Obtain known waypoints adjacent to the unknown cruise area, and construct the minimum coverage path to connect the unknown cruise area based on the spatial distance between the known waypoints and the point of difference; Based on the connection points between the minimum coverage path and the reused path, multiple complete cruise paths containing unknown areas are generated. The multiple complete cruise paths are then sorted according to the comprehensive efficiency rating of adjacent known waypoints to determine the optimal cruise path.

4. The intelligent cruise method based on real-scene image feedback according to claim 3, characterized in that: The step of performing a comprehensive efficiency rating on each historical cruise path and selecting the historical cruise path with the highest comprehensive efficiency rating as the reuse path includes: Extract the cruise duration, energy consumption, and image acquisition completeness of each historical cruise path; The cruise duration, energy consumption, and image acquisition completeness of historical cruise paths are normalized and recorded as the first efficiency factor, the second efficiency factor, and the third efficiency factor, respectively. The first efficiency factor, the second efficiency factor, and the third efficiency factor are weighted to obtain the comprehensive efficiency score of each historical cruise path. Each comprehensive efficiency score corresponds to a comprehensive efficiency rating, and the comprehensive efficiency score and the comprehensive efficiency rating are positively correlated. The overall efficiency scores are sorted in descending order, and the historical cruise route with the highest overall efficiency score is selected as the reuse route.

5. The intelligent cruise method based on real-scene image feedback according to claim 3, characterized in that: The step of constructing a minimum coverage path connecting unknown cruise areas based on the spatial distance between known waypoints and points of difference includes: Obtain all known waypoints adjacent to the unknown cruise area, as well as the difference points that need to be covered in the target area; This is used to calculate the straight-line distance between each known waypoint and the point of difference, generating a distance set that includes all straight-line distances; Select the shortest straight-line distance between known waypoints and difference points from the distance set, and connect the known waypoints and difference points corresponding to the shortest straight-line distances until all difference points corresponding to the unknown cruise area are connected and form a continuous path, thus obtaining the minimum coverage path.

6. The intelligent cruise method based on real-scene image feedback according to claim 1, characterized in that: The step of matching real-scene images with demand feedback information, marking successfully matched real-scene images as valid images, and recording unmatched real-scene images as images to be calibrated includes: Denoising is performed on real-scene images to eliminate image noise. Based on the image feature requirements in the feedback information, image features are extracted from the denoised real-scene image to obtain the target features corresponding to the feedback information. The target features and the demand features in the demand feedback information are converted into vector representations in the same feature space to obtain the first feature vector corresponding to the target features and the second feature vector corresponding to the demand features. The similarity between the first feature vector and the second feature vector is calculated, and the matching score between the target feature and the feedback information is output. The matching score is compared with a preset matching threshold. When the matching score is higher than or equal to the matching threshold, the real-world image corresponding to the matching score is determined to be a valid image. When the matching score is lower than the matching threshold, the corresponding real-world image is recorded as an image to be calibrated.

7. The intelligent cruise method based on real-scene image feedback according to claim 1, characterized in that: The step of performing correlation analysis between the geographic location information of the valid images and the spatial distribution of the target area to verify whether the valid images cover all target types includes: The target area is divided into several grid cells according to a preset grid density, and each grid cell is labeled with the set of target types that should be covered. Obtain the geographic coordinates of each valid image, map the geographic coordinates to the corresponding grid cells, and determine the target types covered within the grid cells; Compare the set of target types that should be covered within the grid cell with the target types that are already covered to determine if there are any missing target types; If there are missing cells, mark the corresponding grid cell as the area to be supplemented for data collection; If there are no missing features, the target feature coverage density of each target type is calculated, and when the target feature coverage density is lower than the preset coverage threshold, the corresponding grid cell is marked as an enhanced acquisition area. After the supplementary acquisition area and the enhanced acquisition area are output, the geographic coordinates of the image to be calibrated are matched to identify the image to be calibrated that is geographically consistent with the supplementary acquisition area or the enhanced acquisition area, and then included in the supplementary acquisition task queue. The geographic locations of the areas to be supplemented or enhanced within the acquisition area are extracted to cut the image to be calibrated. Based on the cut image blocks, feature matching and geographic coordinate association are re-performed, and the integrity and spatial distribution consistency of the target features are verified item by item. After verification, the cut image blocks are added to the valid image set, and the coverage density of the corresponding grid cells is updated.

8. The intelligent cruise method based on real-scene image feedback according to claim 1, characterized in that: The steps of triggering the supplementary patrol mechanism to conduct a secondary patrol of the uncovered area include: Extract all grid cells from the areas to be supplemented and enhanced, and generate a set of boundary coordinates for the uncovered areas; Collect the geometric center point of the uncovered area, and expand outward from the geometric center point to form a supplementary patrol sub-region; The total range requirement of the supplementary patrol sub-area is obtained and compared with the remaining allowable range of the UAV. When the remaining allowable range is greater than or equal to the total range requirement, a supplementary patrol path is generated and the UAV is triggered to perform a second patrol mission. The supplementary patrol path starts from the real-time position of the UAV, passes through the center point of each supplementary patrol sub-area in sequence, and finally returns to the patrol path for return. When the remaining allowable range is less than the total range requirement, the target type missing amount and target feature coverage density in each supplementary patrol sub-region are collected, and the target type missing amount and target feature coverage density are weighted and calculated to generate a priority score. The priority scores are sorted from high to low, and the sub-areas are selected in order of priority score according to the coverage of the remaining allowable range. The supplementary patrol routes are then sent to the UAVs to perform a second patrol mission.

9. An intelligent cruise system based on real-scene image feedback, characterized in that: The intelligent cruise method based on real-scene image feedback as described in any one of claims 1 to 8 includes: An initialization module is used to receive a cruise mission request initiated by a user terminal, wherein the cruise mission request includes the target area, target type, and demand feedback information. The path planning module is used to generate a corresponding cruise path based on the geographical location information of the target area and the historical cruise records of the target area, and to push cruise instructions to the drone. The image calibration module is used to acquire real-time images during the drone's cruise process and transmit the real-time images back in real time. At the same time, it matches the real-time images with the required feedback information, marks the successfully matched real-time images as valid images, and records the unmatched real-time images as images to be calibrated. The image supplementation module is used to perform correlation analysis between the geographic location information of the effective image and the spatial distribution of the target area, verify whether the effective image covers all target types, and when it does not cover all target types, it extracts features from the image to be calibrated and supplements the image of the missing target type corresponding area into the effective image; The supplementary cruise module is used to trigger a supplementary cruise mechanism to conduct a second cruise on the uncovered area when the image to be calibrated fails to fully cover the missing target types, until all target types are covered and then the cruise stops.

10. An electronic device, characterized in that: The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the intelligent cruise method based on real-scene image feedback as described in any one of claims 1 to 8.

Images