Unmanned aerial vehicle image intelligent perception plotting method and system based on view GIS fusion

Abstract

The invention discloses an unmanned aerial vehicle image intelligent perception plotting method and system based on view GIS fusion. The method comprises the steps that a sequence image of a target area is collected through an unmanned aerial vehicle platform; target detection, classification and tracking are carried out based on deep learning, and target pixel coordinates are identified; constructing an intelligent mapping model from a pixel coordinate system to a geographic coordinate system to realize geographic positioning of a target; precise space calibration is carried out on the illegal target; and in combination with space calibration, interactive plotting and visualization of target illegal behaviors are realized at a WebGIS end, and a structured law enforcement evidence packet is formed. According to the method, the POS data and the visual information are deeply fused, the dynamic projection plane estimation technology is adopted, the geographic positioning precision better than that of a traditional method is achieved, the method has obvious advantages especially in complex environments such as urban areas, and high positioning precision can be achieved.

Description

Technical Field

[0001] This invention relates to the field of UAV image perception and plotting technology, and in particular to a method and system for intelligent perception and plotting of UAV images based on view GIS fusion. Background Technology

[0002] With the booming development of the low-altitude economy, drones are increasingly being used in logistics, urban management, traffic monitoring, emergency response, and land surveying. However, the following common technical challenges remain unresolved in low-altitude monitoring and application scenarios:

[0003] First, target spatial positioning is ambiguous. Traditional drone monitoring mainly relies on manual visual inspection or simple image analysis, which cannot accurately and automatically associate the target (pixel coordinates) identified in the video frame with the real-world geographical coordinates (latitude, longitude, and elevation). Operators can usually only describe "a certain car in the picture" but cannot accurately answer "which road and which location this car is on," resulting in poor positioning accuracy and missing spatial information.

[0004] Second, there is a lack of multi-dimensional electronic evidence chains. Existing monitoring systems often only provide single video stream evidence, lacking the ability to automatically bind target behavior with information such as time, space, and attributes to form structured data. This makes post-event review, analysis, and evidence collection extremely difficult, failing to meet the data element requirements of modern intelligent supervision.

[0005] Third, there is a disconnect between information perception and decision-making. The target information (such as vehicles and pedestrians) identified by computer vision algorithms remains at the image level and is difficult to directly integrate into GIS-based command and dispatch and business management systems, forming information silos that are "visible" but "unusable", resulting in low decision-making efficiency.

[0006] Fourth, it has a weak ability to process dynamic targets. Existing technologies mostly focus on the surveying and modeling of static scenes, and lack effective technical means for the continuous tracking, behavior analysis and trajectory mapping of ground moving targets.

[0007] While existing technologies include methods for orthorectification and target localization using single images, these methods are typically cumbersome, lack real-time performance, and heavily rely on high-precision digital surface models (DSMs), resulting in significant errors in complex urban environments.

[0008] Therefore, there is an urgent need for a mapping technology that can integrate visual perception with geospatial data in real time, accurately and automatically. Summary of the Invention

[0009] To address the aforementioned problems, the present invention aims to provide a method and system for intelligent perception and mapping of UAV images based on view-GIS fusion, enabling real-time, accurate, and automated perception, geolocation, and electronic mapping of moving targets during UAV ground observation, forming a structured spatiotemporal data evidence chain, and providing core spatiotemporal data intelligent production capabilities for low-altitude applications.

[0010] This invention provides a method and system for intelligent perception and plotting of UAV images based on view GIS fusion.

[0011] The first aspect: A method for intelligent perception and plotting of UAV imagery based on view-GIS fusion, including:

[0012] S1. Synchronously acquire sequential images of the target area using multi-source sensors on an unmanned aerial vehicle platform;

[0013] S2. Perform deep learning-based target detection, classification and tracking on the sequence of images to identify traffic violation targets and obtain their pixel coordinates;

[0014] S3. Construct an intelligent mapping model from pixel coordinate system to geographic coordinate system, dynamically calculate ground sampling distance GSD, and interpolate and correct the elevation of target point based on digital elevation model DEM to achieve geographic positioning of target.

[0015] S4. Based on the target's geographic location, accurately spatially pinpoint the illegal target according to the type of illegal behavior.

[0016] S5. Combining spatial calibration, interactive plotting and visualization of target illegal acts are achieved on the WebGIS platform, forming a structured law enforcement evidence package.

[0017] In one embodiment of the present invention, when acquiring sequential images in S1, the flight altitude, focal length, camera pitch angle, roll angle, heading angle and RTK / PPK high-precision positioning and attitude determination data corresponding to each frame of the image are recorded simultaneously.

[0018] In one embodiment of the present invention, the process of establishing the intelligent mapping model in S3 includes:

[0019] S31. Project the pixel coordinates (u, v) of the target point back to the camera coordinate system using the camera intrinsic parameter matrix. The formula is:

[0020]

[0021]

[0022]

[0023] in, The altitude of the drone. To obtain the elevation of the target point by interpolation correction based on the DEM, For camera tilt angle, For camera internal parameters;

[0024] S32. Using the rotation matrix R formed by the UAV attitude angles, transform the camera coordinate system to the Northeast-North Celestial Coordinate System. The conversion formula is:

[0025]

[0026] S33. Combining the latitude and longitude of the UAV takeoff point (Lon0, Lat0) with the WGS84 ellipsoid model, convert the world coordinates into latitude, longitude, and elevation geographic coordinates (Lon, Lat, Alt).

[0027] In one embodiment of the present invention, step S3, which involves interpolating and correcting the elevation of the target point based on a digital elevation model (DEM), includes the following steps:

[0028] S51. Dynamically load DEM data of the corresponding accuracy level based on the real-time flight altitude and position of the UAV;

[0029] S52. Calculate the precise elevation of the target point using a bilinear interpolation algorithm. The formula is expressed as:

[0030]

[0031] in The weights of the four neighboring grid points of the DEM grid point where the target point is located. This represents the elevation value of the corresponding grid point.

[0032] In one embodiment of the present invention, the precise spatial calibration of the illegal target in step S4 includes: vertical positioning method, triangulation method, and comprehensive positioning method, wherein:

[0033] The steps of the vertical positioning method include:

[0034] S41. Select a reference point;

[0035] S42. Select a baseline, which is either the road centerline or the road edge line;

[0036] S43. Draw a straight line from the reference point to the point to be measured, and then draw a perpendicular line from the point to be measured to the reference line;

[0037] S44. Measure the lengths of the drawn straight line and perpendicular line to complete the positioning of the point to be measured.

[0038] The steps of the triangulation method include:

[0039] S45. Select two reference points, which are obvious ground features in the image that are far apart and have fixed positions;

[0040] S46. Draw straight lines from the two reference points to the point to be measured;

[0041] S47. Measure the distance between the two straight lines to complete the positioning of the point to be measured.

[0042] The steps of the comprehensive positioning method include:

[0043] S48. Select a reference point;

[0044] S49. Select a baseline, which is either the road centerline or the road edge line;

[0045] S50. Draw a straight line from the reference point to the point to be measured, and then draw a perpendicular line from the point to be measured to the reference line.

[0046] S51. Measure the lengths of the drawn straight line and perpendicular line to complete the positioning of the first point to be measured;

[0047] S52. Use the already located first test point as a new reference point to measure other test points in its vicinity.

[0048] In one embodiment of the present invention, the visualized content in S5 includes: geographic coordinates, timestamps, GSD precision parameters, and snapshots and videos of the illegal process.

[0049] The second aspect: A UAV imagery intelligent perception and plotting system based on view-GIS fusion, comprising:

[0050] Data acquisition module: Based on a drone platform equipped with an RTK positioning unit, oblique photography camera and POS system, it acquires high-definition sequential images and synchronous positioning and attitude data;

[0051] Intelligent perception module: used for automatic target detection, classification and tracking based on deep learning in sequential images;

[0052] Intelligent mapping module: used to perform GSD dynamic calculation, coordinate transformation and elevation interpolation correction to achieve accurate mapping of the target from pixel coordinates to geographic coordinates;

[0053] Plotting Application Module: Provides plotting functions and displays visualizations on the WebGIS platform using a dual-Canvas layered rendering architecture;

[0054] Law enforcement evidence management module: Used to generate, store and manage structured law enforcement evidence packages containing geographic information.

[0055] In one embodiment of the present invention, the law enforcement evidence management module includes a standardized evidence export interface, which supports the generation of law enforcement evidence packages that meet legal requirements, including geographic coordinates, timestamps, GSD precision parameters, and snapshots and videos of the illegal process.

[0056] The plotting application module supports multi-user collaborative plotting, and synchronizes plotting data and law enforcement status between different clients in real time via the WebSocket protocol.

[0057] Third aspect: An electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, performs the steps of the method provided in the first aspect.

[0058] Fourth aspect: A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect.

[0059] The beneficial effects of this invention are:

[0060] 1. The method of the present invention achieves geolocation accuracy superior to traditional methods by deeply integrating POS data and visual information and using dynamic projection surface estimation technology. It has significant advantages, especially in complex environments such as urban areas, and can achieve high positioning accuracy.

[0061] 2. The entire processing flow of the method of the present invention is optimized and designed to support real-time operation on airborne computing units or edge computing devices. The latency from perception to plotting is extremely low, meeting the needs of on-site monitoring and immediate response, and has strong real-time performance.

[0062] 3. The method of this invention requires no manual intervention throughout the entire process, automatically completing the entire chain from target discovery to map plotting, greatly improving efficiency, reducing labor costs, and achieving a high degree of automation.

[0063] 4. The method of this invention produces not just simple videos, but structured spatiotemporal big data that is seamlessly integrated with maps. It can be directly used for analysis, mining, evidence collection and decision-making, and truly builds a complete multi-dimensional electronic evidence chain, realizing the data elementization.

[0064] 5. The system of this invention has good compatibility and modular design, which can be adapted to a variety of UAV platforms and sensors, and can be quickly integrated with various existing GIS platforms and business systems through standard interfaces. Attached Figure Description

[0065] Figure 1 This is a schematic flowchart of the method of the present invention;

[0066] Figure 2 This is a schematic diagram of the system of the present invention;

[0067] Figure 3 This is a schematic diagram illustrating the image-geospatial mapping principle of the present invention.

[0068] Figure 4 This is a flowchart illustrating the dynamic calculation of UAV ground sampling distance according to the present invention.

[0069] Figure 5 This is an example diagram illustrating the intelligent plotting effect of the present invention;

[0070] Figure 6 This is a report diagram illustrating the chain of evidence for this invention;

[0071] Figure 7 This is a schematic diagram of the structure of the electronic device of the present invention. Detailed Implementation

[0072] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0073] Existing intelligent perception and plotting methods suffer from problems such as low data processing efficiency, limited perception accuracy, lack of flexibility and adaptability, insufficient data fusion capabilities, and lack of effective interactivity. To address these issues, this invention provides an intelligent perception and plotting method system for UAV imagery based on view-GIS fusion.

[0074] Example 1:

[0075] This embodiment discloses an intelligent perception and plotting system for UAV images based on view GIS fusion, such as... Figure 2 As shown, it includes:

[0076] The data acquisition module, based on a drone platform equipped with an RTK positioning unit, an oblique photography camera, and a POS system, acquires high-definition sequential images and synchronous positioning and attitude data.

[0077] The input sources utilize oblique photography cameras (e.g., visible light / infrared cameras), POS systems (positioning and attitude determination systems) mounted on the UAV platform to provide high-precision latitude and longitude, altitude, pitch, roll, and heading angles, as well as LiDAR (optional) and GNSS / RTK modules.

[0078] Based on the above UAV platform configuration, video streams / image frames and the precise exterior orientation elements (X, Y, Z, ω, φ, κ) corresponding to each frame can be acquired simultaneously. Here, X, Y, and Z are coordinate information, θ is the pitch angle, φ is the roll angle, and κ is the heading angle.

[0079] The intelligent perception module is used for automatic target detection, classification, and tracking of sequential images based on deep learning.

[0080] The intelligent perception module is equipped with a deep learning model to perform real-time processing on the input video stream and uses pre-trained deep learning models (such as YOLO, Faster R-CNN, DeepSort, etc.) for object detection, classification and tracking.

[0081] For each frame of the image, output the pixel coordinates (bounding box) of the target in the image. The target category (such as "car", "pedestrian", "truck"), unique ID (for tracking), and confidence level are also considered.

[0082] Intelligent mapping module: used to perform GSD dynamic calculation, coordinate transformation and elevation interpolation correction to achieve accurate mapping of the target from pixel coordinates to geographic coordinates;

[0083] Based on the intelligent mapping module, a real-time mapping relationship is established from the image pixel coordinate system to the geographic coordinate system (such as WGS-84). This module abandons the heavy reliance on high-precision DEM / DSM and adopts a dynamic projection surface estimation and coordinate solution method.

[0084] First, a camera model is constructed, and strict collinearity equation conditions are established based on the camera's intrinsic parameters (focal length, principal point, distortion coefficient) and the precise extrinsic parameters provided by the UAV's POS data.

[0085] Secondly, projection plane estimation is performed. For each target identified in the image, it is assumed to be located on an estimated average ground plane (the average elevation can be preset according to the task area, or estimated in real time through sparse point clouds). By iteratively optimizing or introducing auxiliary sensor data (such as laser ranging), the projection plane elevation is dynamically corrected, significantly improving the positioning accuracy in undulating terrain.

[0086] Next, coordinate calculation is performed. The pixel coordinates of the target's bottom center point (the point in contact with the ground) in the image are used to calculate its geographic coordinates (longitude, latitude, and elevation) by applying collinearity equations and combining them with the estimated projection surface elevation. This achieves a precise mapping from (u, v) to (Lon, Lat, Alt).

[0087] The plotting application module provides plotting functions and displays them visually on the WebGIS client through a dual-Canvas layered rendering architecture.

[0088] The plotting application module enables the structured integration and visualization of perception and mapping results.

[0089] The data layer output contains structured spatiotemporal data records with the following fields: [Target ID, Timestamp, Longitude, Latitude, Estimated Elevation, Target Type, Movement Speed ​​(optional), Movement Direction (optional), Image Snapshot]. Each record is a complete electronic evidence point.

[0090] The visualization layer output plots the target location in real time on a 2D electronic map or 3D GIS scene. It dynamically displays the target's movement process in the form of points, icons, or trajectory lines, achieving a "what you see is what you get" map-level visualization.

[0091] Furthermore, the plotting application module supports multi-user collaborative plotting, and uses the WebSocket protocol to synchronize plotting data and law enforcement status between different clients in real time.

[0092] The law enforcement evidence management module is used to generate, store, and manage structured law enforcement evidence packages containing geographic information.

[0093] The law enforcement evidence management module provides a user interface for monitoring, interaction, and data export. It also offers standard API interfaces (such as RESTful APIs) to push generated spatiotemporal data to higher-level command systems, business platforms, or cloud databases in real time, breaking down information silos.

[0094] Furthermore, the law enforcement evidence management module includes a standardized evidence export interface, which supports the generation of law enforcement evidence packages that meet legal requirements. The content includes geographic coordinates, timestamps, GSD precision parameters, and snapshots and videos of the illegal process.

[0095] Example 2:

[0096] This embodiment discloses an intelligent perception and plotting method for UAV imagery based on view-GIS fusion, such as... Figure 1 As shown, the steps include:

[0097] S1. Synchronously acquire sequential images of the target area using multi-source sensors on the UAV platform.

[0098] When acquiring sequential images, the flight altitude, focal length, camera pitch angle, roll angle, heading angle, and RTK / PPK high-precision positioning and attitude determination data corresponding to each frame of the image are recorded simultaneously.

[0099] S2. Perform deep learning-based target detection, classification, and tracking on the sequence images to identify traffic violation targets and obtain their pixel coordinates.

[0100] A pre-trained deep learning object detection model is used to scan the acquired sequence of images frame by frame. This model, trained on a large number of traffic violation target samples, can accurately identify various types of traffic violation targets. After identifying the traffic violation targets, image analysis techniques are used to obtain their pixel coordinates within the images.

[0101] Furthermore, to improve the accuracy and stability of detection, a multi-frame correlation method is used to track the target. This involves analyzing the positional changes of the same target in multiple consecutive frames of imagery to establish the target's motion trajectory.

[0102] S3. Construct an intelligent mapping model from pixel coordinate system to geographic coordinate system, dynamically calculate ground sampling distance GSD, and interpolate and correct the elevation of target point based on digital elevation model DEM to achieve geographic positioning of target.

[0103] The ground sampling distance (GSD) is dynamically calculated. GSD is one of the most important indicators in UAV aerial surveying. GSD directly determines the spatial resolution and accuracy of the image. The smaller the GSD value, the higher the resolution and the clearer the ground details that can be seen (e.g., smaller objects can be identified). The larger the GSD value, the lower the resolution and the more blurred the details.

[0104] The required GSD is determined based on the mission accuracy requirements, such as Figure 4 As shown,

[0105]

[0106] The molecule (H × S) can be understood as "the imaging range resulting from the flight altitude". The higher the flight altitude, the larger H or the larger the sensor (S) becomes, the wider the ground area covered, and the larger the ground size represented by a single pixel (GSD) will be.

[0107] The denominator (f × W) can be understood as the "imaging capability of the camera system". The longer the focal length, the larger f is, which is equivalent to a higher magnification of the telescope, a narrower field of view, and more details are "magnified", resulting in a smaller GSD. The more pixels, the larger W is, which means that the same imaging range is divided into finer parts, so the actual size represented by each pixel is also smaller in GSD.

[0108] Specifically, such as Figure 3 As shown, the process of establishing the intelligent mapping model includes:

[0109] First, the pixel coordinates (u, v) of the target point are projected back onto the camera coordinate system using the camera intrinsic parameter matrix. The formula is:

[0110]

[0111]

[0112]

[0113] in, The altitude of the drone. To obtain the elevation of the target point by interpolation correction based on the DEM, For camera tilt angle, This refers to the camera's internal parameters.

[0114] Then, using the rotation matrix R composed of the UAV attitude angles (pitch angle θ, roll angle φ, and yaw angle κ), the camera coordinate system is transformed to the northeast-sky world coordinate system. The conversion formula is:

[0115]

[0116] Then, by combining the latitude and longitude of the UAV takeoff point (Lon0, Lat0) with the WGS84 ellipsoid model, the world coordinates are converted into latitude, longitude, and elevation geographic coordinates (Lon, Lat, Alt).

[0117] Furthermore, the specific implementation of the intelligent mapping module includes:

[0118] First, select the pixel coordinates (u1, v1), (u2, v2), ... of the same target in two or more images from different viewpoints;

[0119] Then, collinearity equations are established based on the high-precision POS data of the UAV corresponding to each image;

[0120] Then, the three-dimensional world coordinates (Xw, Yw, Zw) of the target point are solved by least squares adjustment.

[0121] Finally, the calculated 3D world coordinates are transformed to the geographic coordinate system, and the (Lon, Lat, Alt) 3D coordinates are output.

[0122] Furthermore, the elevation of the target point is interpolated and corrected based on the Digital Elevation Model (DEM), including:

[0123] First, based on the real-time flight altitude and position of the UAV, DEM data of the corresponding accuracy level is dynamically loaded;

[0124] Then, the precise elevation of the target point is calculated using a bilinear interpolation algorithm. The formula is expressed as:

[0125]

[0126] in, The weights of the four neighboring grid points of the DEM grid point where the target point is located. This represents the elevation value of the corresponding grid point.

[0127] S4. Based on the target's geographic location, accurately spatially pinpoint the illegal target according to the type of illegal behavior.

[0128] Specifically, precise spatial positioning of illegal targets includes: vertical positioning, triangulation, and comprehensive positioning methods, among which:

[0129] The vertical positioning method includes the following steps: selecting a reference point and a reference line, which can be the center line or the edge line of the road; then drawing a straight line from the reference point to the point to be measured, and then drawing a perpendicular line from the point to be measured to the reference line; finally measuring the lengths of the drawn straight line and perpendicular line to complete the positioning of the point to be measured.

[0130] The steps of the triangulation method include: selecting two reference points, which are obvious ground features in the image that are far apart and have fixed positions; drawing straight lines from the two reference points to the point to be measured; and measuring the distance between the two lines to complete the positioning of the point to be measured.

[0131] The steps of the comprehensive positioning method include: selecting a reference point; selecting a baseline, which is either the road centerline or the road edge line; drawing a straight line from the reference point to the point to be measured, and then drawing a perpendicular line from the point to be measured to the baseline; measuring the lengths of the drawn straight line and perpendicular line to complete the positioning of the first point to be measured; using the positioned first point to be measured as a new reference point for measuring other points to be measured in its vicinity.

[0132] S5. Combining spatial calibration, interactive plotting and visualization of target illegal acts are achieved on the WebGIS platform, forming a structured law enforcement evidence package.

[0133] Among them, the visual content is as follows Figure 5 As shown, this includes geographic coordinates, timestamps, GSD precision parameters, and snapshots and videos of the illegal process.

[0134] Application examples:

[0135] like Figure 5 As shown, taking urban traffic monitoring as an example, the implementation process of the method of the present invention is explained in detail:

[0136] Mission initialization: The UAV takes off with an RTK-POS system and a visible light camera. The average elevation of the mission area is preset to 50 meters (based on the DEM data of the area).

[0137] Data Acquisition: The drone transmits a 1080p / 30fps video stream, and each frame of the image is accompanied by high-precision POS data (latitude, longitude, altitude, attitude angle).

[0138] Real-time perception: Onboard computing units (such as Jetson AGX Orin) run YOLOv7 models to detect vehicles in the video in real time and output the pixel coordinates and category of each vehicle.

[0139] Intelligent Mapping: The mapping module reads the POS data and camera intrinsic parameters of the current frame. For each identified vehicle, it takes the pixel coordinates of the midpoint of the bottom of its bounding box. Using 50 meters as the initial elevation, it calculates the preliminary geographic coordinates using the collinearity equation. Combining information from previous and subsequent frames, it estimates the minute elevation changes caused by vehicle movement, performs dynamic corrections, and finally outputs latitude and longitude coordinates accurate to sub-meter level.

[0140] Plotting and output: such as Figure 5 and Figure 6 The system combines information such as [vehicle ID, time, longitude, latitude, and vehicle type] into a JSON data packet. This packet is sent to the ground station in real time via a 4G / 5G link, dynamically displaying the vehicle's location and trajectory on a large map screen. It is also stored in a spatiotemporal database for subsequent use in traffic flow analysis and violation evidence collection.

[0141] The present invention also provides an electronic device, Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 7 As shown, the electronic device may include a processor, a communications interface, memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions from the memory, for example, to execute the following method:

[0142] S1. Synchronously acquire sequential images of the target area using multi-source sensors on an unmanned aerial vehicle platform;

[0143] S2. Perform deep learning-based target detection, classification and tracking on the sequence of images to identify traffic violation targets and obtain their pixel coordinates;

[0144] S3. Construct an intelligent mapping model from pixel coordinate system to geographic coordinate system, dynamically calculate ground sampling distance GSD, and interpolate and correct the elevation of target point based on digital elevation model DEM to achieve geographic positioning of target.

[0145] S4. Based on the target's geographic location, accurately spatially pinpoint the illegal target according to the type of illegal behavior.

[0146] S5. Combining spatial calibration, interactive plotting and visualization of target illegal acts are achieved on the WebGIS platform, forming a structured law enforcement evidence package.

[0147] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments, including, for example:

[0149] S1. Synchronously acquire sequential images of the target area using multi-source sensors on an unmanned aerial vehicle platform;

[0150] S2. Perform deep learning-based target detection, classification and tracking on the sequence of images to identify traffic violation targets and obtain their pixel coordinates;

[0151] S3. Construct an intelligent mapping model from pixel coordinate system to geographic coordinate system, dynamically calculate ground sampling distance GSD, and interpolate and correct the elevation of target point based on digital elevation model DEM to achieve geographic positioning of target.

[0152] S4. Based on the target's geographic location, accurately spatially pinpoint the illegal target according to the type of illegal behavior.

[0153] S5. Combining spatial calibration, interactive plotting and visualization of target illegal acts are achieved on the WebGIS platform, forming a structured law enforcement evidence package.

[0154] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent perception and plotting of UAV images based on view GIS fusion, characterized in that, The application relates to a system and method for intelligent traffic violation detection and spatial positioning based on UAV. The method comprises the following steps: S1, synchronously collecting sequence images of a target area through a multi-source sensor of a UAV platform; S2, performing target detection, classification and tracking on the sequence images based on deep learning, identifying a traffic violation target and obtaining a pixel coordinate of the target; S3, constructing an intelligent mapping model from a pixel coordinate system to a geographic coordinate system, dynamically calculating a ground sampling distance (GSD), and performing interpolation correction on the height of the target point based on a digital elevation model (DEM), so that the geographic positioning of the target is realized; S4, combining the geographic positioning of the target, accurately marking the target according to the type of the target violation behavior; 2.The unmanned aerial vehicle image intelligent perception plotting method based on view GIS fusion according to claim 1, characterized in that, S5, combining the spatial marking, realizing interactive marking and visualization of the target violation behavior on a WebGIS terminal, and forming a structured law enforcement evidence package. 3.The unmanned aerial vehicle image intelligent perception plotting method based on view GIS fusion according to claim 1, characterized in that, In the step S1, when the sequence images are collected, the flight height, focal length, camera pitch angle, roll angle, heading angle and RTK / PPK high-precision positioning and orientation data corresponding to each frame of image are synchronously recorded. S31, back project the pixel coordinates (u, v) of the target point to the camera coordinate system through the camera intrinsic matrix The formula is: wherein, is the altitude of the UAV, is the elevation of the target point obtained by interpolating and correcting the DEM, is the camera pitch angle, is the camera intrinsic parameter; S32, convert the camera coordinate system to the northeast celestial world coordinate system by using a rotation matrix R composed of the unmanned aerial vehicle attitude angle The conversion formula is: The construction process of the intelligent mapping model in the step S3 comprises the following steps: 4.The unmanned aerial vehicle image intelligent perception plotting method based on view GIS fusion according to claim 1, characterized in that, S33, combining the longitude and latitude (Lon0, Lat0) of the takeoff point of the UAV and the WGS84 ellipsoid model, the world coordinates are converted into longitude and latitude elevation geographic coordinates (Lon, Lat, Alt). In the step S3, the interpolation correction on the height of the target point based on the DEM comprises the following steps: The bilinear interpolation algorithm is used to calculate the accurate elevation of the target point The formula is expressed as: wherein is the weight of the four adjacent grid points of the DEM grid point where the target point is located, is the elevation value of the corresponding grid point. 5.The unmanned aerial vehicle image intelligent perception plotting method based on view GIS fusion according to claim 1, characterized in that, According to the real-time flight height and position of the UAV, DEM data of a corresponding precision level is dynamically loaded; The accurate spatial marking of the target in the step S4 comprises the following steps: The vertical positioning method, the triangular positioning method and the comprehensive positioning method, wherein: The vertical positioning method comprises the following steps: S41, selecting a reference point; S42, selecting a reference line, the reference line being a road center line or a road edge line; S43, drawing a straight line from the reference point to a to-be-measured point, and then drawing a perpendicular line from the to-be-measured point to the reference line; S44, measuring the lengths of the straight line and the perpendicular line, and completing the positioning of the to-be-measured point; The triangular positioning method comprises the following steps: S45, selecting two reference points, the reference points being obvious ground object points in the image which are far apart and fixed in position; S46, respectively drawing straight lines from the two reference points to a to-be-measured point; S47, measuring the distance of the two straight lines, and completing the positioning of the to-be-measured point; The comprehensive positioning method comprises the following steps: S48, selecting a reference point; S49, selecting a reference line, the reference line being a road center line or a road edge line; S50, drawing a straight line from the reference point to a to-be-measured point, and then drawing a perpendicular line from the to-be-measured point to the reference line; S51, measuring the lengths of the straight line and the perpendicular line, and completing the positioning of the first to-be-measured point; 6.The unmanned aerial vehicle image intelligent perception plotting method based on view GIS fusion according to claim 1, characterized in that, S52, taking the positioned first to-be-measured point as a new reference point, and using the new reference point to measure other to-be-measured points near the new reference point.

7. An unmanned aerial vehicle image intelligent perception plotting system based on view GIS fusion, applying the method of any one of claims 1 to 6. The visualization content in the step S5 comprises geographic coordinates, a time stamp, a GSD precision parameter and a violation process snapshot and video. The system comprises: a data acquisition module: based on a UAV platform loaded with an RTK positioning unit, a tilt photography camera and a POS system, high-definition sequence images and synchronous positioning and orientation data are collected; Intelligent perception module: for target automatic detection, classification and tracking based on deep learning of sequence images; Intelligent mapping module: for performing GSD dynamic calculation, coordinate conversion and elevation interpolation correction, and realizing accurate mapping of targets from pixel coordinates to geographic coordinates; Plotting application module: provides plotting function and visualizes through double Canvas layered rendering architecture on WebGIS side; Law enforcement evidence management module: for generating, storing and managing structured law enforcement evidence package containing geographic information.

8. The plotting system according to claim 7, characterized in that, The law enforcement evidence management module includes a standardized evidence export interface, which supports the generation of law enforcement evidence packages that meet legal requirements, including geographic coordinates, timestamps, GSD precision parameters, and illegal process snapshots and videos; The plotting application module supports multi-user collaborative plotting function, and synchronizes plotting data and law enforcement status between different clients in real time through WebSocket protocol.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the unmanned aerial vehicle image intelligent perception plotting method based on view GIS fusion according to any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the unmanned aerial vehicle image intelligent perception plotting method based on view GIS fusion according to any one of claims 1 to 6.

Images