Residue soil truck dumping behavior identification and detection method and system based on unmanned aerial vehicle

Abstract

The invention belongs to the technical field of muck truck behavior monitoring, and particularly relates to a muck truck dumping behavior identification and detection method and system based on an unmanned aerial vehicle. The muck truck dumping behavior identification and detection method based on the unmanned aerial vehicle comprises the following steps of obtaining continuous multi-frame image data of a muck truck operation area to be monitored, and performing preprocessing; inputting the preprocessed image data into a muck truck detection model, and performing target identification on the muck truck; and judging whether the muck truck has a dumping behavior or not by adopting a time sequence frame analysis method and based on the muck truck identified in the continuous multi-frame image according to the opening and closing angle of the carriage and the continuous characteristic of whether the material falls or not. According to the invention, the identification accuracy, dumping behavior judgment reliability and positioning precision of the muck truck are improved, and full-process automatic supervision is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of slag car behavior monitoring, and particularly relates to a slag car dumping behavior identification and detection method and system based on a UAV. BACKGROUND

[0002] As the main carrier for urban construction waste collection and transportation, slag cars are prone to cause environmental pollution, road safety hazards, and city appearance damage due to illegal dumping. The current market slag car supervision products are mainly divided into three categories: manual patrol, fixed camera monitoring, and traditional UAV monitoring. Manual patrol relies on on-site patrol by supervisors to identify slag cars and illegal dumping behavior by naked eye; fixed camera monitoring involves laying cameras in key areas to achieve preliminary monitoring using image recognition technology; and traditional UAV monitoring uses single-frame images combined with UAV latitude, longitude, height, and other information to convert the position of the slag car in the picture and attempt to identify the behavior.

[0003] The traditional supervision methods have the following significant shortcomings: Manual patrol has high labor costs and limited coverage, making it difficult to cover remote areas and patrol work in complex environments such as night, rain, etc. Fixed cameras have blind spots and cannot follow the moving track of slag cars for full-process monitoring. Existing UAV monitoring technology can achieve mobile patrol, but due to the single-frame image positioning method, camera pose jitter during UAV flight, and synchronization issues between image frames and UAV state information, the positioning accuracy of the slag car is low (usually ≥5m), which cannot meet the precise coordinate requirements for illegal evidence collection.

[0004] At the same time, most systems can only complete slag car target identification and cannot accurately determine the dumping behavior through time series analysis, which may lead to missed or incorrect judgments, seriously affecting the supervision effect. SUMMARY

[0005] To overcome the shortcomings of the prior art, the present application provides a slag car dumping behavior identification and detection method and system based on a UAV, which improves the accuracy of slag car identification, the reliability of dumping behavior determination, and the positioning accuracy, and realizes full-process automatic supervision.

[0006] To achieve the above-mentioned purpose, one or more embodiments of the present application provide the following technical solutions: The present application provides a slag car dumping behavior identification and detection method based on a UAV in the first aspect.

[0007] The slag car dumping behavior identification and detection method based on a UAV includes the following steps: Obtain continuous multiple frames of image data of the slag car operation area to be monitored and perform preprocessing; The preprocessed image data is input into a muck car detection model to perform target recognition on the muck car; A time sequence frame analysis method is adopted to determine whether the muck car has dumping behavior based on the muck car recognized in the continuous multiple frames of images and the continuous features of the opening and closing angle of the carriage and whether there is material falling.

[0008] The second aspect of the present application provides a muck car dumping behavior recognition and detection system based on a UAV.

[0009] The muck car dumping behavior recognition and detection system based on a UAV comprises: An image acquisition module configured to acquire continuous multiple frames of image data of a muck car operation area to be monitored and perform preprocessing; A target recognition module configured to input the preprocessed image data into a muck car detection model to perform target recognition on the muck car; A behavior determination module configured to adopt a time sequence frame analysis method to determine whether the muck car has dumping behavior based on the muck car recognized in the continuous multiple frames of images and the continuous features of the opening and closing angle of the carriage and whether there is material falling. The third aspect of the present application provides a computer readable storage medium having a program stored thereon, and the program is executed by a processor to implement the steps of the muck car dumping behavior recognition and detection method based on a UAV according to the first aspect of the present application.

[0010] The fourth aspect of the present application provides an electronic device comprising a memory, a processor, and a program stored on the memory and executable on the processor, and the processor executes the program to implement the steps of the muck car dumping behavior recognition and detection method based on a UAV according to the first aspect of the present application.

[0011] The above one or more technical solutions have the following beneficial effects: The present application provides a muck car dumping behavior recognition and detection method and system based on a UAV, which designs a way of combining the continuous features of the opening and closing angle of the carriage and whether there is material falling to determine whether the muck car has dumping behavior, which can more accurately determine the dumping behavior of the muck car. Meanwhile, in the process of recognizing the muck car, a muck car detection model is constructed based on the improved YOLOv11 algorithm, the Coordinate Attention attention mechanism is added to the neck network of the YOLOv11 algorithm to enhance the feature extraction capability of the key details such as the carriage and wheels of the muck car, improve the small target detection precision in complex scenes, and improve the precision of the detection results.

[0012] In the specific determination, the application associates the same muck truck in the continuous multiple frames of images through a target tracking algorithm, extracts the outline features of the muck truck carriage in each frame of image, and calculates the carriage opening angle; whether there is a continuous feature of material falling behind the carriage is identified through dynamic pixel change detection; when it is detected that the carriage opening angle is greater than or equal to a preset angle, there is a continuous N frames of material falling dynamic pixel features behind the carriage and the duration is greater than or equal to a preset time, the behavior of the muck truck is determined in combination with the preset compliance boundary data.

[0013] The application improves the muck truck recognition accuracy, dumping behavior determination reliability and positioning accuracy, and realizes full-process automatic supervision.

[0014] The advantages of the additional aspects of the application will be partially given in the following description, partially will become obvious from the following description, or will be known by the practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0015] The drawings accompanying the specification of the application form a part of the specification and serve to further illustrate the application, the illustrative embodiments of the application and the description thereof serve to explain the application without imposing undue limitation on the application.

[0016] Figure 1 The method flowchart of example one.

[0017] Figure 2 The system structure diagram of example two.

[0018] Figure 3 The intelligent identification analysis module structure diagram of example two.

[0019] Figure 4 The target positioning module structure diagram of example two. DETAILED DESCRIPTION

[0020] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the application belongs.

[0021] It should be noted that the terms used herein are only for the purpose of describing specific embodiments, and are not intended to limit the exemplary embodiments according to the application.

[0022] In the case of no conflict, the embodiments in the application and the features in the embodiments can be combined with each other.

[0023] Example one The muck truck needs to travel according to the designated route to dump at the compliant disposal site during transportation. The illegal dumping behavior often has the characteristics of randomness and concealment, which brings great challenges to supervision. The existing supervision means has the problems of insufficient positioning accuracy, inaccurate behavior judgment, poor real-time performance, etc., and cannot meet the fine supervision demand.

[0024] The moving tracking monitoring of the muck truck can be realized by using the unmanned aerial vehicle to carry a high-definition camera to collect video streams of the operation area, and combining the state information of the unmanned aerial vehicle itself such as GPS and attitude angle. The muck truck target in the image can be recognized by a deep learning algorithm and the pixel coordinates are output. The spatial observation model can be constructed by combining the pixel coordinates of multiple frames and the corresponding unmanned aerial vehicle state data to calculate the GPS coordinates of the muck truck.

[0025] However, when the unmanned aerial vehicle is flying, the camera continuously outputs high-frame-rate video pictures (such as 30 FPS). Although the update frequency of the unmanned aerial vehicle state information is relatively high, there may still be frame synchronization problems, and single-frame data is easily affected by attitude jitter, resulting in large positioning errors. At the same time, simple single-frame target recognition cannot capture dynamic characteristics such as changes in the carriage attitude and material falling, and it is difficult to accurately judge the dumping behavior. The embodiment discloses a muck truck dumping behavior recognition and detection method based on an unmanned aerial vehicle, which aims to solve the above problems and improve the recognition accuracy of the muck truck, the reliability of the dumping behavior judgment, and the positioning accuracy to realize the automatic supervision of the whole process.

[0026] As shown in Figure 1 The muck truck dumping behavior recognition and detection method based on the unmanned aerial vehicle includes the following steps: Obtain continuous multiple frames of image data of the operation area of the muck truck to be monitored and perform preprocessing; Input the preprocessed image data into a muck truck detection model to recognize the target of the muck truck; Using a time sequence frame analysis method, based on the muck truck recognized in the continuous multiple frames of images, according to the continuous characteristics of the carriage opening angle and whether there is material falling, it is judged whether the muck truck has dumping behavior.

[0027] Next, the technical solutions of the embodiment will be explained and described in detail.

[0028] The present solution mainly consists of four steps: unmanned aerial vehicle data acquisition, intelligent recognition analysis, target positioning, and result output and control, to realize muck truck recognition, behavior judgment, accurate positioning, and supervision visualization.

[0029] (I) Unmanned aerial vehicle data acquisition: The core function of this step is to collect continuous multiple frames of image data of the operation area and record the unmanned aerial vehicle state parameters at the time of each frame image acquisition synchronously, to provide data support for subsequent recognition and positioning.

[0030] The unmanned aerial vehicle is selected from an industrial-grade inspection unmanned aerial vehicle, carries a high-definition camera unit, a combined navigation unit, a flight control unit and a wireless transmission unit.

[0031] The flight control unit controls the unmanned aerial vehicle to cruise according to a preset route, and the high-definition camera unit collects continuous images with a resolution of ≥2000 million pixels at a frame rate of ≥30 FPS, covering the construction site, the cleaning and transporting route, the disposal field and the key monitoring area.

[0032] The combined navigation unit adopts GPS / IMU / RTK differential positioning technology, synchronously records the unmanned aerial vehicle state data at each frame image collection, including GPS coordinates (longitude, latitude, altitude), attitude angle (pitch angle, yaw angle, roll angle), flight height and camera internal parameters (focal length, image center pixel coordinates, etc.), wherein the GPS positioning accuracy is ≤0.1 m, the IMU attitude measurement accuracy is ≤0.05°, and the data update frequency is ≥100 Hz.

[0033] The wireless transmission unit adopts 4G / 5G and unmanned aerial vehicle image transmission dual-mode communication technology, and transmits the collected image data and state data to the on-board edge computing device or the ground relay node in real time, with a transmission delay of ≤200 ms, ensuring real-time data.

[0034] (2) Intelligent identification and analysis: This step is responsible for pre-processing, slag soil truck target detection and dumping behavior judgment of the collected multiple images, and outputs accurate slag soil truck pixel coordinates and behavior results.

[0035] It includes image preprocessing, slag soil truck detection and dumping behavior judgment, and the specific process is as follows: 1) Image preprocessing: After receiving continuous multiple images: First, carry out defogging and noise reduction processing - use dark channel prior algorithm to eliminate image blur caused by weather such as rain and fog, and remove image noise through Gaussian filtering; Then, based on SIFT algorithm, perform inter-frame image registration to eliminate inter-frame offset caused by unmanned aerial vehicle flight jitter, and ensure the continuity of the position of the slag soil truck target in multiple images; Finally, perform frame rate synchronization processing to accurately match the image frame and the unmanned aerial vehicle state data.

[0036] 2) Slag soil truck detection: An improved YOLOv11 algorithm is used to construct a slag soil truck detection model, and the model input is the pre-processed image data.

[0037] The Coordinate Attention attention mechanism is added to the neck network of the YOLOv11 algorithm to enhance the feature extraction capability of key details such as the slag soil truck compartment and wheels, and improve the small target detection accuracy in complex scenes.

[0038] Coordinate Attention (CA) mechanism mainly enhances the model's attention to spatial information by introducing spatial position encoding, thereby improving the expressiveness of features.

[0039] Spatial information encoding: The CA mechanism introduces spatial coordinate information into the feature map, and through weighted processing of the features at each position, the network can focus on the importance of different spatial positions.

[0040] Coordinate direction enhancement: The CA mechanism weights the features based on coordinate information (including position and relative position), enhancing the network's attention to position details, thereby effectively improving the extraction of small target (such as wheels, carriages, etc.) features.

[0041] Enhancing sensitivity to small targets: Small targets usually exist in low-resolution or medium-resolution feature maps. After adding CA, the network will pay more attention to the detailed features in these low-resolution layers. This is because CA will make each position on these feature maps be weighted and associated with its coordinate information, thereby improving the recognition ability of small targets in local regions.

[0042] Denoising and enhancing target information: In complex scenes, background information is usually very complex. By introducing the CA mechanism, the network can effectively reduce background interference and focus on key details of the target, avoiding misidentification of background information as target features.

[0043] Fine-grained feature fusion: The CA mechanism allows the network to fuse information of different scales by jointly processing spatial information and feature maps, improving the ability to extract fine-grained features of small targets such as carriages and wheels. This feature fusion helps to accurately locate small targets in low-resolution feature maps.

[0044] The model outputs the pixel coordinate frame (x1, y1, x2, y2) of the dump truck in each frame image, the detection confidence, and the target center pixel coordinates (u, v), with a detection accuracy of ≥98%. Detection results with a confidence lower than 0.95 will be excluded to ensure the reliability of target recognition.

[0045] 3) Dumping behavior judgment: Using a time sequence frame analysis method, based on the multi-frame target information output by the dump truck detection unit, and combining the changes in material features in the image to achieve behavior judgment.

[0046] First, the target tracking algorithm is used to associate the same dump truck in consecutive frames, extract the contour features of the dump truck carriage in each frame, and calculate the opening angle of the carriage. At the same time, through dynamic pixel change detection, it is identified whether there is a continuous feature of material falling behind the carriage.

[0047] When the detection of the opening angle of the carriage is ≥ 30°, there are continuous 3 frames or more of material falling dynamic pixel features behind the carriage, and the duration is ≥ 1.5s, combined with the preset compliance disposal field GIS boundary data, it is determined whether the muck truck has a violation of dumping behavior.

[0048] (Three), target positioning: Based on the multi-frame muck truck pixel coordinates output by the intelligent recognition analysis step and the state data collected by the unmanned aerial vehicle, the high-precision GPS coordinates of the muck truck are calculated through the GPS of the unmanned aerial vehicle at different positions and the target coordinates at different positions, solving the problem of insufficient positioning accuracy of traditional single frame.

[0049] Specifically, it includes coordinate screening, observation model construction and coordinate solution, and the specific process is as follows: 1) Coordinate screening unit: From the output results of the intelligent recognition analysis step, the center pixel coordinates (u1, v1), (u2, v2)…(u N ,v N ) of the same muck truck in N frames (N≥3) continuous images are screened, and the screening condition is that the adjacent frame unmanned aerial vehicle displacement is ≥ 0.5m and the pixel coordinate corresponding confidence is ≥ 0.95, to ensure that the selected coordinate data has sufficient spatial difference and reliability.

[0050] Among them, (u1, v1) is the center pixel coordinate of the first frame image selected; (u2, v2) is the center pixel coordinate of the second frame image selected; (u N ,v N ) is the center pixel coordinate of the Nth frame image selected.

[0051] 2) Observation model construction: Based on the camera imaging principle, an association model of pixel coordinates and geographic coordinates is established.

[0052] Define the GPS coordinates of the unmanned aerial vehicle when the i-th frame image is collected as (X i ,Y i ,Z i ), the pitch angle as θ i , the yaw angle as α i , the roll angle as γ i , the camera focal length as f, the image center pixel coordinates as (u i ,v i ), and the muck truck geographic coordinates as (X, Y, Z).

[0053] Construct the observation equation of pixel coordinates (u i ,v i ) and muck truck geographic coordinates (X, Y, Z) to realize the mapping from the pixel domain to the geographic domain: [(ui - v0) / f, 1] i - v0) / f, 1] = R i × [(X - X i ), (Y - Y i ), (Z - Z i )] / ||(X - X i ), (Y - Y i ), (Z - Z i )||.

[0054] where R i is the rotation matrix converted from the attitude angles of the UAV, and ||·|| represents the modulus of the vector.

[0055] The left side is the normalized homogeneous coordinate vector of the pixel coordinates, and the right side is the result of the normalized conversion of the geographical coordinate vector of the sludge truck relative to the UAV by the rotation matrix.

[0056] 3) Coordinate solving unit: First, the attitude angles (θ i , α i , γ i ) of the UAV are converted into a rotation matrix R i by the Rodrigues matrix, and the formula is: R i = R(γ i ) × R(θ i ) × R(α i ), where R(γ i ), R(θ i ), and R(α i ) are the rotation matrices corresponding to the roll angle, pitch angle, and yaw angle, respectively. Substitute the rotation matrix R i into the observation equation to obtain: [(u i -u0) / f, (v i -v0) / f, 1] = R i ×[(X-X i ),(Y-Y i ),(Z-Z i )] / ||(X-X i ),(Y-Y i ),(Z-Z i )|| where ||·|| represents the modulus of the vector. Finally, N-frame observation equations are constructed to form an overdetermined equation set, and the high-precision GPS coordinates (X, Y, Z) of the slag car are solved by minimizing the sum of squares of errors using the least squares method, and the positioning error is ≤1.5 m.

[0057] (Four), result output and management and control: This step is responsible for receiving the behavior judgment result of the intelligent recognition analysis step and the GPS coordinate data of the target positioning step, realizing data storage, abnormal warning and visual display, and providing intuitive supervision basis and operation entrance for supervisors.

[0058] Specifically, it includes a cloud management step and a terminal display step, and the specific process is as follows: 1) Cloud management step: The distributed database is used to store the original image data, the slag car detection result, the dumping behavior judgment result, the GPS coordinate data and the unmanned aerial vehicle state data, and supports retrieval according to time, region, behavior type and other dimensions, and the data storage period is ≥2 years.

[0059] When a violation dumping behavior is detected, the abnormal warning unit sends warning information to the supervision terminal within 100ms through SMS, APP push and other ways, with the slag car GPS coordinates, on-site images and behavior judgment basis.

[0060] At the same time, the data analysis unit generates a high-risk area heat map, a time period distribution curve and a vehicle type statistical report based on historical data, providing data support for supervision strategy optimization.

[0061] 2) Terminal display step: Multi-interface visual display function is provided, the map visualization interface is based on electronic map API, superimposed with unmanned aerial vehicle cruising track, slag car GPS position mark (violation dumping mark is red, and compliant dumping mark is green), and compliant disposal field boundary, and clicking the mark can view the detailed information and evidence images of the slag car; The real-time monitoring interface synchronously plays the unmanned aerial vehicle video stream, superimposed with the slag car detection frame, confidence and behavior state label, and supports one-key screenshot and video evidence; The data tracing interface supports querying historical detection records according to time and region, generates driving and violation track diagram of a single slag car, and the data can be exported in PDF or Excel format, which is convenient for subsequent archiving and law enforcement.

[0062] Embodiment two The embodiment discloses a slag car dumping behavior recognition and detection system based on an unmanned aerial vehicle.

[0063] The slag car dumping behavior recognition and detection system based on the unmanned aerial vehicle comprises: An image acquisition module is configured to acquire continuous multiple frames of image data of a slag soil truck operation area to be monitored and to perform preprocessing. A target recognition module is configured to input the preprocessed image data into a slag soil truck detection model to perform target recognition on the slag soil truck. A behavior judgment module is configured to use a time sequence frame analysis method to judge whether the slag soil truck has dumping behavior based on the slag soil truck recognized in the continuous multiple frames of image data and according to the continuous characteristics of the carriage opening angle and whether there is material falling.

[0064] Next, the scheme of the present embodiment will be explained in detail.

[0065] The present scheme mainly consists of four modules: an unmanned aerial vehicle data acquisition module, an intelligent recognition and analysis module, a target positioning module, and a result output and control module. The modules work cooperatively to realize slag soil truck recognition, behavior judgment, accurate positioning, and supervision visualization.

[0066] (I) Unmanned aerial vehicle data acquisition module: This module is the data input end of the system, and the core function is to acquire continuous multiple frames of image data of the operation area and to record the unmanned aerial vehicle state parameters at each frame of image acquisition time synchronously to provide data support for subsequent recognition and positioning.

[0067] The unmanned aerial vehicle is an industrial-grade inspection unmanned aerial vehicle, which is equipped with a high-definition camera unit, a combined navigation unit, a flight control unit, and a wireless transmission unit.

[0068] The flight control unit controls the unmanned aerial vehicle to cruise according to the preset route, and the high-definition camera unit acquires continuous images with a resolution of ≥20 million pixels at a frame rate of ≥30 FPS, covering the construction site, the clean-up route, the disposal site, and the key monitoring area.

[0069] The combined navigation unit uses GPS / IMU / RTK differential positioning technology to record the unmanned aerial vehicle state data at each frame of image acquisition time synchronously, including GPS coordinates (longitude, latitude, and altitude), attitude angles (pitch angle, yaw angle, and roll angle), flight height, and camera internal parameters (focal length, image center pixel coordinates, etc.), wherein the GPS positioning accuracy is ≤0.1 m, the IMU attitude measurement accuracy is ≤0.05°, and the data update frequency is ≥100 Hz.

[0070] The wireless transmission unit uses 4G / 5G and unmanned aerial vehicle image transmission dual-mode communication technology to transmit the acquired image data and state data to the on-board edge computing device or the ground relay node in real time, with a transmission delay of ≤200 ms, ensuring real-time data.

[0071] (II) Intelligent recognition and analysis module: The module is the core analysis unit of the system, responsible for pre-processing the collected multiple images, slag truck target detection and dumping behavior judgment, and outputting accurate slag truck pixel coordinates and behavior results.

[0072] The module internally includes an image pre-processing unit, a slag truck detection unit and a dumping behavior judgment unit, and the specific process is as follows: 1) Image pre-processing unit: After receiving multiple consecutive images, first, carry out defogging and noise reduction processing - use the dark channel prior algorithm to eliminate image blurring caused by weather such as rain and haze, and remove image noise through Gaussian filtering; then, based on the SIFT algorithm, perform inter-frame image registration to eliminate inter-frame offset caused by UAV flight jitter, ensuring the continuity of the position of the slag truck target in multiple images; finally, perform frame rate synchronization processing to accurately match the image frames with the UAV state data.

[0073] 2) Slag truck detection unit: Based on the improved YOLOv11 algorithm, a slag truck detection model is constructed, and the model input is the pre-processed image data.

[0074] In the neck network of the YOLOv11 algorithm, the Coordinate Attention attention mechanism is added to enhance the feature extraction capability of key details such as the slag truck's carriage and wheels, and to improve the detection accuracy of small targets in complex scenes.

[0075] The model output is the pixel coordinate frame (x1, y1, x2, y2) of the slag truck in each image, the detection confidence and the target center pixel coordinates (u, v), where the detection accuracy is ≥98%, and the detection result with a confidence lower than 0.95 will be excluded to ensure the reliability of target recognition.

[0076] 3) Dumping behavior judgment unit: Using a time sequence frame analysis method, based on the multiple frame target information output by the slag truck detection unit, and combining the material feature changes in the image to realize behavior judgment.

[0077] First, through the target tracking algorithm, the same slag truck in consecutive frames is associated, the contour features of the slag truck carriage in each frame are extracted, and the carriage opening angle is calculated; At the same time, through dynamic pixel change detection, it is identified whether there is a continuous feature of material falling behind the carriage.

[0078] When the carriage opening angle ≥30°, there are continuous 3 frames or more of material falling dynamic pixel features behind the carriage and the duration is ≥1.5s, combined with the pre-set compliant disposal field GIS boundary data, it is judged whether the slag truck has illegal dumping behavior.

[0079] (Three) Target positioning module: This module is based on the multi-frame pixel coordinates of the dump truck output by the intelligent recognition and analysis module and the status data recorded by the UAV data acquisition module. It calculates the high-precision GPS coordinates of the dump truck by using the GPS of the UAV at different positions and the target coordinates at different positions, thus solving the problem of insufficient positioning accuracy in traditional single-frame positioning.

[0080] The module contains a coordinate filtering unit, an observation model construction unit, and a coordinate calculation unit. The specific process is as follows: 1) Coordinate filtering unit: From the results output by the intelligent recognition and analysis module, filter the center pixel coordinates (u1, v1), (u2, v2), ..., (u...) of the same dump truck in N consecutive frames (N≥3) of images. N ,v N The selection criteria are that the displacement of the UAV in adjacent frames is ≥0.5m and the confidence level of the corresponding pixel coordinates is ≥0.95, to ensure that the selected coordinate data has sufficient spatial difference and reliability.

[0081] 2) Observation model construction unit: Based on camera imaging principles, a correlation model between pixel coordinates and geographic coordinates is established. The GPS coordinates of the UAV during the acquisition of the i-th frame are defined as (X...). i ,Y i Z i The pitch angle is θ i Yaw angle is α i The roll angle is γ i The camera focal length is f, and the image center pixel coordinates are (u i ,v i The geographical coordinates of the dump truck are (X,Y,Z), and pixel coordinates (u) are constructed. i ,v i The observation equations for the pixel domain and the geographic coordinates (X,Y,Z) of the dump truck are used to achieve the mapping from the pixel domain to the geographic domain: [(u i - u0) / f, (v i - v0) / f, 1] = R i × [(X - X i ), (Y - Y i ), (Z - Z i )] / ||(X- X i ), (Y - Y i ), (Z - Z i )||.

[0082] Among them, R iThis is the rotation matrix after the UAV attitude angle is transformed. ||·|| represents the magnitude of the vector. The left side is the homogeneous coordinate vector after the pixel coordinates are normalized, and the right side is the result of the normalization of the geographic coordinate vector of the dump truck relative to the UAV after transformation by the rotation matrix.

[0083] 3) Coordinate calculation unit: First, the attitude angle (θ) of the UAV is determined using the Rodrigues matrix. i ,α i ,γ i Convert to rotation matrix R i The formula is R i =R(γ i )×R(θ i )×R(α i ), where R(γ) i ), R(θ) i ), R(α) i These are the rotation matrices corresponding to the roll, pitch, and yaw angles, respectively; then, substituting the rotation matrices into the observation equations, we obtain: [(u i -u0) / f, (v i -v0) / f, 1] = R i ×[(XX i ),(YY i ),(ZZ i )] / ||(XX i ),(YY i ),(ZZ i (where ||·|| represents the magnitude of the vector); Finally, an overdetermined set of equations was constructed for the N-frame observation equations. The least squares method was used to minimize the sum of squared errors, and the high-precision GPS coordinates (X,Y,Z) of the dump truck were obtained by solving the equations, with a positioning error ≤1.5m.

[0084] (iv) Result Output and Control Module: This module is responsible for receiving the behavior judgment results from the intelligent identification and analysis module and the GPS coordinate data from the target positioning module, realizing data storage, anomaly warning and visualization, providing regulatory personnel with intuitive regulatory basis and operation entry.

[0085] The module includes a cloud management unit and a terminal display unit. The specific process is as follows: 1) Cloud Management Unit: The system uses a distributed database to store raw image data, dump truck detection results, dumping behavior judgment results, GPS coordinate data, and drone status data. It supports retrieval by time, region, behavior type, and other dimensions, and the data storage period is ≥2 years.

[0086] When a violation dumping behavior is detected, the abnormality early warning unit sends early warning information to the supervision terminal within 100 ms through short message, APP push, etc., with the slag car GPS coordinates, on-site images and behavior judgment basis.

[0087] At the same time, the data analysis unit generates a high-risk area heat map, a time period distribution curve and a vehicle type statistical report based on historical data, providing data support for supervision strategy optimization.

[0088] 2) Terminal display unit: Provide multi-interface visual display function, the map visual interface is based on electronic map API, superimposed with unmanned aerial vehicle cruising track, slag car GPS position mark (violation dumping mark is red, and compliant dumping mark is green) and compliant disposal field boundary, and clicking the mark can view the detailed information and evidence image of the slag car; The real-time monitoring interface synchronously plays the unmanned aerial vehicle video stream, superimposed with the slag car detection frame, confidence and behavior state label, supports one-key screenshot and video evidence taking; The data tracing interface supports querying historical detection records according to time and region, generates driving and violation track diagram of a single slag car, and the data can be exported in PDF or Excel format, facilitating subsequent archiving and law enforcement use.

[0089] Embodiment three The purpose of this embodiment is to provide a computer readable storage medium.

[0090] The computer readable storage medium has a computer program stored thereon, and the program is executed by a processor to implement the steps in the slag car dumping behavior identification and detection method based on an unmanned aerial vehicle according to Embodiment 1 of the present disclosure.

[0091] Embodiment four The purpose of this embodiment is to provide an electronic device.

[0092] The electronic device includes a memory, a processor and a program stored on the memory and executable on the processor, and the processor executes the program to implement the steps in the slag car dumping behavior identification and detection method based on an unmanned aerial vehicle according to Embodiment 1 of the present disclosure.

[0093] The steps and methods involved in the above embodiments two, three and four correspond to Embodiment one, and the specific embodiments can be referred to the related description part of Embodiment one. The term "computer readable storage medium" should be understood as including a single medium or multiple media of one or more instruction sets; it should also be understood as including any medium capable of storing, encoding or carrying instruction sets for execution by a processor and causing the processor to perform any method in the present disclosure.

[0094] Those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computer devices, or alternatively, they can be realized by program codes executable by the computer devices, so that they can be stored in the storage devices and executed by the computer devices, or they can be respectively manufactured into individual integrated circuit modules, or a plurality of modules or steps among them can be manufactured into a single integrated circuit module. The present application is not limited to any specific combination of hardware and software.

[0095] The specific embodiments of the present application described above in conjunction with the accompanying drawings are not intended to limit the protection scope of the present application. Those skilled in the art should understand that various modifications or changes made on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A method for recognizing and detecting dumping behavior of construction waste trucks based on unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Acquire continuous multi-frame image data of the operating area of ​​the dump trucks to be monitored, and perform preprocessing; The preprocessed image data is input into the dump truck detection model to perform target recognition of the dump truck; Using a time-series frame analysis method, based on the dump trucks identified in multiple consecutive frames of images, the method determines whether the dump trucks are dumping. This is done by considering the continuous features of the truck bed opening angle and whether material is falling.

2. The method for identifying and detecting dumping behavior of construction waste trucks based on unmanned aerial vehicles as described in claim 1, characterized in that, Acquire continuous multi-frame image data of the area where the dump trucks are operating to be monitored, specifically including: The drones are controlled to patrol according to preset routes, which cover construction sites, waste disposal routes, disposal sites and key monitoring areas. The drone was used to collect continuous multi-frame image data of the area where the dump trucks were operating to be monitored. The system synchronously records the UAV status parameters at the moment of each image acquisition, including GPS coordinates, attitude angles, flight altitude, and camera intrinsic parameters.

3. The method for identifying and detecting dumping behavior of construction waste trucks based on unmanned aerial vehicles as described in claim 1, characterized in that, The specific process of the preprocessing includes: Dehazing and noise reduction are performed: Dark channel prior algorithm is used to eliminate image blur caused by weather, and Gaussian filtering is used to remove image noise; Inter-frame image registration is performed based on the SIFT algorithm to eliminate inter-frame offset caused by drone flight jitter and ensure the continuity of the target position of the dump truck in multiple frames. Frame rate synchronization processing is performed to ensure accurate matching of image frames with UAV status data.

4. The method for identifying and detecting dumping behavior of construction waste trucks based on unmanned aerial vehicles as described in claim 1, characterized in that, The preprocessed image data is input into the dump truck detection model to perform target recognition of the dump trucks, specifically including: A dump truck detection model was constructed based on the improved YOLOv11 algorithm, with preprocessed image data as the input. The Coordinate Attention mechanism is added to the neck network of the YOLOv11 algorithm to enhance the feature extraction capability of key details at the location of the dump truck compartment; The output of the dump truck detection model is the pixel bounding box of the dump truck in each frame of the image, the detection confidence score, and the pixel coordinates of the target center.

5. The method for identifying and detecting dumping behavior of construction waste trucks based on unmanned aerial vehicles as described in claim 1, characterized in that, Using a time-series frame analysis method, based on the dump trucks identified in multiple consecutive frames of images, the method determines whether the dump trucks are dumping, based on the continuous features of the truck bed opening angle and whether material is falling. Specifically, this includes: By using a target tracking algorithm to associate the same dump truck in multiple consecutive frames of images, the contour features of the dump truck's cargo compartment in each frame are extracted, and the opening and closing angle of the cargo compartment is calculated. By detecting dynamic pixel changes, it can identify whether there are continuous features of materials falling at the rear of the carriage; When the opening angle of the truck bed is detected to be greater than or equal to a preset angle, and there are continuous N frames or more of dynamic pixel features of material falling behind the truck bed for a duration greater than or equal to a preset time, the dump truck is judged to have engaged in illegal dumping behavior in combination with the preset compliance boundary data.

6. The method for identifying and detecting dumping behavior of construction waste trucks based on unmanned aerial vehicles as described in claim 2, characterized in that, This also includes calculating the GPS coordinates of the dump truck based on the pixel coordinates of the dump truck in multiple frames and the state parameters of the drone: Filter the center pixel coordinates of the same dump truck in multiple consecutive frames of images; Based on the camera imaging principle, a correlation model between the center pixel coordinates and the geographic coordinates is established: an observation equation is constructed between the center pixel coordinates and the geographic coordinates of the dump truck to realize the mapping from the pixel domain to the geographic domain. An overdetermined set of equations was constructed for the multi-frame observation equations. The sum of squared errors was minimized using the least squares method, and the GPS coordinates of the dump truck were obtained by solving the equations.

7. The method for identifying and detecting dumping behavior of construction waste trucks based on unmanned aerial vehicles as described in claim 6, characterized in that, It also includes data storage, anomaly warning, and visualization based on the behavior determination results of whether the dump truck has dumped waste and the GPS coordinate data of the dump truck.

8. A drone-based system for recognizing and detecting dumping behavior of construction waste trucks, characterized in that, include: The image acquisition module is configured to acquire multiple consecutive frames of image data of the operating area of ​​the dump truck to be monitored and perform preprocessing. The target recognition module is configured to input preprocessed image data into the dump truck detection model to perform target recognition on the dump truck; The behavior determination module is configured to: use a time-series frame analysis method to determine whether the dump truck is dumping based on the continuous features of the opening and closing angle of the truck bed and whether there is material falling, based on the identification of the dump truck in multiple consecutive frames of images.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the method for identifying and detecting dumping behavior of construction waste trucks based on unmanned aerial vehicles as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the method for identifying and detecting dumping behavior of construction waste trucks based on unmanned aerial vehicles as described in any one of claims 1-7.

Images