Residue soil scattering monitoring method and system based on unmanned aerial vehicle

Abstract

The invention discloses a muck scattering monitoring method and system based on an unmanned aerial vehicle. The muck scattering monitoring method comprises the following steps: acquiring a ground image in real time through imaging equipment carried on the unmanned aerial vehicle; segmenting a road area from the ground image to obtain a road image, and performing target detection on the road image; tracking all uncovered muck truck targets and all suspected muck missing and scattering targets; determining muck targets which are being left and scattered from all the suspected muck left and scattered targets; and determining all non-covering muck truck targets in front of the muck target which is being left and scattered at present, and determining non-covering muck trucks which left and scatter the muck target. According to the invention, based on the unmanned aerial vehicle platform and the mounted imaging equipment, the uncovered muck trucks and the scattered muck are tracked, the muck targets which are being scattered are found in time, and the muck targets are associated with the corresponding uncovered muck trucks, so that the on-road monitoring management of the muck trucks is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and particularly to a method and system for monitoring soil and gravel spillage based on an unmanned aerial vehicle (UAV). Background Art

[0002] With the acceleration of the urbanization process, the transportation activities of soil and gravel trucks on urban roads are becoming increasingly frequent. However, during the transportation process, soil and gravel trucks are prone to spillage, that is, due to the failure to cover the soil and gravel on the trucks, the soil and gravel are omitted onto the road surface during transportation, which not only pollutes the environment but may also cause traffic accidents.

[0003] Currently, the monitoring of soil and gravel truck spillage is only completed through manual inspections, which has low work efficiency and is difficult to conduct real-time monitoring and identification of large areas. At the same time, the results obtained from manual inspections are lagging, that is, when it is manually determined that there is spillage on the road surface, the soil and gravel truck is already outside the line of sight, so it is impossible to remind the driver of the soil and gravel truck to conduct inspections and management. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a method and system for monitoring soil and gravel spillage based on an unmanned aerial vehicle (UAV). It tracks uncovered soil and gravel trucks and spilled soil and gravel based on the UAV platform and the mounted imaging device, timely discovers the soil and gravel targets that are being spilled, and associates the soil and gravel targets with the corresponding uncovered soil and gravel trucks to achieve the monitoring and management of soil and gravel trucks on the road.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] On the one hand, a method for monitoring soil and gravel spillage based on an unmanned aerial vehicle (UAV) is provided, which includes the following steps:

[0007] Control the UAV to fly along a preset route, and during the flight process, obtain ground images in real time through the imaging device carried on the UAV;

[0008] Segment the road area from the ground images to obtain road images, and perform target detection on the road images to determine whether there are uncovered soil and gravel truck targets on the road;

[0009] During the time period from time T to time T+M, track all uncovered soil and gravel truck targets and all suspected spilled soil and gravel targets;

[0010] Determine the soil and gravel targets that are being spilled from all suspected spilled soil and gravel targets;

[0011] Determine all uncovered soil and gravel truck targets in front of the currently spilled soil and gravel target at time T+M, and determine the uncovered soil and gravel truck that spilled the soil and gravel target.

[0012] Preferably, the value range of M is 0.5 - 3 seconds.

[0013] Preferably, determining the muck targets that are currently being spilled from all suspected muck spill targets includes the following steps:

[0014] Determine the muck targets in the uncovered muck truck compartments and the muck targets that have already been spilled onto the road surface;

[0015] Determine the remaining suspected muck spill targets other than all the muck targets in the uncovered muck truck compartments and all the muck targets that have already been spilled onto the road surface as the muck targets that are currently being spilled.

[0016] Preferably, determining the muck targets in the uncovered muck truck compartments includes the following steps:

[0017] At the moment of T + M, obtain the set U of muck targets of the current uncovered muck truck, and this set U of muck targets contains at least one suspected muck spill target, and for each suspected muck spill target in this set U of muck targets, the IoU with the current uncovered muck truck is greater than 0. Repeat this step to obtain the set of muck targets of each uncovered muck truck;

[0018] Determine the uncovered muck truck Zi with the largest IoU with the current suspected muck spill target at the moment of T + M, and determine whether the current suspected muck spill target is the suspected muck spill target with the largest area in the set Ui of muck targets of this uncovered muck truck Zi. If so, consider the current suspected muck spill target as the muck target in the compartment of the uncovered muck truck Zi. Repeat this step to determine the muck targets in the compartments of each uncovered muck truck.

[0019] Preferably, determining the muck targets that have already been spilled onto the road surface includes the following steps:

[0020] Obtain the positions of the same suspected muck spill target at the moment of T and T + M respectively. If the distance of this suspected muck spill target at the moments of T and T + M is less than or equal to the threshold, then consider that this suspected muck spill target has not moved during the time period from T to T + M, and this suspected muck spill target is the muck target that has already been spilled onto the road surface.

[0021] Preferably, determining the uncovered muck truck that spilled this muck target includes the following steps:

[0022] If at the moment of T + M, there is only one uncovered muck truck target in front of the current muck target that is currently being spilled, then directly determine this uncovered muck truck as the vehicle that spilled this muck target.

[0023] Preferably, determining the uncovered muck truck that spilled this muck target includes the following steps:

[0024] If at time T+M, there are several uncovered dump trucks in front of the current soil and gravel target being spilled, then obtain the center point coordinates of the soil and gravel target and the center point coordinates of each uncovered dump truck target in front of the soil and gravel target;

[0025] Obtain the pixel distance between the center point of the current soil and gravel target being spilled and the center point of each uncovered dump truck target in front of the soil and gravel target;

[0026] Determine the uncovered dump truck target corresponding to the minimum pixel distance as the vehicle that spilled the soil and gravel target.

[0027] Preferably, the soil and gravel spill monitoring method further includes the following steps:

[0028] Calculate the actual physical position of the soil and gravel target when it falls onto the road surface, and track the uncovered dump truck that spilled the soil and gravel target, and send the information of the soil and gravel target on the road surface and the information of the uncovered dump truck that spilled the soil and gravel target to the monitoring terminal through the unmanned aerial vehicle.

[0029] Preferably, the information of the soil and gravel target on the road surface includes one or more of the image of the soil and gravel target that has been spilled on the road surface and the actual physical position; the information of the uncovered dump truck that spilled the soil and gravel target includes one or more of the image of the uncovered dump truck that spilled the soil and gravel target and the actual physical position.

[0030] On the other hand, a soil and gravel spill monitoring system based on an unmanned aerial vehicle is also provided, which includes:

[0031] Unmanned aerial vehicle;

[0032] Imaging device, which is carried on the unmanned aerial vehicle and, during the flight of the unmanned aerial vehicle, obtains ground images in real time;

[0033] Road segmentation unit, which is used to segment the road area from the ground image to obtain a road image;

[0034] Target detection unit, which is used to perform target detection on the road image to determine whether there are uncovered dump truck targets on the road;

[0035] Target tracking unit, which is used to track all uncovered dump truck targets and all suspected soil and gravel spill targets during the time period from time T to time T+M to obtain the movement trajectories of all uncovered dump truck targets and the movement trajectories of all suspected soil and gravel spill targets;

[0036] Soil and gravel target filtering unit, which is used to determine the soil and gravel targets being spilled from all suspected soil and gravel spill targets;

[0037] A spilled construction waste truck determination unit, which is used to determine all uncovered construction waste truck targets in front of the current construction waste target that is being spilled at time T+M, and to determine the uncovered construction waste truck that spilled the construction waste target.

[0038] The present invention can track uncovered construction waste trucks and spilled construction waste based on a drone platform and a mounted imaging device, timely detect construction waste targets that are being spilled, and associate the construction waste targets with the corresponding uncovered construction waste trucks, so as to realize the on-road monitoring and management of construction waste trucks. While ensuring road traffic safety, it greatly reduces various pollutions on urban roads and overcomes the problem of low efficiency in manual inspection of construction waste trucks spilling construction waste. Description of the Drawings

[0039] Figure 1 It is a flowchart of the steps of the construction waste spill monitoring method based on drones in the present invention;

[0040] Figure 2 It is the construction waste target on the road surface identified by image recognition in the present invention and the uncovered construction waste truck target that spilled the construction waste target;

[0041] Figure 3 It is a schematic diagram of the movement trajectories of the uncovered construction waste truck target and the suspected construction waste spill target in the present invention;

[0042] Figure 4 It is a schematic structural diagram of the construction waste spill monitoring system based on drones in the present invention. Detailed Embodiments

[0043] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0044] Embodiment 1:

[0045] As Figure 1 shown, this embodiment provides a construction waste spill monitoring method based on drones, which includes the following steps:

[0046] S1. Control the drone to fly along a preset route. During the flight, the ground image is continuously obtained through the imaging device mounted on the drone. Among them, the imaging device includes a high-resolution visible light camera; at the same time, the imaging device obtains the ground image at an angle of 45°-90° from above to obtain a clear and complete ground image;

[0047] S2. Segment the road area from the ground image to obtain a road image. The segmentation of the road image can be achieved based on algorithms such as the MATLAB differential evolution algorithm and image semantic segmentation algorithm;

[0048] S3. Perform object detection on the road image to determine whether there are uncovered dump truck targets on the road. In this embodiment, there can be multiple uncovered dump truck targets, such as uncovered dump trucks Z1, Z2, Z3,..., Zi, etc.; among them, the detection of the uncovered dump truck targets can be achieved based on object detection algorithms of the YOLO series (such as YOLOv8), etc. The uncovered dump truck refers to a dump truck that does not cover the construction waste loaded on the dump truck. When there are uncovered dump truck targets on the road, step S4 is executed;

[0049] S4. In the time period from time T to time T+M, track all uncovered dump truck targets and all suspected spilled construction waste targets through tracking algorithms such as ByteTrack to obtain the movement trajectories of all uncovered dump truck targets and the movement trajectories of all suspected spilled construction waste targets; the value range of M is 0.5 - 3 seconds;

[0050] The suspected spilled construction waste targets include one or more of the construction waste targets in the uncovered dump truck carriage, the construction waste targets that have been spilled onto the road surface, and the construction waste targets that are being spilled. The construction waste targets in the uncovered dump truck carriage refer to the construction waste targets loaded in the uncovered dump truck carriage and that will not be spilled onto the road surface. The construction waste targets that are being spilled refer to the construction waste targets that are already in the air and will be spilled onto the road surface;

[0051] S5. Determine the construction waste targets that are being spilled from all suspected spilled construction waste targets, which specifically includes the following steps:

[0052] At time T+M, obtain the set U of construction waste targets of the current uncovered dump truck. The set U of construction waste targets contains at least one suspected spilled construction waste target, and for each suspected spilled construction waste target in the set U of construction waste targets, the IoU (i.e., Intersection-over-Union) with the current uncovered dump truck is greater than 0, where IoU = the intersection of the detection frame of the uncovered dump truck and the detection frame of the suspected spilled construction waste target / the union of the detection frame of the uncovered dump truck and the detection frame of the suspected spilled construction waste target. Repeat this step to obtain the set of construction waste targets for each uncovered dump truck;

[0053] Determine the uncovered dump truck Zi with the largest IoU with the current suspected spilled construction waste target at time T+M, and determine whether the current suspected spilled construction waste target is the suspected spilled construction waste target with the largest area in the construction waste target set Ui of the uncovered dump truck Zi. If so, it is considered that the current suspected spilled construction waste target is the construction waste target in the carriage of the uncovered dump truck Zi, and repeat this step to determine the construction waste target in each uncovered dump truck carriage;

[0054] Obtain the positions of the same suspected spilled construction waste target at time T and T+M respectively. If the distance of the suspected spilled construction waste target at time T and T+M is less than or equal to the threshold (for example, the threshold = 3), it is considered that the suspected spilled construction waste target has not moved during the time period from time T to T+M, and the suspected spilled construction waste target is the construction waste target that has been spilled onto the road surface (such as Figure 2 construction waste C and construction waste D in); among them, the positions of the suspected spilled construction waste target at time T and T+M can be determined by the detection box coordinates of the suspected spilled construction waste target at time T and T+M, and the distance at time T and T+M can also be calculated through the detection box coordinates;

[0055] Determine the remaining suspected spilled construction waste targets other than all the construction waste targets in the uncovered dump truck carriages and all the construction waste targets that have been spilled onto the road surface as the construction waste targets that are being spilled (such as Figure 3 construction waste A and construction waste B in);

[0056] S6. Determine all the uncovered dump truck targets in front of the current construction waste target that is being spilled at time T+M, and determine the uncovered dump truck that spills the construction waste target;

[0057] Among them, in front of the construction waste target is the driving direction of the uncovered dump truck, which can be determined by the moving trajectory of the uncovered dump truck; all the uncovered dump truck targets in front of the construction waste target can be determined according to the detection box coordinates of the current construction waste target that is being spilled and the uncovered dump truck targets at time T+M;

[0058] At the same time, determining the uncovered dump truck that spills the construction waste target includes the following steps:

[0059] If there is only one uncovered dump truck target in front of the current construction waste target that is being spilled at time T+M, directly identify this uncovered dump truck as the vehicle that spills the construction waste target;

[0060] If there are several uncovered dump trucks in front of the current construction waste target that is being spilled at time T+M, obtain the center point coordinates of the construction waste target and the center point coordinates of each uncovered dump truck target in front of the construction waste target; among them, the center point of the construction waste target and the center point of the uncovered dump truck target can both be determined through the coordinates of the target detection box;

[0061] Obtain the pixel distance between the center point of the currently spilling construction waste target and the center point of each uncovered construction waste truck target in front of the construction waste target;

[0062] Determine the uncovered construction waste truck target corresponding to the minimum pixel distance as the vehicle that spills the construction waste target;

[0063] For example, at time T+M, there may be several uncovered construction waste trucks in front of the currently spilling construction waste target (such as construction waste A). At this time, it is necessary to determine from which uncovered construction waste truck construction waste A is specifically spilled. Further, calculate the pixel distance between the center point of construction waste A and the center point of each uncovered construction waste truck. The minimum pixel distance indicates that construction waste A is closest to the uncovered construction waste truck (such as uncovered construction waste truck Z2), and the probability that construction waste A spills from this uncovered construction waste truck is relatively high. It can be considered that this uncovered construction waste truck is the vehicle that spills construction waste A;

[0064] S7. Based on the longitude and latitude of the unmanned aerial vehicle, the imaging attitude angle, height and other information of the camera, calculate the actual physical position of the currently spilling construction waste target when it falls on the road surface, and track the uncovered construction waste truck that spills the construction waste target, and send the construction waste target information on the road surface and the information of the uncovered construction waste truck that spills the construction waste target (such as Figure 2 shown) to the monitoring terminal;

[0065] Among them, the construction waste target information on the road surface includes one or several of the images of the construction waste target that has been spilled on the road surface, the actual physical position, etc.; the information of the uncovered construction waste truck that spills the construction waste target includes one or several of the images of the uncovered construction waste truck that spills the construction waste target, the actual physical position, etc.

[0066] Thus, based on the flexible unmanned aerial vehicle platform and the mounted imaging device in this embodiment, the uncovered construction waste trucks and the spilled construction waste are tracked, the currently spilling construction waste target is discovered in time, and the construction waste target is associated with the corresponding uncovered construction waste truck to realize the on-road monitoring and management of the construction waste trucks, ensuring road traffic safety while greatly reducing various pollutions on urban roads and overcoming the problem of low efficiency of manual inspection of construction waste trucks spilling construction waste.

[0067] Embodiment 2:

[0068] This embodiment provides a construction waste spillage monitoring system based on an unmanned aerial vehicle, which can implement the construction waste spillage monitoring method described in Embodiment 1, as Figure 4 shown, the construction waste spillage monitoring system includes:

[0069] Unmanned aerial vehicle 1;

[0070] An imaging device 2, which is mounted on the drone 1 and, during the flight of the drone 1, acquires ground images in real time;

[0071] A road segmentation unit 3, which is used to segment the road area from the ground image to obtain a road image;

[0072] A target detection unit 4, which is used to perform target detection on the road image to determine whether there are uncovered muck truck targets on the road;

[0073] A target tracking unit 5, which is used to track all uncovered muck truck targets and all suspected spilled muck targets during the time period from time T to time T+M to obtain the movement trajectories of all uncovered muck truck targets and the movement trajectories of all suspected spilled muck targets;

[0074] A muck target filtering unit 6, which is used to determine the muck targets that are currently spilling from all suspected spilled muck targets, and the process is the same as step S5;

[0075] A spilled muck truck determination unit 7, which is used to determine all uncovered muck truck targets in front of the currently spilling muck target at time T+M, and to determine the uncovered muck truck that spilled the muck target, and the process is the same as step S6;

[0076] A monitoring terminal 8, which is used to receive the muck target information on the road surface sent by the drone 1 and the information of the uncovered muck truck that spilled the muck target.

[0077] In summary, the present invention can track uncovered muck trucks and spilled muck based on a drone platform and a mounted imaging device, timely discover the muck targets that are currently spilling, and associate the muck targets with the corresponding uncovered muck trucks to achieve the on-road monitoring and management of muck trucks, while ensuring road traffic safety and greatly reducing various pollutions on urban roads, and overcoming the problem of low efficiency of manual inspection of muck trucks spilling muck.

[0078] It should be noted that the technical features in the above-mentioned embodiments 1-2 can be combined arbitrarily, and the combined technical solutions all fall within the protection scope of this application. In this text, terms such as "including", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0079] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for monitoring the spillage of construction waste based on an unmanned aerial vehicle, characterized in that, It includes the following steps: Control the drone to fly along a preset route, and during the flight, obtain ground images in real time through the imaging device carried on the drone; Segment the road area from the ground images to obtain road images, and perform object detection on the road images to determine whether there are uncovered muck truck targets on the road; Track all uncovered muck truck targets and all suspected spilled muck targets during the time period from T to T+M; Determine the muck targets that are currently spilling from all suspected spilled muck targets; Determine all uncovered muck truck targets in front of the currently spilling muck target at T+M, and determine the uncovered muck truck that spilled the muck target.

2. The method for monitoring soil and gravel spillage according to claim 1, wherein, The value range of M is 0.5 - 3 seconds.

3. The method for monitoring soil and debris spillage according to claim 1, characterized in that Determining the muck targets that are currently spilling from all suspected spilled muck targets includes the following steps: Determine the muck targets in the uncovered muck truck carriage and the muck targets that have been spilled onto the road surface; Determine the remaining suspected spilled muck targets other than all the muck targets in the uncovered muck truck carriages and all the muck targets that have been spilled onto the road surface as the muck targets that are currently spilling.

4. The method for monitoring soil and gravel spillage according to claim 3, wherein, Determining the muck targets in the uncovered muck truck carriage includes the following steps: At T+M, obtain the muck target set U of the current uncovered muck truck, and this muck target set U contains at least one suspected spilled muck target, and in this muck target set U, the IoU of each suspected spilled muck target and the current uncovered muck truck is greater than 0. Repeat this step to obtain the muck target set of each uncovered muck truck; Determine the uncovered muck truck Zi with the largest IoU with the current suspected spilled muck target at T+M, and determine whether the current suspected spilled muck target is the suspected spilled muck target with the largest area in the muck target set Ui of this uncovered muck truck Zi. If so, consider the current suspected spilled muck target as the muck target in the uncovered muck truck Zi carriage. Repeat this step to determine the muck targets in each uncovered muck truck carriage.

5. The monitoring method for soil residue scattering according to claim 3, characterized in that, Determining the muck targets that have been spilled onto the road surface includes the following steps: Obtain the positions of the same suspected spilled muck target at T and T+M respectively. If the distance of this suspected spilled muck target at T and T+M is less than or equal to the threshold, it is considered that this suspected spilled muck target has not moved during the time period from T to T+M, and this suspected spilled muck target is the muck target that has been spilled onto the road surface.

6. The monitoring method for soil and gravel spillage according to claim 1, characterized in that, Determining the uncovered muck truck that spilled the muck target includes the following steps: If at T+M, there is only one uncovered muck truck target in front of the currently spilling muck target, directly identify this uncovered muck truck as the vehicle that spilled the muck target.

7. The method for monitoring soil and debris spillage according to claim 1, characterized in that, Determining the uncovered muck truck that spilled the muck target includes the following steps: If at T+M, there are several uncovered muck trucks in front of the currently spilling muck target, obtain the center point coordinates of this muck target and the center point coordinates of each uncovered muck truck target in front of this muck target; Obtain the pixel distance between the center point of the currently spilling construction waste target and the center point of each uncovered construction waste truck target in front of the construction waste target; Determine the uncovered construction waste truck target corresponding to the minimum pixel distance as the vehicle that spills the construction waste target.

8. The method for monitoring soil residue scattering according to claim 1, characterized in that, The construction waste spillage monitoring method further includes the following steps: Calculate the actual physical position of the construction waste target when it falls onto the road surface during the current spilling of the construction waste target, track the uncovered construction waste truck that spills the construction waste target, and send the construction waste target information on the road surface and the information of the uncovered construction waste truck that spills the construction waste target to the monitoring terminal through the drone.

9. The method for monitoring soil residue spillage according to claim 8, wherein, The construction waste target information on the road surface includes one or more of the construction waste target image already spilled on the road surface and the actual physical position; the information of the uncovered construction waste truck that spills the construction waste target includes one or more of the image of the uncovered construction waste truck that spills the construction waste target and the actual physical position.

10. An unmanned aerial vehicle-based monitoring system for construction waste spillage, characterized in that, It includes: Drone; Imaging device, which is carried on the drone and obtains ground images in real time during the flight of the drone; Road segmentation unit, which is used to segment the road area from the ground image to obtain a road image; Target detection unit, which is used to detect targets in the road image to determine whether there are uncovered construction waste truck targets on the road; Target tracking unit, which is used to track all uncovered construction waste truck targets and all suspected construction waste spillage targets during the time period from time T to time T+M to obtain the movement trajectories of all uncovered construction waste truck targets and the movement trajectories of all suspected construction waste spillage targets; Construction waste target filtering unit, which is used to determine the currently spilling construction waste targets from all suspected construction waste spillage targets; Spilled construction waste truck determination unit, which is used to determine all uncovered construction waste truck targets in front of the currently spilling construction waste target at time T+M and determine the uncovered construction waste truck that spills the construction waste target.

Images