Method and system for monitoring occupation of a road based on a drone nacelle device
By combining high-precision remote sensing imagery and deep learning with a drone pod device, road encroachment phenomena in rural areas can be identified, solving the problem of difficulty in identification in rural environments using traditional methods, and achieving high-precision monitoring and management support for road encroachment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU CHAOMO INTELLIGENT TECHNOLOGY CO LTD
- Filing Date
- 2025-01-20
- Publication Date
- 2026-07-21
Smart Images
Figure CN122435481A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the fields of UAV remote sensing technology, image processing technology and target detection, specifically to a method and system for monitoring rural roadside dwellings occupying roads based on a UAV pod device. Background Technology
[0002] Currently, monitoring technologies for road obstruction mainly include: target detection for identifying obstructing objects, deep learning for judging obstruction behavior, and comparison of real-time remote sensing image data with standard remote sensing image data. However, existing technologies have the following shortcomings in rural road scenarios: insufficient monitoring coverage in rural roads, and the lack of lane lines, guardrails, green belts, and road markings makes traditional video surveillance-based methods difficult to apply; the complex shapes of buildings and debris on both sides of rural roads make it difficult to determine whether they obstruct the road; existing road measurement methods, such as the GSD principle, are affected by the flight altitude and angle of drones, resulting in large calculation errors, and have high requirements for the selected reference objects. Summary of the Invention
[0003] This disclosure provides a method and system for monitoring rural roadside residential buildings encroaching on roads based on an unmanned aerial vehicle (UAV) pod device, in order to solve the problem that it is difficult to accurately identify road encroachment phenomena in rural road environments in the prior art.
[0004] This disclosure provides an embodiment of a road occupancy monitoring method based on a drone pod device. The method includes: controlling a drone with a pod to cruise and acquiring ground images through a camera in the pod; for each frame of the ground image, extracting ground object features through semantic segmentation, and identifying a reference object and a monitored object based on the ground object features using a pre-trained recognition model, wherein the size and specifications of the reference object are known; calculating the actual size of the monitored object based on the size and specifications of the reference object, the number of pixels the reference object spans in the ground image, and the number of pixels the monitored object spans in the ground image; and determining whether there is a road occupancy phenomenon by comparing the actual size with the standard value of the monitored object.
[0005] In embodiments of this disclosure, the monitoring object may include rural residential roads without marking lines, edge guardrails, or green belts.
[0006] In embodiments of this disclosure, the standard value of the monitored object may include the registered width of the monitored object during construction or a preset value determined based on rural road traffic flow.
[0007] In embodiments of this disclosure, calculating the actual size of the monitored object based on the size specifications of the reference object, the number of pixels the reference object spans in each frame of the ground image, and the number of pixels the monitored object spans in each frame of the ground image includes: calculating according to the following formula:
[0008]
[0009] a represents the number of pixels spanned by the reference object in the ground image, b represents the number of pixels spanned by the monitored object in the ground image, c represents the size specification of the reference object, and d represents the actual size of the monitored object.
[0010] In embodiments of this disclosure, the reference object may include a trash can or utility pole of the same size.
[0011] In embodiments of this disclosure, the method may further include the step of training the recognition model, comprising: pre-acquiring images of the area to be monitored, including a reference object and a monitored object, taken from multiple angles; extracting multi-angle features of the reference object and the monitored object through semantic segmentation based on the images taken from multiple angles; and training a deep neural network based on the multi-angle features to obtain a pre-trained recognition model.
[0012] In embodiments of this disclosure, the ground image includes a multispectral image captured by a multispectral camera and a panchromatic image captured by a panchromatic camera. The method may further include an image preprocessing step, which includes: performing atmospheric correction and radiometric calibration on the multispectral image and radiometric calibration on the panchromatic image; and fusing the multispectral image and panchromatic image using the Lαβ color space transfer method, including:
[0013] Step 1: Convert the RGB color space of the multispectral image P1 to the LMS color space, and calculate according to equation (1):
[0014]
[0015] Step 2: Convert the LMS color space to the logarithmic color space and calculate according to equation (2):
[0016]
[0017] Step 3: The color channels of the logarithmic color space are further converted into three orthogonal color channels to remove the correlation between the color spaces, as shown in Equation (3):
[0018]
[0019] exist In the diagram, L represents the non-color component, α represents the component from green to red, and β represents the component from blue to yellow; and
[0020] Step 4: Replace with panchromatic image P′1 The L non-color component in the image is converted back to an RGB image through an inverse transform.
[0021] Embodiments of this disclosure also provide a road occupancy monitoring system based on a drone pod device. The system comprises: an image acquisition module configured to control a drone carrying a pod to cruise and acquire ground images via a camera in the pod; a target recognition module configured to extract ground object features for each frame of ground image through semantic segmentation, and identify a reference object and a monitored object based on the ground object features using a pre-trained recognition model, wherein the size and specifications of the reference object are known; a data calculation module calculating the actual size of the monitored object based on the size and specifications of the reference object, the number of pixels the reference object spans in the ground image, and the number of pixels the monitored object spans in the ground image; and a decision control module determining whether there is road occupancy by comparing the actual size with the standard value of the monitored object.
[0022] Embodiments of this disclosure provide an electronic device, which includes a storage medium, a processor, and instructions or code stored on the storage medium and executable on the processor. When the processor executes the program, it implements the road occupancy monitoring method as described above.
[0023] Embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the road occupancy monitoring method described above.
[0024] This disclosure utilizes a drone pod device combined with high-precision remote sensing image acquisition, semantic segmentation and target recognition, reference object ratio calculation, road occupancy determination, and high-precision positioning to achieve automated and accurate monitoring of road occupancy by rural residences. By calculating the road width of the target object based on common rural reference objects such as garbage cans and utility poles, measurement accuracy is improved, overcoming the problem of large errors caused by the altitude and angle of the drone in traditional methods. Combined with a deep learning target recognition model, road information can be accurately extracted in complex rural environments such as those without road markings or guardrails. Finally, through real-time road occupancy determination and high-precision positioning, accurate labeling and monitoring of road occupancy areas are achieved, providing efficient and reliable technical support for rural road management and traffic planning. Attached Figure Description
[0025] Figure 1 This is a flowchart of a road occupancy monitoring method based on an unmanned aerial vehicle (UAV) pod device according to an embodiment of the present disclosure;
[0026] Figure 2 This is a flowchart of image preprocessing according to embodiments of the present disclosure;
[0027] Figure 3 This is a flowchart illustrating the calculation of the size of the monitored object according to an embodiment of the present disclosure;
[0028] Figure 4 This is a flowchart illustrating the decision-making process for determining whether a road is obstructed, based on an embodiment of this disclosure.
[0029] Figure 5 This is a block diagram of a road occupancy monitoring system based on an unmanned aerial vehicle (UAV) pod device according to embodiments of the present disclosure; and
[0030] Figure 6 This is a block diagram of an electronic device for performing a road occupancy monitoring method according to an embodiment of the present disclosure. Detailed Implementation
[0031] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various modifications, variations, and equivalents of the methods, apparatus, and / or systems described herein will be apparent to those skilled in the art. For example, the order of operations described herein is merely illustrative and is not limited to the order set forth herein; changes that will be apparent to those skilled in the art may be made, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features and structures well-known to those skilled in the art may be omitted. The features described herein may be implemented in different forms and will not be construed as being limited to the examples described herein. Rather, the examples provided herein make this disclosure thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.
[0032] Figure 1 This is a flowchart of a road occupancy monitoring method based on an unmanned aerial vehicle (UAV) pod device according to an embodiment of the present disclosure; Figure 2 This is a flowchart of image preprocessing according to embodiments of the present disclosure; Figure 3 This is a flowchart illustrating the calculation of the size of the monitored object according to an embodiment of the present disclosure; Figure 4 This is a flowchart illustrating the decision-making process for determining whether a road is obstructed, based on an embodiment of this disclosure.
[0033] Reference Figure 1 In step S1, the drone with a pod is controlled to cruise and collect ground images through the camera in the pod.
[0034] In an embodiment, the ground image may include a multispectral image captured by a multispectral camera and a panchromatic image captured by a panchromatic camera.
[0035] Reference Figure 2 The road occupancy monitoring method may further include an image preprocessing step.
[0036] The image preprocessing steps include: performing atmospheric correction and radiometric calibration on the multispectral image and radiometric calibration on the panchromatic image; and fusing the multispectral image and the panchromatic image using the Lαβ color space transfer method.
[0037] The steps for fusing the multispectral image and the panchromatic image include:
[0038] Step 1: Convert the RGB color space of the multispectral image P1 to the LMS color space, and calculate according to equation (1):
[0039]
[0040] Step 2: Convert the LMS color space to the logarithmic color space and calculate according to equation (2):
[0041]
[0042] Step 3: The color channels of the logarithmic color space are further converted into three orthogonal color channels to remove the correlation between the color spaces, as shown in Equation (3):
[0043]
[0044] exist In the diagram, L represents the non-color component, α represents the component from green to red, and β represents the component from blue to yellow; and
[0045] Step 4: Replace with panchromatic image P′1 The L non-color component in the image is converted back to an RGB image, i.e., a high-resolution multispectral image, through inverse transformation.
[0046] The image fusion method described above yields a fused image that combines the high spectral resolution of a multispectral image with the high spatial resolution of a panchromatic image, effectively improving the accuracy of identifying road boundaries and surrounding features. The Lαβ color space transfer method, by decomposing color information, enhances the color contrast between different features, resulting in clearer color distribution characteristics for target roads, buildings, and vegetation, thus improving the accuracy of semantic segmentation and target recognition. Furthermore, this fusion method enhances the overall brightness and detail of the image by preserving the L-non-color components of the panchromatic image, ensuring clear extraction of road information even in low-light or complex environments. This fusion strategy effectively reduces recognition errors caused by factors such as changes in illumination and differences in reflectivity of ground features, thereby improving the reliability and adaptability of UAV remote sensing monitoring systems and making it suitable for detecting road occupancy in complex rural road environments.
[0047] Return to reference Figure 1 In step S2, for each frame of ground image, ground object features are extracted through semantic segmentation, and reference objects and monitoring objects are identified through a pre-trained recognition model based on the ground object features, wherein the size and specifications of the reference objects are known.
[0048] Specifically, the monitoring targets include rural residential roads without road markings, edge guardrails, or green belts. Due to the complex shapes and lack of standardized configurations of these roads, using UAV remote sensing images combined with semantic segmentation and target recognition technology can accurately extract road boundary information for monitoring road occupancy. The reference objects include uniformly sized garbage bins or utility poles; these objects are typically uniformly configured by the county or township, have fixed dimensions, and can serve as known benchmarks for road width calculation. This will be further explained in detail below with reference to step S3.
[0049] In step S3, the actual size of the monitored object is calculated based on the size specifications of the reference object, the number of pixels the reference object spans in the ground image, and the number of pixels the monitored object spans in the ground image.
[0050] In remote sensing images, by statistically analyzing the number of pixels of a reference object and combining it with its actual size, the conversion ratio between pixels and actual size can be accurately calculated, thereby measuring the true width of the road. Because the reference object and the detected object are obtained from the same ground image (e.g., the same frame), this reference object-based calculation method is less affected by factors such as UAV altitude and tilt angle compared to the traditional GSD (Ground Resolution) method, effectively reducing measurement errors and improving the accuracy and reliability of road occupancy detection.
[0051] During road width calculation, to ensure measurement accuracy and calculation validity, the system requires each image frame to simultaneously include a reference object (such as a trash can or utility pole) and the monitored object (the road). However, in some cases, only one may be included. In such cases, it is impossible to establish a mapping relationship between pixel size and actual size, resulting in inaccurate road width calculation. To address this anomaly, the frame data can be automatically discarded; that is, the frame will not participate in subsequent width calculations or road occupancy determination. Compensation calculations can be attempted using adjacent frames based on multiple consecutive images captured by the drone. If multiple adjacent frames do not meet the requirements, the system returns to step S1 to re-capture the area corresponding to that frame, ensuring that both the reference object and the monitored object appear in the same frame, thus making the monitoring data complete and reliable.
[0052] Reference Figure 1 and Figure 3 The actual size of the monitored object is calculated according to the following formula:
[0053]
[0054] a represents the number of pixels spanned by the reference object in the ground image, b represents the number of pixels spanned by the monitored object in the ground image, c represents the size specification of the reference object, and d represents the actual size of the monitored object.
[0055] Return to reference Figure 1 In step S4, the actual size is compared with the standard value of the monitored object to determine whether there is a road obstruction.
[0056] Reference Figure 1 and Figure 4 In step S4, the presence of road encroachment is determined by comparing the actual dimensions of the monitored object (e.g., the actual width of the monitored road) with a preset standard value. The standard value of the monitored object includes the registered width at the time of construction or a preset value determined based on rural road traffic flow. The standard value of the monitored object can be determined in two ways: one is based on the registered width at the time of road construction, using the design width in the construction plan as a standard reference value; the other is to preset a width threshold based on the actual traffic flow of rural roads, dynamically adjusting the standard value under different traffic demands to improve adaptability. This dual standard value setting method makes this disclosure applicable not only to the detection of road encroachment on newly built roads but also to the actual conditions of old or renovated roads, improving the rationality of the monitoring results.
[0057] If the actual size is smaller than the standard value, it is determined that the buildings or debris on both sides of the road section are occupying the road, and an occupancy warning signal is generated, and RTK positioning marking is performed on the occupied area.
[0058] RTK (Real-Time Kinematic) positioning markers are a real-time dynamic measurement technology based on carrier phase observations and Real-Time Differential GPS (RTDGPS). This technology enables high-precision positioning of road occupancy areas, obtaining accurate geographic coordinate data and transmitting the location information in real-time to a backend monitoring system. Upon receiving this data, the backend system can further generate road occupancy monitoring reports, visualize the road occupancy situation, and provide early warning information, allowing management departments or relevant personnel to take appropriate measures, thereby effectively improving the intelligence and automation level of rural road occupancy supervision.
[0059] Figure 5 This is a block diagram of a road occupancy monitoring system based on an unmanned aerial vehicle (UAV) pod device, according to an embodiment of the present disclosure.
[0060] Reference Figure 5 The road occupancy monitoring system 10 based on the UAV pod device includes: an image acquisition module 501, a target recognition module 502, a data calculation module 503, and a decision control module 504.
[0061] Image acquisition module 501 is configured to control the cruise of a UAV with a pod, acquiring ground images via a camera in the pod. Image acquisition module 501 is configured to perform reference... Figure 1 The described steps S1 and references Figure 2 The image preprocessing described herein is omitted here for brevity.
[0062] The target recognition module 502 is configured to extract ground object features through semantic segmentation for each frame of ground image, and identify reference objects and monitored objects based on the ground object features using a pre-trained recognition model. The size and specifications of the reference objects are known. The target recognition module 502 is configured to perform reference... Figure 1 Step S2, which is described in detail, is omitted here for brevity.
[0063] The data calculation module 503 is configured to calculate the actual size of the monitored object based on the size specifications of the reference object, the number of pixels spanned by the reference object in the ground image, and the number of pixels spanned by the monitored object in the ground image. The data calculation module 503 is configured to perform reference... Figure 1 and Figure 3 Step S3, which is described in detail, is omitted here for brevity.
[0064] The decision control module 504 is configured to determine whether there is road obstruction by comparing the actual dimensions with the standard values of the monitored object. The decision control module 504 is configured to execute reference... Figure 1 and Figure 4 Step S4, which is described in detail, is omitted here for brevity.
[0065] Figure 6This is a block diagram of an electronic device for performing a road occupancy monitoring method according to an embodiment of the present disclosure.
[0066] Reference Figure 6 An electronic device 600 according to embodiments of the present disclosure may include a processor 610 and a memory 620. The processor 610 may include (but is not limited to) a central processing unit (CPU), a digital signal processor (DSP), a microcomputer, a field-programmable gate array (FPGA), a system-on-a-chip (SoC), a microprocessor, an application-specific integrated circuit (ASIC), etc. The memory 620 may store computer programs to be executed by the processor 610. The memory 620 includes high-speed random access memory and / or non-volatile computer-readable storage media. When the processor 610 executes the computer program stored in the memory 620, the road occupancy monitoring method based on the unmanned aerial vehicle (UAV) pod device described above can be implemented.
[0067] The road occupancy monitoring method based on an unmanned aerial vehicle (UAV) pod according to embodiments of this disclosure can be programmed into a computer program and stored on a computer-readable storage medium. When the computer program is executed by a processor, the road occupancy monitoring method based on an UAV pod as described above can be implemented. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. In one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.
[0068] While some embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.
[0069] It should be noted that the functions or steps that the storage medium or electronic device can achieve are described in the relevant descriptions in the foregoing method embodiments, and will not be repeated here.
[0070] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the disclosed patent. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A method for monitoring road occupancy based on an unmanned aerial vehicle (UAV) pod device, characterized in that, The drone with a pod as its payload cruises and collects ground images through a camera in the pod. For each frame of ground image, ground object features are extracted through semantic segmentation, and reference objects and monitoring objects are identified through a pre-trained recognition model based on the ground object features, wherein the size and specifications of the reference objects are known. The actual size of the monitored object is calculated based on the size specifications of the reference object, the number of pixels the reference object spans in the ground image, and the number of pixels the monitored object spans in the ground image; and Whether there is road obstruction is determined by comparing the actual size with the standard value of the monitored object.
2. The method for monitoring road occupancy according to claim 1, characterized in that, The monitoring targets include rural residential roads without markings, edge guardrails, or green belts.
3. The method for monitoring road occupancy according to claim 1, characterized in that, The standard value of the monitored object includes the registered width of the monitored object during construction or a preset value determined based on rural road traffic flow.
4. The method for monitoring road occupancy according to claim 1, characterized in that, The step of calculating the actual size of the monitored object based on the size specifications of the reference object, the number of pixels the reference object spans in the ground image, and the number of pixels the monitored object spans in the ground image includes: calculating according to the following formula: a represents the number of pixels spanned by the reference object in the ground image, b represents the number of pixels spanned by the monitored object in the ground image, c represents the size specification of the reference object, and d represents the actual size of the monitored object.
5. The method for monitoring road occupancy according to claim 1, characterized in that, The reference objects include trash cans or utility poles of the same size.
6. The method for monitoring road occupancy according to claim 1, characterized in that, The method further includes the step of training the recognition model, including: Pre-acquire images of the area to be monitored, including the reference object and the monitored object, taken from multiple angles; Based on the images captured from multiple angles, multi-angle features of the reference object and the monitored object are extracted through semantic segmentation; based on the multi-angle features of the reference object and the monitored object, a deep neural network is trained to obtain a pre-trained recognition model.
7. The method for monitoring road occupancy according to claim 1, characterized in that, The ground image includes a multispectral image captured by a multispectral camera and a panchromatic image captured by a panchromatic camera, and the method further includes an image preprocessing step, which includes: Atmospheric correction and radiometric calibration are performed on the multispectral image, and radiometric calibration is performed on the panchromatic image; The multispectral image and the panchromatic image are fused using the Lαβ color space transfer method, including: Step 1: Convert the RGB color space of the multispectral image P1 to the LMS color space, and calculate according to equation (1): Step 2: Convert the LMS color space to the logarithmic color space and calculate according to equation (2): Step 3: The color channels of the logarithmic color space are further converted into three orthogonal color channels to remove the correlation between the color spaces, as shown in Equation (3): exist In the diagram, L represents the non-color component, α represents the component from green to red, and β represents the component from blue to yellow; and Step 4: Replace with panchromatic image P′1 The L non-color component in the image is converted back to an RGB image through an inverse transform.
8. A road occupancy monitoring system based on an unmanned aerial vehicle (UAV) pod device, characterized in that, The road occupancy monitoring system includes: The image acquisition module is configured to control the cruise of a drone with a pod, and to acquire ground images through a camera in the pod. The target recognition module is configured to extract ground object features through semantic segmentation for each frame of ground image, and identify reference objects and monitoring objects based on the ground object features through a pre-trained recognition model, wherein the size and specifications of the reference objects are known. The data calculation module is configured to calculate the actual size of the monitored object based on the size specifications of the reference object, the number of pixels the reference object spans in the ground image, and the number of pixels the monitored object spans in the ground image. The decision control module is configured to determine whether there is a road obstruction by comparing the actual size with the standard value of the monitored object.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the road occupancy monitoring method according to any one of claims 1 to 7.
10. A computing device, characterized in that, The computing device includes: processor; and A memory storing a computer program that, when executed by a processor, implements the road occupancy monitoring method according to any one of claims 1 to 7.