Smart city supervision service system based on video monitoring

Abstract

The invention discloses a smart city supervision service system based on video monitoring, and relates to the technical field of smart cities, point cloud data of original video streams and surface three-dimensional coordinate points are acquired by deploying a plurality of shooting devices on each main road of a city, and the point cloud data of the original video streams and the surface three-dimensional coordinate points are acquired by analyzing the original video streams; the obtained pavement damage size data and pavement damage area data are associated, if the current main road is not in a failure state, a surface settlement analysis instruction is sent, the road surface settlement condition is further analyzed in combination with the three-dimensional point cloud data, and a surface flatness coefficient Xpz is generated; the method comprises the following steps: acquiring road surface damage size data and road surface damage area data, correlating with the corresponding road surface damage size data and road surface damage area data, calculating a comprehensive disease assessment index Zbh, comparing the comprehensive disease assessment index Zbh with a preset disease assessment threshold P, and judging whether a road meets normal driving requirements or not so as to generate a corresponding road maintenance grade instruction and execute corresponding maintenance measures.

Description

Technical Field

[0001] The present invention relates to the technical field of smart cities, and in particular to a smart city supervision service system based on video surveillance. Background Art

[0002] As an important direction of modern urban development, smart cities cover the digital and intelligent management of various urban infrastructures to improve urban operation efficiency, residents' quality of life and sustainable development capabilities. Road supervision in smart cities is a key area, among which the monitoring and maintenance of urban roads are particularly important, especially the disease monitoring and management of trunk roads. With the acceleration of urbanization, the problem of road diseases is becoming increasingly serious. How to monitor and effectively evaluate road conditions in real time has become an urgent problem to be solved in smart city management.

[0003] At present, the disease monitoring and management of urban trunk roads mostly rely on manual inspections, local inspections or fixed-point monitoring. This traditional method has many limitations. First, manual inspections are not only long-term and inefficient, but also easily affected by human omissions and subjective judgments, which makes it difficult to find existing cracks and potholes in time, thus affecting the timely maintenance of roads. Secondly, the existing evaluation methods lack effective comprehensive analysis methods and only rely on a single detection index to assess the degree of road disease. Moreover, monitoring equipment can often only cover specific areas and it is difficult to provide comprehensive and continuous road surface data. There is also a lack of comprehensive analysis and evaluation capabilities for road diseases, resulting in inaccurate evaluation results, which in turn affects the rationality of repair decisions and resource allocation. Therefore, there is an urgent need for a smart city supervision service system based on video surveillance. Through full-process dynamic monitoring, data collection and intelligent analysis, it can obtain road cracks, potholes, and settlement information in real time, improve the accuracy and timeliness of road status evaluation, and further improve the intelligent level of road maintenance efficiency. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a smart city supervision service system based on video surveillance, which solves the problems in the above-mentioned background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a smart city supervision service system based on video surveillance, including a video surveillance module, a data acquisition module, a road damage analysis module, a surface flatness analysis module, a road disease assessment module and a management module;

[0006] The video monitoring module is based on multiple shooting devices deployed on various main roads in the city to shoot panoramic images of the main roads, and input the respectively acquired original video streams and point cloud data of surface three-dimensional coordinate points into the central monitoring platform for storage;

[0007] The data acquisition module is used to extract key image frames from the original video stream in the central monitoring platform, construct a continuous image frame sequence, and extract the road surface damage size data and road surface damage area data after grayscale and denoising.

[0008] The pavement damage analysis module is used to analyze the pavement damage size data and the pavement damage area data to determine whether the current trunk road is in a failure state. If not, a surface settlement analysis instruction is issued;

[0009] The surface flatness analysis module is used to analyze the surface settlement of the trunk road that is not in a failure state after receiving the surface settlement analysis instruction, and obtain the surface flatness coefficient Xpz;

[0010] The road damage assessment module is used to associate the surface roughness coefficient Xpz with the corresponding road damage size data and road damage area data, and obtain the comprehensive damage assessment index Zbh by fitting;

[0011] The management module is used to compare and analyze the comprehensive disease assessment index Zbh with a preset disease assessment threshold P, determine whether the main roads that are not currently in a failed state meet the normal driving requirements of urban vehicles, generate corresponding road maintenance level instructions and execute them.

[0012] Preferably, the video monitoring module includes a deployment unit and a monitoring unit;

[0013] The deployment unit is used to deploy multiple high-resolution cameras and depth cameras on various main roads in the city, and connect the multiple high-resolution cameras and depth cameras to the central monitoring platform through wired connections, and the central monitoring platform is used to store data;

[0014] The monitoring unit is used to use a high-resolution camera equipped with a multi-spectral sensor to perform timed shooting of panoramic images of the main road by setting a shooting period, obtain the original video stream of the main road, and obtain the horizontal height difference Dj of each point on the road surface based on the structured light technology used by the depth camera and combined with the reference plane of the initial road surface state, and construct point cloud data of the three-dimensional coordinate points on the surface; the obtained original video stream and point cloud data of the main road are sent to the central monitoring platform for storage.

[0015] Preferably, the data acquisition module is used to extract key image frames at the same time interval from the original video stream in the central monitoring platform, construct a continuous image frame sequence, extract feature points in the continuous image frames by a scale-invariant feature transformation algorithm, align the matching feature points in two adjacent frames, and then use the homography matrix to perform continuous superposition perspective transformation on the continuous image frames to generate a road surface superposition image; after the road surface superposition image is grayed, it is denoised by a Gaussian filter to eliminate random noise in the road surface superposition image, and then the Canny operator is used to detect the boundary features of the crack edge, pothole edge and outer contour edge in the road surface superposition image to generate a road surface gradient edge image;

[0016] Based on the generated road gradient edge image, after feature extraction and classification, the road damage size data and road damage area data are obtained respectively. The road damage size data includes each crack width Lfk, each crack length Lfc and each pothole diameter Lkz. The road damage area data includes each crack area Sfm, each pothole area Skm and the total road surface area Sz.

[0017] Preferably, the pavement damage analysis module includes a damage size analysis unit, a damage area analysis unit and a determination unit;

[0018] The damage size analysis unit is used to analyze the road surface damage size data to obtain the road surface damage size degree coefficient Xcd. The road surface damage size degree coefficient Xcd is obtained by the following formula:

[0019]

[0020] Where Lfk i Expressed as the width of the i-th crack, Lfc i It is represented by the length of the i-th crack, i = 1, 2, 3, ..., n, n represents the number of cracks, Lkz j It is represented as the diameter of the jth pothole, j = 1, 2, 3, ..., m, m is the number of potholes, α1, α2 and α3 are all weight values;

[0021] According to the historical data in the central monitoring platform, the damage size threshold T is pre-set, and the road damage size coefficient Xcd is compared with the damage size threshold T to generate a relevant road damage size signal. The specific process is as follows:

[0022] When the pavement damage size coefficient Xcd is greater than or equal to the damage size threshold T, a "pavement damage size severe signal" is generated; when the pavement damage size coefficient Xcd is less than the damage size threshold T, a "pavement damage size mild signal" is generated, and the generated "pavement damage size severe signal" and "pavement damage size mild signal" are sent to the damage degree assessment unit;

[0023] The relevant road surface damage size degree signal includes a “road surface damage size severe signal” and a “road surface damage size mild signal”.

[0024] Preferably, the damaged area analysis unit is used to analyze the pavement damaged area data to obtain the pavement damaged area ratio Vzb, and the pavement damaged area ratio Vzb is obtained by the following formula:

[0025]

[0026] Where Sfm i It is represented by the i-th crack area, i = 1, 2, 3, ..., n, n represents the number of cracks, Skm j It is represented as the area of ​​the jth pothole, j = 1, 2, 3, ..., m, m represents the number of potholes, and Sz represents the total road surface area;

[0027] According to the numerical value of the road damage area ratio Vzb, a relevant road damage area degree signal is generated. The specific process is as follows:

[0028] When the pavement damage area ratio Vzb is greater than or equal to 15%, a "pavement damage area severe signal" is generated; when the pavement damage area ratio Vzb is less than 15%, a "pavement damage area mild signal" is generated, and the generated "pavement damage area severe signal" and "pavement damage area mild signal" are sent to the damage degree assessment unit;

[0029] The relevant road surface damage area degree signal includes a "road surface damage area severe signal" and a "road surface damage area mild signal".

[0030] Preferably, the determination unit is used to analyze the received relevant road surface damage size degree signal and relevant road surface damage area degree signal to determine whether to issue a surface settlement analysis instruction. The specific process is as follows:

[0031] According to the relevant road surface damage size degree signals, a set No. 1 X is established, the "light road surface damage size signal" is marked as element a1, and the "heavy road surface damage size signal" is marked as element a2, and element a1∈set No. 1 X, element a2∈set No. 1 X;

[0032] According to the relevant road damage area degree signals, a second set Y is established, the "light road damage area signal" is marked as element b1, the "heavy road damage area signal" is marked as element b2, and the element b1∈the second set Y, the element b2∈the second set Y;

[0033] Take the union of set 1 X and set 2 Y. If X∪Y={a1, b2} or {a2, b1} or {a2, b2}, it means that the current trunk road is in an invalid state. Inform the relevant municipal departments to completely resurface the current trunk road.

[0034] If X∪Y={a1,b1}, it means that the current trunk road is not in a failed state, and a surface settlement analysis instruction is issued.

[0035] Preferably, the surface flatness analysis module is used to construct a road surface settlement distribution map based on the horizontal height difference Dj of each road surface point in the point cloud data of the three-dimensional coordinate points of the surface after receiving the surface settlement analysis instruction. The road surface settlement distribution map is used to display the settlement status of different areas of the road surface, and the road surface settlement distribution map is evenly divided into a number of groups of grid unit areas, and the point cloud data of the three-dimensional coordinate points of the surface are mapped into each grid unit area to extract the settlement amount Ttx in each grid unit area, and the settlement amount mean is obtained according to the averaging algorithm.

[0036] Based on the constructed road surface settlement distribution map, the surface settlement of the main roads that are not in a failure state is analyzed to obtain the surface flatness coefficient Xpz, which is obtained by the following formula:

[0037]

[0038] Where, Ttx s It is expressed as the amount of settlement in the sth grid unit area, s = 1, 2, 3, ..., a, a represents the number of grid unit areas, Expressed as the mean sedimentation amount.

[0039] Preferably, the road damage assessment module is used to associate the surface roughness coefficient Xpz of the trunk road that is not in a failed state with the corresponding pavement damage size data and pavement damage area data, and after dimensionless processing, to obtain the comprehensive damage assessment index Zbh of the current trunk road by fitting. The comprehensive damage assessment index Zbh is obtained by the following formula:

[0040]

[0041] Where Xcd is the pavement damage size coefficient, Vzb is the pavement damage area ratio, β1, β2 and β3 are the weight values ​​of the surface flatness coefficient Xpz, the pavement damage size coefficient Xcd and the pavement damage area ratio Vzb, respectively, and A is the correction constant.

[0042] Preferably, the management module includes a comparison unit and an execution unit;

[0043] The comparison unit is used to pre-set the disease assessment threshold P according to the historical data in the central monitoring platform, and compare and analyze the comprehensive disease assessment index Zbh with the preset disease assessment threshold P to determine whether the trunk road that is not in a failure state currently meets the normal driving requirements of urban vehicles, and generate a corresponding road maintenance level instruction, the specific contents of which are as follows:

[0044] If the comprehensive disease assessment index Zbh ≥ the disease assessment threshold P, it means that the trunk road is not currently in a failed state and does not meet the normal driving requirements of urban vehicles, and a first-level road maintenance instruction is generated;

[0045] If the comprehensive disease assessment index Zbh is less than the disease assessment threshold P, it indicates that the trunk road is not currently in a failed state, meets the normal driving requirements of urban vehicles, and generates a secondary road maintenance instruction.

[0046] Preferably, the execution unit is used to receive the corresponding road maintenance level instruction generated by the comparison unit and execute the corresponding maintenance measures, the specific contents of which are as follows:

[0047] When receiving a first-level road maintenance order, the execution content is as follows: immediately notify the relevant municipal departments to start the emergency repair procedure, arrange a professional repair team to go to the site for a comprehensive inspection, and confirm the diseased area, mill the cracks and potholes, re-lay asphalt, fill the uneven road surface with asphalt and compact it, and after the repair is completed, conduct a quality inspection of the repaired road section to confirm that the repair quality meets the requirements, and formulate a long-term maintenance plan;

[0048] When receiving the secondary road maintenance instruction, the execution content is: continue to monitor and update the data of the main roads that are not currently in an invalid state and meet the normal driving requirements of urban vehicles, to ensure that the comprehensive disease assessment index Zbh is always less than the disease assessment threshold P.

[0049] The present invention provides a smart city supervision service system based on video surveillance, which has the following beneficial effects:

[0050] (1) Through full-process dynamic monitoring, data collection and intelligent analysis, it is possible to obtain information on road cracks, potholes and settlement in real time, thereby improving the accuracy and timeliness of road condition assessment and further enhancing the intelligent level of road maintenance efficiency. By taking panoramic images at regular intervals and combining them with point cloud data of three-dimensional coordinate points on the surface for in-depth processing, it is possible to monitor the road damage and flatness indicators in real time. During the road damage analysis process, it is possible to automatically extract road damage size data and road damage area data. After dimensionless processing and comprehensive analysis, it is possible to determine whether the current road is in a failure state. If not, a surface settlement analysis instruction is issued to ensure that potential risks are discovered in a timely manner. If the failure state has not been reached, further surface flatness analysis is performed and compared with the corresponding road damage size data and road damage area data. After dimensionless processing, the comprehensive disease assessment index Zbh of the current trunk roads is fitted. After comparative analysis, it is determined whether the trunk roads that are not in a failed state meet the normal driving requirements of urban vehicles, and the corresponding road maintenance level instructions are generated and corresponding maintenance measures are executed. In general, it provides a basis for road maintenance decision-making in intelligent road supervision in smart city supervision services, optimizes the allocation and scheduling of maintenance resources, avoids the inefficiency and subjective deviation of manual inspections, and effectively reduces traffic accidents and traffic congestion caused by road diseases. At the same time, it not only improves the accuracy and timeliness of road monitoring, but also optimizes the efficiency of road maintenance management, promotes the digitalization and intelligent management of road monitoring in smart city construction, and provides strong technical support for improving the travel experience of urban residents and ensuring traffic safety.

[0051] (2) Through the video monitoring module and data acquisition module, it can fully cover every part of the main road and collect road status information in real time. By combining structured light technology and depth cameras, it can obtain raw video streams and point cloud data to reflect the surface details of the road, especially cracks and potholes. After grayscale, denoising and feature extraction, these data can accurately extract road damage size data and road damage area data. This monitoring method can obtain relevant information on road diseases in a timely manner and provide data support for subsequent evaluation and decision-making.

[0052] (3) The pavement damage analysis module can timely assess the damage status of urban trunk roads, which helps to identify and screen roads that have already failed in advance, reducing the number of major traffic accidents caused by road function failure. First, the damage size analysis unit and the damage area analysis unit combine the size and area data of cracks and potholes, and conduct a comprehensive analysis of the obtained pavement damage size coefficient Xcd and pavement damage area ratio Vzb to determine whether the current trunk road is in a failed state, ensuring that major safety hazards are dealt with in a timely manner. If it is not in a failed state, a surface settlement analysis instruction will be issued. At the same time, when it has not reached a failed state, it can actively conduct further flatness analysis to provide a scientific basis for subsequent road maintenance.

[0053] (4) The pavement damage assessment module and management module are based on the main roads that are not in a failed state, and further conduct a comprehensive analysis of the surface flatness coefficient Xpz, the pavement damage size coefficient Xcd and the pavement damage area ratio Vzb to generate a comprehensive damage assessment index Zbh, and compare it with the preset damage assessment threshold P; this intelligent assessment method can not only issue a first-level road maintenance instruction in a timely manner when the road damage reaches a certain standard, but also automatically generate a second-level road maintenance instruction when the damage is relatively minor to guide the maintenance work; the implementation of this process further improves the intelligent level of urban road supervision and maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 The present invention is a flow chart of a smart city supervision service system based on video surveillance. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Example 1

[0057] See also Figure 1 , the present invention provides a smart city supervision service system based on video monitoring, including a video monitoring module, a data acquisition module, a road damage analysis module, a surface flatness analysis module, a road disease assessment module and a management module;

[0058] The video monitoring module takes panoramic images of the main roads in the city based on multiple shooting devices deployed on various main roads in the city, and inputs the respectively acquired original video streams and point cloud data of surface three-dimensional coordinate points into the central monitoring platform for storage;

[0059] The data acquisition module is used to extract key image frames from the original video streams in the central monitoring platform, construct a continuous image frame sequence, and after grayscale and denoising processing, extract road surface damage size data and road surface damage area data respectively;

[0060] The road surface damage analysis module is used to analyze the road surface damage size data and the road surface damage area data to determine whether the current main road is in a failure state. If not, it issues a surface settlement analysis instruction;

[0061] The surface flatness analysis module is used to analyze the surface settlement condition of the current main road that is not in a failure state after receiving the surface settlement analysis instruction, and obtain the surface flatness coefficient Xpz;

[0062] The road disease assessment module is used to associate the surface flatness coefficient Xpz with the corresponding road surface damage size data and road surface damage area data, and fit to obtain a comprehensive disease assessment index Zbh;

[0063] The management module is used to compare and analyze the comprehensive disease assessment index Zbh with a preset disease assessment threshold P, determine whether the current main road that is not in a failure state meets the normal driving requirements of urban vehicles, and generate and execute corresponding road maintenance level instructions.

[0064] In this embodiment, the supervision and maintenance efficiency of urban roads is comprehensively improved through intelligent technical means. By using multiple shooting devices deployed on the main roads of the city and combining with structured light technology, the collected original video stream and point cloud data of the surface three-dimensional coordinate points are input into the central monitoring platform for storage, and then the key image frames are extracted from the original video stream. After grayscale and denoising, the acquired road surface damage size data and road surface damage area data are analyzed to determine whether the current main road is in a failure state. If not, a surface settlement analysis instruction is issued to further analyze the surface settlement of the road, calculate the surface flatness coefficient Xpz, and further evaluate the flatness state of the road. On this basis, through the infinite The road surface damage processing technology comprehensively analyzes the size, area and surface flatness of the road damage to generate a comprehensive disease assessment index Zbh to further evaluate the health status of the road; by comparing with the preset disease assessment threshold P, it can quickly determine whether the current road meets the normal driving requirements of urban vehicles, and automatically generate corresponding maintenance level instructions to guide subsequent maintenance operations. This process not only realizes real-time monitoring and automated analysis of road conditions, improves the accuracy and response speed of road maintenance, but also reduces the cost and time of manual inspections, improves the efficiency of resource allocation, and ensures the safety and smoothness of urban traffic through scientific data analysis and intelligent decision-making. At the same time, it also provides strong technical support for the long-term sustainable development of urban roads.

[0065] Example 2

[0066] Please refer to Figure 1 ,Specifically: the video monitoring module includes a deployment unit and a monitoring unit;

[0067] The deployment unit is used to deploy multiple high-resolution cameras and depth cameras on various main roads in the city, and connect the multiple high-resolution cameras and depth cameras to the central monitoring platform through wired connections, and the central monitoring platform is used to store data;

[0068] The monitoring unit is used to use a high-resolution camera equipped with a multi-spectral sensor to perform timed shooting of panoramic images of the main road by setting a shooting period, obtain the original video stream of the main road, and obtain the horizontal height difference Dj of each point on the road surface based on the structured light technology used by the depth camera and combined with the reference plane of the initial road surface state, and construct point cloud data of the three-dimensional coordinate points on the surface; the obtained original video stream and point cloud data of the main road are sent to the central monitoring platform for storage.

[0069] In this embodiment, through the highly integrated deployment unit and monitoring unit, all-round monitoring and analysis of urban trunk roads can be achieved; the deployment unit accurately arranges multiple high-resolution cameras and depth cameras along the road, and directly connects with the central monitoring platform through wired connection to ensure real-time transmission and storage of data; the monitoring unit uses the multi-spectral sensor carried by the high-resolution camera to perform timed panoramic shooting of the trunk road to capture the original video stream of the road. At the same time, through the structured light technology of the depth camera and combined with the reference surface data, the horizontal height difference Dj of each point on the road surface is accurately obtained to generate high-precision surface three-dimensional point cloud data; this process provides accurate basic data for the health status assessment of the road, which can not only clearly reflect the slight deformation and damage of the road surface, but also accurately quantify the degree of settlement or damage of the road surface. By storing and analyzing the original video data and point cloud data in real time, the central monitoring platform can provide a scientific basis for subsequent road disease assessment and maintenance decisions, improve the automation and intelligence level of road monitoring, and effectively reduce the errors and lags of manual inspections, providing efficient and accurate support for urban road management.

[0070] Example 3

[0071] Please refer to Figure 1 Specifically: the data acquisition module is used to extract key image frames at the same time interval from the original video stream in the central monitoring platform, construct a continuous image frame sequence, extract feature points in the continuous image frames through a scale-invariant feature transformation algorithm, align the matching feature points in two adjacent frames, and then use the homography matrix to perform continuous superposition perspective transformation on the continuous image frames to generate a road surface superposition image; after the road surface superposition image is grayed, it is denoised through a Gaussian filter to eliminate random noise in the road surface superposition image, and then the Canny operator is used to detect the boundary features of crack edges, pothole edges and outer contour edges in the road surface superposition image to generate a road surface gradient edge image;

[0072] Based on the generated road gradient edge image, after feature extraction and classification, the road damage size data and road damage area data are obtained respectively. The road damage size data includes each crack width Lfk, each crack length Lfc and each pothole diameter Lkz. The road damage area data includes each crack area Sfm, each pothole area Skm and the total road surface area Sz.

[0073] In this embodiment, through video analysis and precise data extraction, it is possible to efficiently provide data support for detailed monitoring and evaluation of road conditions; first, key image frames are extracted according to the same time interval, and the feature points in the image are extracted using a scale-invariant feature transformation algorithm, and adjacent image frames are aligned and perspective transformed through a homography matrix to generate a high-precision road surface overlay image; then, after grayscale processing and Gaussian filter denoising, random noise in the road surface overlay image is eliminated, and the Canny operator is used to detect the boundary features of cracks, potholes and the outer contour of the road surface to form a clear gradient edge image; this process not only enhances the accuracy of the image, but also lays a solid foundation for subsequent damage analysis; based on the gradient edge image, the road surface damage size data and road surface damage area data can be accurately obtained through feature extraction and classification; accurate data extraction not only provides a quantitative basis for the early detection and evaluation of road diseases, but also makes maintenance decisions more accurate and scientific. Through this highly automated data processing process, the accuracy and efficiency of intelligent road supervision are improved.

[0074] Example 4

[0075] Please refer to Figure 1 ,Specifically: the pavement damage analysis module includes a damage size analysis unit, a damage area analysis unit and a determination unit;

[0076] The damage size analysis unit is used to analyze the road surface damage size data to obtain the road surface damage size degree coefficient Xcd. The road surface damage size degree coefficient Xcd is obtained by the following formula:

[0077]

[0078] Where Lfk i Expressed as the width of the i-th crack, Lfc i It is represented by the length of the i-th crack, i = 1, 2, 3, ..., n, n represents the number of cracks, Lkz j It is represented by the diameter of the jth pothole, j = 1, 2, 3, ..., m, m is the number of potholes, α1, α2 and α3 are all weight values;

[0079] According to the historical data in the central monitoring platform, the damage size threshold T is pre-set, and the road damage size coefficient Xcd is compared with the damage size threshold T to generate a relevant road damage size signal. The specific process is as follows:

[0080] When the pavement damage size coefficient Xcd is greater than or equal to the damage size threshold T, a "pavement damage size severe signal" is generated; when the pavement damage size coefficient Xcd is less than the damage size threshold T, a "pavement damage size mild signal" is generated, and the generated "pavement damage size severe signal" and "pavement damage size mild signal" are sent to the damage degree assessment unit;

[0081] The relevant road surface damage size degree signal includes a “road surface damage size severe signal” and a “road surface damage size mild signal”.

[0082] Specifically, the damaged area analysis unit is used to analyze the road damaged area data to obtain the road damaged area ratio Vzb, and the road damaged area ratio Vzb is obtained by the following formula:

[0083]

[0084] Where Sfm i It is represented by the i-th crack area, i = 1, 2, 3, ..., n, n represents the number of cracks, Skm j It is represented as the area of ​​the jth pothole, j = 1, 2, 3, ..., m, m represents the number of potholes, and Sz represents the total road surface area;

[0085] According to the numerical value of the road damage area ratio Vzb, a relevant road damage area degree signal is generated. The specific process is as follows:

[0086] When the pavement damage area ratio Vzb is greater than or equal to 15%, a "pavement damage area severe signal" is generated; when the pavement damage area ratio Vzb is less than 15%, a "pavement damage area mild signal" is generated, and the generated "pavement damage area severe signal" and "pavement damage area mild signal" are sent to the damage degree assessment unit;

[0087] The relevant road surface damage area degree signal includes a "road surface damage area severe signal" and a "road surface damage area mild signal".

[0088] Specifically, the determination unit is used to analyze the received relevant road surface damage size degree signal and relevant road surface damage area degree signal to determine whether to issue a surface settlement analysis instruction. The specific process is as follows:

[0089] According to the relevant road surface damage size degree signals, a set No. 1 X is established, the "light road surface damage size signal" is marked as element a1, and the "heavy road surface damage size signal" is marked as element a2, and element a1∈set No. 1 X, element a2∈set No. 1 X;

[0090] According to the relevant road damage area degree signals, a second set Y is established, the "light road damage area signal" is marked as element b1, the "heavy road damage area signal" is marked as element b2, and the element b1∈the second set Y, the element b2∈the second set Y;

[0091] Take the union of set 1 X and set 2 Y. If X∪Y={a1, b2} or {a2, b1} or {a2, b2}, it means that the current trunk road is in a failed state and has major structural problems. It is impossible to restore the original function of the road surface by repairing it. In this case, notify the relevant municipal departments to completely resurface the current trunk road.

[0092] If X∪Y={a1,b1}, it means that the current trunk road is not in a failed state, and a surface settlement analysis instruction is issued.

[0093] In this embodiment, by accurately analyzing the pavement damage size data and the pavement damage area data, a scientific basis for early identification and screening of functional failures caused by road damage is provided; first, the damage size analysis unit quantifies the severity of the road damage size by calculating the pavement damage size degree coefficient Xcd according to each crack width Lfk, each crack length Lfc and each pothole diameter Lkz, and compares it with the preset damage size degree threshold T to generate damage signals of different degrees; the damage area analysis unit calculates the pavement damage area proportion Vzb, and makes a detailed assessment of the damage proportion range according to the pavement damage area proportion Vzb; by analyzing the damage size and Through comprehensive analysis of the damaged area signal, the judgment unit can determine whether to issue a surface settlement analysis instruction and make a decision on whether the road needs to be completely repaved. If the current main road pavement has failed, a repaving instruction can be issued quickly to avoid more serious traffic safety hazards caused by delayed repairs. If the current main road pavement has not failed, a flatness analysis instruction will be further issued. Through this highly integrated analysis process, different degrees of road damage can be identified and screened in advance in real-time monitoring, helping management departments to quickly take effective maintenance measures, improve urban road management efficiency and road safety, reduce the occurrence of traffic accidents, and ensure the continuous and efficient operation of the urban transportation system.

[0094] Example 5

[0095] Please refer to Figure 1Specifically: the surface flatness analysis module is used to construct a road surface settlement distribution map based on the horizontal height difference Dj of each road surface point in the point cloud data of the three-dimensional coordinate points of the surface after receiving the surface settlement analysis instruction. The road surface settlement distribution map is used to display the settlement status of different areas of the road surface. The road surface settlement distribution map is evenly divided into a number of groups of grid unit areas, and the point cloud data of the three-dimensional coordinate points of the surface are mapped to each grid unit area to extract the settlement amount Ttx in each grid unit area, and the settlement amount mean is obtained according to the averaging algorithm.

[0096] Based on the constructed road surface settlement distribution map, the surface settlement of the main roads that are not in a failure state is analyzed to obtain the surface flatness coefficient Xpz, which is obtained by the following formula:

[0097]

[0098] Where, Ttx s It is expressed as the amount of settlement in the sth grid unit area, s = 1, 2, 3, ..., a, a represents the number of grid unit areas, Expressed as the mean sedimentation amount.

[0099] In this embodiment, the surface settlement of the main roads that are not in a failed state is further analyzed to accurately calculate the settlement of the road surface, providing a scientific basis for the evaluation of road surface flatness and road maintenance decisions; after receiving the surface settlement analysis instruction, the point cloud data of the three-dimensional coordinate points is used to construct a road surface settlement distribution map to show the settlement conditions of different areas of the road surface in detail; by dividing the settlement distribution map into multiple grid units, the settlement amount Ttx of each grid unit can be extracted, and the mean settlement amount Ttx can be calculated by the averaging algorithm. This provides accurate settlement data for each grid area; these data are further used to calculate the surface flatness coefficient Xpz of the road surface and to quantitatively evaluate the road flatness. In this way, settlement problems on the road surface can be identified and potential flatness risks can be discovered in a timely manner, which can provide a scientific basis for road repair and maintenance.

[0100] Example 6

[0101] Please refer to Figure 1 Specifically: the road damage assessment module is used to associate the surface flatness coefficient Xpz of the trunk road that is not in an invalid state with the corresponding road surface damage size data and road surface damage area data, and after dimensionless processing, fit to obtain the comprehensive disease assessment index Zbh of the current trunk road. The comprehensive disease assessment index Zbh is obtained by the following formula:

[0102]

[0103] Where Xcd is the pavement damage size coefficient, Vzb is the pavement damage area ratio, β1, β2 and β3 are the weight values ​​of the surface flatness coefficient Xpz, the pavement damage size coefficient Xcd and the pavement damage area ratio Vzb, respectively, and A is the correction constant.

[0104] In this embodiment, the overall road damage condition is scientifically evaluated by comprehensively analyzing the multi-dimensional indicators of road surface flatness, damage size and damage area ratio; by weighted combination of the surface flatness coefficient Xpz, the road surface damage size coefficient Xcd and the road surface damage area ratio Vzb, and after dimensionless processing, a comprehensive damage assessment index Zbh can be generated to quantify the road health status indicator; the comprehensive damage assessment index Zbh can more comprehensively reflect all aspects of the road surface state, timely reveal the problems and damage degree of the road, and provide clear priority judgments for road maintenance and repair; this data-driven evaluation method not only improves the accuracy of road damage analysis, but also makes road maintenance decisions more scientific and accurate. Through accurate damage assessment, relevant municipal departments can realize dynamic monitoring and accurate maintenance of trunk roads, reduce road safety hazards, and extend the service life of roads, thereby improving the overall operation efficiency and safety of urban transportation systems.

[0105] Example 7

[0106] Please refer to Figure 1 ,Specifically: the management module includes a comparison unit and an execution unit;

[0107] The comparison unit is used to pre-set the disease assessment threshold P according to the historical data in the central monitoring platform, and compare and analyze the comprehensive disease assessment index Zbh with the preset disease assessment threshold P to determine whether the trunk road that is not in a failure state currently meets the normal driving requirements of urban vehicles, and generate a corresponding road maintenance level instruction, the specific contents of which are as follows:

[0108] If the comprehensive disease assessment index Zbh ≥ the disease assessment threshold P, it means that the trunk road is not currently in a failed state and does not meet the normal driving requirements of urban vehicles, and a first-level road maintenance instruction is generated;

[0109] If the comprehensive disease assessment index Zbh is less than the disease assessment threshold P, it indicates that the trunk road is not currently in a failed state, meets the normal driving requirements of urban vehicles, and generates a secondary road maintenance instruction.

[0110] Specifically, the execution unit is used to receive the corresponding road maintenance level instruction generated by the comparison unit and execute the corresponding maintenance measures, the specific contents of which are as follows:

[0111] When receiving a first-level road maintenance order, the execution content is as follows: immediately notify the relevant municipal departments to start the emergency repair procedure, arrange a professional repair team to go to the site for a comprehensive inspection, and confirm the diseased area, mill the cracks and potholes, re-lay asphalt, fill the uneven road surface with asphalt and compact it, and after the repair is completed, conduct a quality inspection of the repaired road section to confirm that the repair quality meets the requirements, and formulate a long-term maintenance plan;

[0112] When receiving the secondary road maintenance instruction, the execution content is: continue to monitor and update the data of the main roads that are not currently in an invalid state and meet the normal driving requirements of urban vehicles, to ensure that the comprehensive disease assessment index Zbh is always less than the disease assessment threshold P.

[0113] In this embodiment, by comparing and analyzing the comprehensive disease assessment index Zbh with the preset disease assessment threshold P, the health status of the road is accurately judged, and then the corresponding road maintenance level instructions are generated; if the main road that is not in a failed state does not meet the normal driving requirements of urban vehicles, a first-level road maintenance instruction is generated, and an emergency repair procedure is immediately started to ensure that serious problems are quickly resolved and the road's driving function is restored; otherwise, a second-level road maintenance instruction is generated to continue monitoring the road status and updating data. This automated and intelligent maintenance mechanism effectively improves the efficiency and accuracy of road management, ensures the timeliness and rationality of road repairs, and reduces the deviation of human judgment. Through systematic management, it can achieve more refined road maintenance while ensuring urban traffic safety, extend the service life of roads, optimize resource allocation, and improve the overall operation efficiency and reliability of urban transportation systems. This intelligent road supervision model not only promotes the digital and intelligent management of road monitoring in smart city construction, but also provides strong technical support for improving the travel experience of urban residents and ensuring traffic safety.

[0114] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A smart city supervision service system based on video surveillance, characterized by: It includes video monitoring module, data acquisition module, road damage analysis module, surface flatness analysis module, road disease assessment module and management module; The video monitoring module is based on multiple shooting devices deployed on various main roads in the city to shoot panoramic images of the main roads, and input the respectively acquired original video streams and point cloud data of surface three-dimensional coordinate points into the central monitoring platform for storage; The data acquisition module is used to extract key image frames from the original video stream in the central monitoring platform, construct a continuous image frame sequence, and extract the road surface damage size data and road surface damage area data after grayscale and denoising. The pavement damage analysis module is used to analyze the pavement damage size data and the pavement damage area data to determine whether the current trunk road is in a failure state. If not, a surface settlement analysis instruction is issued; The surface flatness analysis module is used to analyze the surface settlement of the trunk road that is not in a failure state after receiving the surface settlement analysis instruction, and obtain the surface flatness coefficient Xpz; The road damage assessment module is used to associate the surface roughness coefficient Xpz with the corresponding road damage size data and road damage area data, and obtain the comprehensive damage assessment index Zbh by fitting; The management module is used to compare and analyze the comprehensive disease assessment index Zbh with a preset disease assessment threshold P, determine whether the main roads that are not currently in a failed state meet the normal driving requirements of urban vehicles, generate corresponding road maintenance level instructions and execute them.

2. According to claim 1, a smart city supervision service system based on video surveillance is characterized in that: The video monitoring module includes a deployment unit and a monitoring unit; The deployment unit is used to deploy multiple high-resolution cameras and depth cameras on various main roads in the city, and connect the multiple high-resolution cameras and depth cameras to the central monitoring platform through wired connections, and the central monitoring platform is used to store data; The monitoring unit is used to use a high-resolution camera equipped with a multi-spectral sensor to take timed panoramic images of the main road by setting a shooting period, obtain the original video stream of the main road, and obtain the horizontal height difference Dj of each point on the road surface based on the structured light technology used by the depth camera and combined with the reference plane of the initial road surface state, and construct point cloud data of the three-dimensional coordinate points on the surface; the obtained original video stream and point cloud data of the main road are sent to the central monitoring platform for storage.

3. According to claim 2, a smart city supervision service system based on video surveillance is characterized in that: The data acquisition module is used to extract key image frames at the same time interval from the original video stream in the central monitoring platform, construct a continuous image frame sequence, extract feature points in the continuous image frames by using a scale-invariant feature transformation algorithm, align the matching feature points in two adjacent frames, and then use a homography matrix to continuously superimpose perspective transformation on the continuous image frames to generate a road surface superposition image; After the road surface overlay image is grayed out, it is denoised by a Gaussian filter to eliminate random noise in the road surface overlay image. Then, the Canny operator is used to detect the boundary features of crack edges, pothole edges, and outer contour edges in the road surface overlay image to generate a road surface gradient edge image. Based on the generated road gradient edge image, after feature extraction and classification, the road damage size data and road damage area data are obtained respectively. The road damage size data includes each crack width Lfk, each crack length Lfc and each pothole diameter Lkz. The road damage area data includes each crack area Sfm, each pothole area Skm and the total road surface area Sz.

4. The smart city supervision service system based on video surveillance according to claim 3 is characterized by: The pavement damage analysis module includes a damage size analysis unit, a damage area analysis unit and a determination unit; The damage size analysis unit is used to analyze the road surface damage size data to obtain the road surface damage size degree coefficient Xcd. The road surface damage size degree coefficient Xcd is obtained by the following formula: Where Lfk i Expressed as the width of the i-th crack, Lfc i It is represented by the length of the i-th crack, i = 1, 2, 3, ..., n, n represents the number of cracks, Lkz j It is represented by the diameter of the jth pothole, j = 1, 2, 3, ..., m, m is the number of potholes, α1, α2 and α3 are all weight values; According to the historical data in the central monitoring platform, the damage size threshold T is pre-set, and the road damage size coefficient Xcd is compared with the damage size threshold T to generate a relevant road damage size signal. The specific process is as follows: When the pavement damage size coefficient Xcd is greater than or equal to the damage size threshold T, a "pavement damage size severe signal" is generated; when the pavement damage size coefficient Xcd is less than the damage size threshold T, a "pavement damage size mild signal" is generated, and the generated "pavement damage size severe signal" and "pavement damage size mild signal" are sent to the damage degree assessment unit; The relevant road surface damage size degree signal includes a "road surface damage size severe signal" and a "road surface damage size mild signal".

5. A smart city supervision service system based on video surveillance according to claim 4, characterized in that: The damaged area analysis unit is used to analyze the road damaged area data to obtain the road damaged area ratio Vzb, and the road damaged area ratio Vzb is obtained by the following formula: Where Sfm i It is represented by the i-th crack area, i = 1, 2, 3, ..., n, n represents the number of cracks, Skm j It is represented as the area of ​​the jth pothole, j = 1, 2, 3, ..., m, m represents the number of potholes, and Sz represents the total road surface area; According to the numerical value of the road damage area ratio Vzb, a relevant road damage area degree signal is generated. The specific process is as follows: When the pavement damage area ratio Vzb is greater than or equal to 15%, a "pavement damage area severe signal" is generated; when the pavement damage area ratio Vzb is less than 15%, a "pavement damage area mild signal" is generated, and the generated "pavement damage area severe signal" and "pavement damage area mild signal" are sent to the damage degree assessment unit; The relevant road surface damage area degree signal includes a "road surface damage area severe signal" and a "road surface damage area mild signal".

6. A smart city supervision service system based on video surveillance according to claim 4, characterized in that: The determination unit is used to analyze the received relevant road surface damage size degree signal and relevant road surface damage area degree signal to determine whether to issue a surface settlement analysis instruction. The specific process is as follows: According to the relevant road surface damage size degree signals, a set No. 1 X is established, and the "light road surface damage size signal" is marked as element a1, and the "heavy road surface damage size signal" is marked as element a2, and element a1∈set No. 1 X, element a2∈set No. 1 X; According to the relevant road damage area degree signal, the second set Y is established, the "light road damage area signal" is marked as element b1, the "heavy road damage area signal" is marked as element b2, and the element b1∈the second set Y, the element b2∈the second set Y; Take the union of set 1 X and set 2 Y. If X∪Y={a1, b2} or {a2, b1} or {a2, b2}, it means that the current trunk road is in an invalid state. Inform the relevant municipal departments to completely resurface the current trunk road. If X∪Y={a1,b1}, it means that the current trunk road is not in a failed state, and a surface settlement analysis instruction is issued.

7. A smart city supervision service system based on video surveillance according to claim 2, characterized in that: The surface flatness analysis module is used to construct a road surface settlement distribution map based on the horizontal height difference Dj of each road surface point in the point cloud data of the three-dimensional coordinate points of the surface after receiving the surface settlement analysis instruction. The road surface settlement distribution map is used to display the settlement status of different areas of the road surface. The road surface settlement distribution map is evenly divided into a number of groups of grid unit areas, and the point cloud data of the three-dimensional coordinate points of the surface are mapped to each grid unit area to extract the settlement amount Ttx in each grid unit area, and the settlement amount mean is obtained according to the averaging algorithm. Based on the constructed road surface settlement distribution map, the surface settlement of the main roads that are not in a failure state is analyzed to obtain the surface flatness coefficient Xpz, which is obtained by the following formula: Where, Ttx s It is expressed as the amount of settlement in the sth grid unit area, s = 1, 2, 3, ..., a, a represents the number of grid unit areas, Expressed as the mean sedimentation amount.

8. The smart city supervision service system based on video surveillance according to claim 7 is characterized by: The road damage assessment module is used to associate the surface roughness coefficient Xpz of the trunk road that is not in a failed state with the corresponding road damage size data and road damage area data, and after dimensionless processing, fit to obtain the comprehensive disease assessment index Zbh of the current trunk road. The comprehensive disease assessment index Zbh is obtained by the following formula: Where Xcd is the pavement damage size coefficient, Vzb is the pavement damage area ratio, β1, β2 and β3 are the weight values ​​of the surface flatness coefficient Xpz, the pavement damage size coefficient Xcd and the pavement damage area ratio Vzb, respectively, and A is the correction constant.

9. A smart city supervision service system based on video surveillance according to claim 8, characterized in that: The management module includes a comparison unit and an execution unit; The comparison unit is used to pre-set the disease assessment threshold P according to the historical data in the central monitoring platform, and compare and analyze the comprehensive disease assessment index Zbh with the preset disease assessment threshold P to determine whether the trunk road that is not in a failure state currently meets the normal driving requirements of urban vehicles, and generate a corresponding road maintenance level instruction, the specific contents of which are as follows: If the comprehensive disease assessment index Zbh ≥ the disease assessment threshold P, it means that the trunk road is not currently in a failed state and does not meet the normal driving requirements of urban vehicles, and a first-level road maintenance instruction is generated; If the comprehensive disease assessment index Zbh is less than the disease assessment threshold P, it indicates that the trunk road is not currently in a failed state, meets the normal driving requirements of urban vehicles, and generates a secondary road maintenance instruction.

10. A smart city supervision service system based on video surveillance according to claim 9, characterized in that: The execution unit is used to receive the corresponding road maintenance level instruction generated by the comparison unit and execute the corresponding maintenance measures, the specific contents of which are as follows: When receiving a first-level road maintenance order, the execution content is as follows: immediately notify the relevant municipal departments to start the emergency repair procedure, arrange a professional repair team to go to the site for a comprehensive inspection, and confirm the diseased area, mill the cracks and potholes, re-lay asphalt, fill the uneven road surface with asphalt and compact it, and after the repair is completed, conduct a quality inspection of the repaired road section to confirm that the repair quality meets the requirements, and formulate a long-term maintenance plan; When receiving the secondary road maintenance instruction, the execution content is: continue to monitor and update the data of the main roads that are not currently in an invalid state and meet the normal driving requirements of urban vehicles, to ensure that the comprehensive disease assessment index Zbh is always less than the disease assessment threshold P.

Images