Intelligent detection method and system for flat side stone disease based on linear continuity constraint

Abstract

The application discloses a flat side stone disease intelligent detection method and system based on linear continuity constraint. The method comprises the following steps: acquiring a road video stream and vehicle dynamic parameters; extracting a flat side stone and a surrounding pavement area from the road video stream to obtain two-dimensional image information; converting the two-dimensional image information into a visual mimic geometry model representing a three-dimensional geometric contour with a spatial scale; constructing a three-dimensional reference model conforming to an actual road line based on the visual mimic geometry model; comparing the spatial differences between the visual mimic geometry model and the three-dimensional reference model, calculating and identifying potential diseases to obtain residual types, residual values and geometric parameters; classifying the diseases, evaluating the risk level, and generating maintenance suggestions. Through the implementation of the method of the application, intelligent identification, classification and degree evaluation of flat side stone diseases can be realized, the accuracy and automation level of flat side stone disease detection are improved, and the development of road maintenance towards a more intelligent and refined direction is promoted.

Description

TECHNICAL FIELD

[0001] The present application relates to computer vision, more particularly to a method and system for intelligent detection of flat curb diseases based on linear continuity constraints. BACKGROUND

[0002] With the acceleration of urbanization, urban road networks are expanding, and flat curbs, as a key component of roads, have functions not only limited to guiding and drainage, but also are crucial to ensuring driving safety and comfort. However, over time and with the increasing service life of flat curbs, diseases such as settlement, displacement, and damage gradually appear, which directly affect the safety and service life of roads.

[0003] Currently, the inspection of flat curbs mainly relies on manual inspection or methods based on two-dimensional vision technology. Although these methods meet the basic needs to some extent, they have significant limitations. Specifically, traditional computer vision methods are easily affected by environmental factors such as shadows, parked vehicles, etc., resulting in a high false detection rate and difficulty in accurately quantifying the extent and type of diseases. In addition, most existing technologies focus on two-dimensional image analysis, which cannot accurately distinguish between real diseases and environmental disturbances, and lack the ability to finely assess the extent of diseases, thus failing to fully meet the requirements of modern road maintenance for fine management. The current technical bottleneck is that most detection schemes rely on independent local point clouds or features for analysis, ignoring the modeling and constraints of the overall linear continuity of road facilities. This method is easily disturbed by local abnormal points, and cannot guarantee the overall continuity and stability of the model. At the same time, the lack of effective disease precise quantification and classification mechanism makes it difficult to objectively evaluate the extent and risk level of diseases.

[0004] Therefore, it is necessary to design a new method to realize intelligent identification, classification, and extent evaluation of flat curb diseases, improve the accuracy and automation level of flat curb disease detection, and promote the development of road maintenance towards a more intelligent and fine direction. SUMMARY

[0005] The present application aims to overcome the defects of the prior art and provide an intelligent detection method for flat curb diseases based on linear continuity constraints.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solution: an intelligent detection method for flat curb diseases based on linear continuity constraints, comprising:

[0007] acquiring a road video stream and vehicle dynamic parameters;

[0008] extracting the area of flat curbs and their surrounding road surfaces from the road video stream to obtain two-dimensional image information;

[0009] convert the two-dimensional image information into a visual mimic geometry model representing a three-dimensional geometric profile with spatial scale;

[0010] construct a three-dimensional reference model conforming to actual road alignment based on the visual mimic geometry model; wherein the three-dimensional reference model comprises a geometric curve describing a flat curb condition, the geometric curve comprising at least one of a main line reflecting horizontal continuity of the flat curb, an edge line describing a connection relationship of the flat curb, an auxiliary line evaluating verticality of the curb, and an outer edge line reflecting lateral displacement and inclination trend of the curb; and the three-dimensional reference model dynamically adjusts and optimizes a local abnormal area by real-time monitoring of residual error values between the visual mimic geometry model and the three-dimensional reference model;

[0011] compare spatial differences between the visual mimic geometry model and the three-dimensional reference model, calculate and identify potential diseases to obtain residual error types, residual error values, and geometric parameters;

[0012] classify diseases according to the residual error types, residual error values, and geometric parameters, evaluate risk levels, and generate maintenance recommendations.

[0013] Further technical solutions thereof are as follows: the three-dimensional reference model conforming to actual road alignment is constructed based on the visual mimic geometry model, comprising:

[0014] on the basis of the visual mimic geometry model, a sample section with complete structure and continuity is selected as a reference, and a fitting method with road alignment continuity constraint conditions is applied to establish a smooth and continuous initial three-dimensional reference model in three-dimensional space;

[0015] calculate spatial differences between the visual mimic geometry model and the initial three-dimensional reference model;

[0016] when the increase amplitude in the spatial differences meets the requirements, then enable local dynamic optimization to correct data on both sides of the area with the increase amplitude meeting the requirements, to obtain geometric data after the disease area;

[0017] reconstruct the initial three-dimensional reference model according to the geometric data after the disease area and the geometric data before the disease area in the visual mimic geometry model, to obtain the three-dimensional reference model.

[0018] Further technical solutions thereof are as follows: the local dynamic optimization is enabled to correct data on both sides of the area with the increase amplitude meeting the requirements, to obtain geometric data after the disease area, comprising:

[0019] a sliding window or segmented re-fitting strategy is adopted to update data on both sides of the area with the increase amplitude meeting the requirements, to obtain geometric data after the disease area.

[0020] A further technical solution is that the two-dimensional image information is obtained by extracting the curbstone and the surrounding pavement area from the road video stream.

[0021] The key frames in the road video stream are subjected to semantic segmentation to obtain a semantic segmentation result.

[0022] The semantic segmentation result is optimized based on the established road facility prior knowledge base, and the area of the curbstone and the surrounding pavement is extracted from the semantic segmentation result according to the spatial topological relationship between the curbstone and the lane line and the curb, to obtain the two-dimensional image information.

[0023] A further technical solution is that the road facility prior knowledge base includes information about road design specifications, typical sizes of curbstones, and spatial position relationships between curbstones and lane lines and curbs.

[0024] A further technical solution is that the two-dimensional image information is converted into a visual mimic geometry model representing a three-dimensional geometric contour with a spatial scale, including:

[0025] An edge detection technique is applied to the two-dimensional image information to identify the boundaries of the curbstone and the curbstone and to determine their geometric contours to obtain edges.

[0026] A contour tracking algorithm is used to continuously trace the edges to form closed geometric shapes.

[0027] A monocular depth estimation algorithm is used to analyze the curbstone and curbstone area to predict the depth information of each point.

[0028] The geometric shapes are combined with the corresponding depth information, and three-dimensional coordinates are added to each contour point to obtain a point set with coordinates.

[0029] A point cloud model is constructed based on the point set with coordinates to display the three-dimensional shape of the curbstone and the curbstone.

[0030] The point cloud model in the image coordinate system is mapped to the world coordinate system through mathematical transformation to obtain a visual mimic geometry model.

[0031] A further technical solution is that the spatial difference between the visual mimic geometry model and the three-dimensional reference model is compared to calculate and identify potential diseases to obtain residual types and geometric parameters, including:

[0032] According to the scene and the demand, a sampling mode is selected and the number of sampling points on each curbstone or curbstone is calculated.

[0033] The shortest Euclidean distance of each sampling point to the four curves is calculated to quantify the spatial difference to obtain geometric parameters.

[0034] detect potential disease points beyond the error range using a dynamic threshold based on the geometric parameters, wherein the dynamic threshold is adaptively adjusted according to the spatial resolution of input data, modeling accuracy and road environmental conditions;

[0035] analyze the minimum residual error between specified curves based on the geometric parameters, quantify the disease type and degree of curb and flat stone, to obtain residual error type and residual error value.

[0036] Further technical solutions thereof are as follows: the disease is classified and the risk level is evaluated according to the residual error type, residual error value and geometric parameters, and maintenance suggestions are generated, including:

[0037] analyze the negative distribution of residual error value on the Z-axis according to the residual error type and the geometric parameters to identify settlement-type diseases; detect the residual error in the X / Y-axis direction according to the residual error type and the geometric parameters to determine displacement-type diseases; compare the actual sampling points with the theoretical sampling points and evaluate the fluctuation of residual error value to determine damage-type diseases, and use image recognition to exclude false positives caused by legal facilities; determine missing-type diseases from obvious data gaps or reconstruction blanks in the continuous model prediction area;

[0038] comprehensively consider the residual error type, residual error value, geometric parameters and specific location to generate a risk evaluation level and propose corresponding maintenance priority suggestions.

[0039] Further technical solutions thereof are as follows: after the disease is classified and the risk level is evaluated according to the residual error type, residual error value and geometric parameters, and maintenance suggestions are generated, it further includes:

[0040] generate a detailed report and output the detailed report, wherein the detailed report includes disease images, time stamps, vehicle GPS coordinates in vehicle dynamic parameters, residual error type and geometric parameters, classification information and risk level.

[0041] The application also provides a flat curb disease intelligent detection system based on linear continuity constraints, including:

[0042] An acquisition unit is configured to acquire a road video stream and vehicle dynamic parameters.

[0043] A semantic processing unit is configured to extract the area of flat curb and its surrounding pavement from the road video stream to obtain two-dimensional image information.

[0044] A conversion unit is configured to convert the two-dimensional image information into a visual morphing geometric model representing a three-dimensional geometric contour with spatial proportions.

[0045] A reference model generating unit is configured to construct a three-dimensional reference model conforming to actual road alignment based on the vision mimic geometry model; wherein the three-dimensional reference model comprises a geometric curve describing a flat curb condition, and the geometric curve comprises at least one of a main line reflecting the horizontal continuity of the flat curb, an edge line describing the connection relationship of the flat curb, an auxiliary line evaluating the verticality of the curb, and an outer edge line reflecting the lateral displacement and inclination trend of the curb; and the three-dimensional reference model dynamically adjusts and optimizes the local abnormal area by monitoring the residual value between the vision mimic geometry model and the three-dimensional reference model in real time.

[0046] A residual analysis unit is configured to compare the spatial difference between the vision mimic geometry model and the three-dimensional reference model, calculate and identify potential diseases to obtain a residual type and a geometric parameter.

[0047] A classification unit is configured to classify the diseases according to the residual type and the geometric parameter, evaluate the risk level, and generate a maintenance suggestion.

[0048] Compared with the prior art, the present application has the beneficial effects that: the present application extracts a two-dimensional image of the flat curb area by analyzing the road video stream, and converts it into a three-dimensional geometry model. The model contains key geometric curves and can reflect the state of the flat curb. By comparing the differences between the real-time data and the established three-dimensional reference model, potential diseases are automatically identified and classified, and their severity and risk level are evaluated, and finally maintenance suggestions are generated. This method improves the accuracy and automation level of flat curb disease detection, and promotes the intelligent and fine development of road maintenance.

[0049] The application will be further described below in conjunction with the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0051] Figure 1 The flowchart of the flat curb disease intelligent detection method based on linear continuity constraint provided by the embodiment of the present application;

[0052] Figure 2 The schematic diagram of the road video stream provided by the embodiment of the present application;

[0053] Figure 3 The flat curb schematic diagram of the road video stream provided by the embodiment of the present application;

[0054] Figure 4 The schematic diagram of the road video stream provided by the embodiment of the present application;Figure 3 An enlarged schematic view of the square frame of FIG. 1 1 ;

[0055] Figure 5 A schematic view of a visual mimicry geometric model and a geometric curve provided for an embodiment of the present application;

[0056] Figure 6 A corresponding coordinate projection schematic view of a sink-type disease provided for an embodiment of the present application;

[0057] Figure 7 A corresponding coordinate projection schematic view of a displacement-type disease provided for an embodiment of the present application;

[0058] Figure 8 A corresponding coordinate projection schematic view of a breakage-type disease provided for an embodiment of the present application;

[0059] Figure 9 A corresponding coordinate projection schematic view of a missing-type disease provided for an embodiment of the present application;

[0060] Figure 10 A schematic block diagram of a flat side stone disease intelligent detection system based on linear continuity constraint provided for an embodiment of the present application;

[0061] Figure 11 A schematic block diagram of a computer device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0063] It should be understood that, when used in the specification and the appended claims, the terms “comprise” and “include” indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0064] It should also be understood that the terms used herein in the specification and the appended claims are only for the purpose of describing particular embodiments and do not intend to limit the present application. As used in the specification and the appended claims of the present application, unless otherwise clearly indicated by the context, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well.

[0065] It should be further understood that the term "and / or" used in the description and claims of the present application refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0066] Please refer to Figure 1 , Figure 1 The flowchart of the intelligent detection method for flat side stone disease based on linear continuity constraint provided by the embodiments of the present application. The intelligent detection method for flat side stone disease based on linear continuity constraint is applied to a server. The server interacts with a camera and a sensor, extracts and converts two-dimensional image information into a three-dimensional visual mimic geometry model with spatial scale, and constructs a three-dimensional reference model conforming to the actual road alignment by acquiring road video stream and vehicle dynamic parameters. The state of flat side stone is described by geometric curve, and the residual value between the model and the dynamic adjustment model is monitored and adjusted in real time to optimize the local abnormal area. By comparing the spatial differences between the visual mimic geometry model and the three-dimensional reference model, the potential disease is calculated and identified, the risk level is classified and evaluated, and maintenance suggestions are generated. The whole process adopts automation technology, and the whole process from data acquisition, processing to analysis is intelligent, which effectively improves the accuracy and efficiency of flat side stone disease detection, and promotes the development of road maintenance to a more intelligent and refined direction. In addition, by generating a report containing detailed disease information, targeted maintenance decisions are further supported.

[0067] Figure 1 The flowchart of the intelligent detection method for flat side stone disease based on linear continuity constraint provided by the embodiments of the present application. As shown in Figure 1 , the method comprises the following steps S110 to S160.

[0068] S110, acquiring road video stream and vehicle dynamic parameters.

[0069] In this embodiment, as shown in Figure 2 , the road video stream refers to the continuous video data of the road in front of the vehicle collected in real time by the high-definition camera installed on the vehicle (such as the vehicle-mounted camera behind the front windshield). These video data not only contain the visual information of flat side stone and its surrounding environment, but also include road conditions, traffic signs and other key elements. The quality and stability of the video image are crucial for subsequent semantic recognition, region extraction and three-dimensional reconstruction. In order to ensure that the acquired video data is clear and stable, factors such as camera resolution, frame rate, anti-shake performance need to be considered, and in actual application, video preprocessing operations such as noise reduction, contrast enhancement may also be needed to improve analysis accuracy.

[0070] Vehicle dynamic parameters refer to relevant physical quantities collected during vehicle driving, mainly including but not limited to vehicle speed, acceleration, GPS coordinates and other information. Specifically:

[0071] Speed: Reflects the current speed at which the vehicle is traveling, which is important for calculating relative motion and predicting changes in object positions.

[0072] Acceleration: Provides information about the vehicle's acceleration or deceleration, which helps understand changes in the vehicle's state of motion, especially in cases of sudden braking or rapid acceleration, which can affect the stability of the video data and the accuracy of subsequent analysis.

[0073] GPS coordinates: Accurately records the vehicle's specific location on the Earth's surface, providing a geospatial reference for subsequent data correlation. This is crucial for accurately locating detected disease locations on a map and developing targeted maintenance plans.

[0074] These dynamic parameters not only help better understand and interpret the scenes in the video, but also to some extent compensate for image distortion or displacement problems caused by vehicle movement, thereby improving the accuracy and reliability of the entire system for flat curb disease detection. Combining video images with vehicle dynamic parameters can achieve more accurate and smooth three-dimensional reconstruction, further optimizing the reference model construction and optimization steps based on linear continuity constraints.

[0075] S120, extracting the area of the flat curb and its surrounding pavement from the road video stream to obtain two-dimensional image information.

[0076] In this embodiment, two-dimensional image information refers to image data containing specific objects (such as flat curbs and their surrounding pavements) obtained after processing from the road video stream. These information is the basis for subsequent three-dimensional reconstruction, disease detection and analysis. Specifically, two-dimensional image information includes but is not limited to:

[0077] The optimized accurate area of the flat curb and its surrounding pavement is based on the prior knowledge base of road facilities to optimize the semantic segmentation results, ensuring accurate identification and extraction of the specific area of the flat curb and its surrounding pavement.

[0078] In an embodiment, the above step S120 can include steps S121-S122.

[0079] S121, performing semantic segmentation on key frames in the road video stream to obtain semantic segmentation results.

[0080] In this embodiment, the semantic segmentation results in the key frames are obtained by classifying each pixel in the image into different categories (such as flat curbs, lane lines, curbs, etc.) through semantic segmentation technology, thereby obtaining the position and shape of each object in the image.

[0081] In this step, first, it is necessary to determine which frames are key frames for further analysis. Usually, those frames that can provide the most informative or representative information are selected as key frames. Next, advanced computer vision algorithms (such as convolutional neural networks, CNN) are used to perform semantic segmentation on the selected key frames. The result of semantic segmentation is a labeled map, in which each pixel is assigned to a predefined class label (such as curb, lane line, background, etc.). This process aims to distinguish different types of elements in the image and lay the foundation for the next step of processing.

[0082] S122, based on the established road facility prior knowledge base, the semantic segmentation result is optimized, and the curb and its surrounding pavement area are extracted from the semantic segmentation result according to the spatial topological relationship between the curb, lane line and curb, to obtain two-dimensional image information.

[0083] In this embodiment, the road facility prior knowledge base includes information about road design specifications, typical sizes of curbs, and spatial position relationships between curbs, lane lines and curbs.

[0084] In this step, please refer to Figure 3 and Figure 4 , the results of semantic segmentation are further optimized using a pre-constructed road facility prior knowledge base. This knowledge base contains information about road design specifications, typical sizes of curbs, and their spatial position relationships relative to lane lines and curbs. Based on these prior knowledge, the location of curbs and their surrounding areas can be more accurately located. The specific method is as follows:

[0085] Apply spatial topological rules: according to the spatial relationship between the curb, lane line and curb, filter and correct the initial semantic segmentation result to ensure that the identified object conforms to the actual road structure.

[0086] Refine boundaries and details: for the initially identified curb area, edge detection and other image processing techniques are used to refine its boundaries to improve extraction accuracy.

[0087] Generate the final two-dimensional image information: combine all the processing steps above to output a two-dimensional image that accurately labels the curb and its surrounding pavement area, for subsequent three-dimensional reconstruction and disease analysis.

[0088] This method not only improves the accuracy of curb recognition, but also effectively reduces the amount of invalid calculation, providing high-quality data support for subsequent depth estimation and three-dimensional modeling. At the same time, this strategy also enhances the robustness of the system, making it stable in various complex environments.

[0089] S130, convert the two-dimensional image information into a visual mimic geometry model representing the three-dimensional geometric profile with spatial scale.

[0090] In this embodiment, the visual mimic geometry model is a mathematical representation of the three-dimensional geometry of an object (such as a flat stone) extracted and reconstructed from a two-dimensional image using computer vision techniques. This model not only contains the spatial coordinate information (X, Y, Z) of the target object, but also reflects its physical size and morphological characteristics in the real world. It usually exists in the form of point cloud data, each point contains its precise position in the world coordinate system, and can be used for subsequent analysis and processing.

[0091] In an embodiment, the above step S130 can include steps S131-S136.

[0092] S131, apply edge detection techniques to the two-dimensional image information to identify the boundaries of flat stones and side stones, and clearly define their geometric contours to obtain edges.

[0093] In this embodiment, the edge refers to the boundary between different regions in the image, specifically the location where the pixel value changes significantly.

[0094] For flat side stones, the edge refers to the clear dividing line between flat stones, side stones and the surrounding environment. These edge information is the basis for constructing geometric contours.

[0095] S132, use contour tracking algorithm to continuously trace along the edges to form a closed geometric shape.

[0096] In this embodiment, the closed geometric shape refers to connecting adjacent edge points based on edge detection results through contour tracking algorithm, and finally forming a closed figure, as shown in Figure 5 , accurately depicting the specific contour of flat side stones. This closed geometric shape helps to accurately define the shape and size of flat side stones, providing a basis for further three-dimensional modeling.

[0097] S133, use monocular depth estimation algorithm to analyze the flat side stone area and predict the depth information of each point.

[0098] In this embodiment, the depth information refers to the distance information in the real world corresponding to each pixel point in the two-dimensional image, i.e. the distance from the point to the camera. Through monocular depth estimation technology, each point of flat side stones and its surrounding area can be assigned a depth value, realizing the spatial conversion from two-dimensional to three-dimensional.

[0099] S134, combine the geometric shape with the corresponding depth information, add three-dimensional coordinates to each contour point to obtain a set of points with coordinates.

[0100] In this embodiment, the set of points with coordinates refers to the result of combining the geometric shape (composed of contour points) with depth information. Each contour point now has corresponding X, Y, Z coordinates, forming a point set that describes the three-dimensional structure of the flat side stone. This point set is a key step in building a three-dimensional model.

[0101] S135, based on the set of points with coordinates, a point cloud model showing the three-dimensional form of the flat stone and side stone is constructed.

[0102] In this embodiment, the point cloud model refers to a three-dimensional model composed of a large number of discrete points, each point representing a specific location on the surface of the flat side stone, with clear three-dimensional coordinates. The point cloud model can intuitively show the three-dimensional form of the flat side stone, facilitating subsequent analysis, measurement and disease detection.

[0103] S136, mapping the point cloud model in the image coordinate system to the world coordinate system through mathematical transformation to obtain a visual mimicry geometric model.

[0104] This step involves converting the point cloud model in the image coordinate system to the world coordinate system, ensuring that the position and direction of all points accurately reflect their state in the actual environment. This usually involves a series of coordinate transformation operations, including but not limited to translation, rotation and scaling, etc., the purpose is to make the model consistent with the scale and orientation of the real world, so as to obtain an accurate visual mimicry geometric model.

[0105] Through the above steps, the system can extract and convert high-quality three-dimensional geometric models from the original two-dimensional video stream, providing reliable data support for subsequent road facility health monitoring and maintenance.

[0106] Please refer to Figures 3 to 5 , first through edge detection technology to identify and extract the boundary of flat stone and side stone, and then use contour tracking algorithm to continuously outline along these boundaries to form closed geometric shapes. In this process, not only the two-dimensional contour of the target object is determined, but also the foundation for subsequent three-dimensional reconstruction is laid. Then, using monocular depth estimation algorithm to analyze the identified flat side stone area, predicting the depth information of each point, thus giving each contour point a specific distance value. Combined with these depth information and two-dimensional geometric shape, the system adds accurate three-dimensional coordinates (X, Y, Z) to each contour point, constructing a dense and spatially proportional point cloud model.

[0107] Finally, to ensure that the generated three-dimensional model accurately reflects the position and shape of the curb stone in the real world, the system performs a coordinate transformation operation to map the point cloud model in the image coordinate system to the world coordinate system. This step is crucial to ensure the authenticity and accuracy of the model, as it allows each point in the model to correspond to a location in the actual physical space, facilitating subsequent precise measurement and disease detection. In this way, the final visual mimicry geometric model not only clearly displays the three-dimensional structure of the curb stone, but also supports more detailed spatial analysis and maintenance decisions.

[0108] S140, based on the visual mimicry geometric model, a three-dimensional reference model conforming to the actual road alignment is constructed; wherein the three-dimensional reference model includes a geometric curve describing the state of the curb stone, the geometric curve includes at least one of a main line reflecting the horizontal continuity of the curb stone, an edge line describing the connection relationship of the curb stone, an auxiliary line evaluating the verticality of the curb stone, and an outer edge line reflecting the lateral displacement and inclination trend of the curb stone; and the three-dimensional reference model dynamically adjusts and optimizes the local abnormal area by monitoring the residual value between the visual mimicry geometric model and the three-dimensional reference model in real time.

[0109] This step aims to use the visual mimicry geometric model (i.e., convert from two-dimensional image information to three-dimensional spatial data) to construct a three-dimensional reference model that accurately reflects the actual road alignment. This model includes geometric curves that describe the state of the curb stone, such as main lines, edge lines, auxiliary lines, and outer edge lines, to accurately capture the geometric shape of the curb stone and its changes. In addition, through real-time monitoring and dynamic adjustment mechanisms, this method can optimize local abnormal areas and improve the accuracy of disease detection.

[0110] In an embodiment, the above-mentioned step S140 can include steps S141-S144.

[0111] S141, on the basis of the visual mimicry geometric model, a sample section with complete structure and continuity is selected as a reference, and a fitting method with road alignment continuity constraints is applied to establish a smooth and continuous initial three-dimensional reference model in three-dimensional space.

[0112] In this embodiment, the initial three-dimensional reference model refers to a smooth and continuous ideal model constructed in three-dimensional space by selecting a sample section with complete structure and continuity as a reference and applying a fitting method with road alignment continuity constraints (such as B-spline curve) based on the visual mimicry geometric model. This model represents the geometric shape that the curb stone should have in the absence of disease, including but not limited to:

[0113] Main line: the main line of the curb stone top along the road direction, reflecting the horizontal continuity of the curb stone along the road direction.

[0114] Edge line: The edge line at the junction of the flat stone and the side stone, describing the connection between the flat stone and the side stone.

[0115] Auxiliary line: The auxiliary line above the junction line, used to evaluate the verticality of the side stone.

[0116] Outer edge line: The outer edge or right side line of the side stone, reflecting the lateral displacement and inclination trend of the side stone.

[0117] This model is mainly used for comparison and optimization in subsequent steps, as a standard or "scale" to judge the actual road conditions.

[0118] Based on the visual mimicry geometric model, a sample section with complete structure and continuity is selected as a reference, and a fitting method with road line continuity constraint condition (such as B-spline curve) is applied to establish a smooth and continuous initial three-dimensional reference model in three-dimensional space. The core of this step is to select a representative section as a reference to ensure that the selected sample section has good structural integrity and continuity. Based on these data points, a specific algorithm (such as B-spline curve) is used for fitting to generate an ideal three-dimensional model without disease.

[0119] S142, calculate the spatial difference between the visual mimicry geometric model and the initial three-dimensional reference model.

[0120] In this embodiment, the spatial difference refers to the deviation value between the visual mimicry geometric model and the initial three-dimensional reference model at each corresponding point. Specifically, it can be quantified and understood in the following way:

[0121] Calculation method: Usually use mathematical tools such as Euclidean distance to measure the difference between two models at the same coordinate point. For example, for each measured point P(x, y, z), the system will calculate the shortest Euclidean distance from it to the four theoretical curves (main line, edge line, auxiliary line, outer edge line):

[0122] ;

[0123] ;

[0124] ;

[0125] ;

[0126] The spatial difference helps to identify potential problem areas, i.e. those with significant deviations from the ideal state. These deviations may indicate the presence of diseases such as settlement, displacement, damage or loss, etc. Based on these differences, the system can further analyze and take appropriate repair measures.

[0127] In summary, the initial three-dimensional reference model provides an ideal reference framework, while the spatial discrepancies are used to measure the deviation between the actual situation and this ideal state, thereby supporting precise disease detection and repair work.

[0128] S143、When the increase amplitude meets the requirements in the spatial discrepancy, then enable local dynamic optimization to modify the data on both sides of the area where the increase amplitude meets the requirements, to obtain the geometric data after the disease area.

[0129] In this embodiment, the geometric data after the disease area refers to the geometric parameters obtained by detailed measurement and recording of the disease area after identifying and locating the road curbstone disease area. These data are crucial for assessing the extent, type of disease, and developing repair solutions.

[0130] Specifically, the geometric data after the disease area includes but is not limited to the following aspects:

[0131] Coordinate information: records the specific location coordinates (x, y, z) of the disease point or disease area, to accurately determine its geographical position.

[0132] Dimensional measurement: length: the extension length of the disease along the road direction.

[0133] Width: the maximum expansion width of the disease transversely.

[0134] Depth: the vertical depth of disease types such as cracks and potholes.

[0135] Shape description: description of the morphological characteristics of the disease area, such as whether it is a linear crack, a circular pothole, or an irregular settlement.

[0136] Relative displacement: comparison of the relative displacement between the disease area and the surrounding healthy part, including horizontal displacement and vertical displacement.

[0137] Inclination angle: if the curbstone or other structure is inclined, the angle of deviation from the vertical line needs to be measured.

[0138] Surface roughness: assess the flatness of the disease area surface, especially important for potholes, damage, etc.

[0139] Other characteristics: may contain more details according to specific circumstances, such as material loss, corrosion degree, etc.

[0140] Through the collection and analysis of these geometric data, the specific situation of the disease can be more accurately understood, and appropriate repair strategies can be designed to restore the functionality and safety of road facilities. In addition, these data can also be used for subsequent monitoring, to evaluate the repair effect and prevent future problems.

[0141] Specifically, a sliding window or piecewise re-fitting strategy is employed to update the data on both sides of the area with acceptable increase amplitude, resulting in more accurate geometric data after the disease area.

[0142] Specifically, a sliding window strategy is used to update the data on both sides of the disease area. First, a suitable size window is defined, which slides from one end to the other on the data sequence. When the window slides to include the suspected disease area and its nearby healthy area, the algorithm identifies the disease area (i.e., the area with acceptable increase amplitude). For this marked area, the system will use all data points within the window to re-fit, generating a more accurate geometric model. This step usually involves using mathematical methods such as least squares to reduce errors, ensuring that the new model accurately reflects the actual situation. Then, by comparing the differences between the original data and the newly fitted model, the exact location, range and extent of the disease can be determined, and the damaged parts can be adjusted or repaired accordingly.

[0143] The piecewise re-fitting strategy is another effective means, especially suitable for larger or more complex disease areas. In this strategy, the entire data set is divided into several small segments, each of which is analyzed independently. Once a specific segment with abnormal increase amplitude is identified as a disease area, a detailed re-fitting process is performed on this segment and the adjacent healthy area. The advantage of this is that it can handle local changes more carefully and improve overall accuracy. In specific implementation, polynomial regression or other curve fitting techniques may be applied to the selected small segment first, and then gradually expanded outward until the entire affected area and a wide enough safety boundary are covered. Finally, by merging the new fitting results of each small segment, the complete disease area geometric data is formed, providing a scientific basis for subsequent repair work. This method not only effectively improves data accuracy, but also flexibly adapts to different sizes and types of disease conditions.

[0144] When there is a significant increase in spatial differences, a local dynamic optimization mechanism is activated to correct the data in the affected area. Specifically, a sliding window or piecewise re-fitting strategy is employed to update the data on both ends of the abnormal interval, resulting in more accurate geometric data after the disease area. Once a region with significant residual error is found, the system will automatically perform local optimization on that region, including but not limited to sliding window analysis and piecewise re-fitting, thereby reducing errors and improving model accuracy.

[0145] S144, reconstructing the initial three-dimensional reference model according to the geometric data after the disease area and the geometric data before the disease area in the visual mimicry geometric model, to obtain a three-dimensional reference model.

[0146] According to the newly acquired post-disease area geometric data and the pre-disease area geometric data in the original visual mimicry geometric model, the initial three-dimensional reference model is reconstructed, and finally the optimized three-dimensional reference model is obtained. This step involves integrating the pre and post data to form a new and more accurate three-dimensional reference model. It not only takes into account the actual situation of the disease area, but also retains the good data of the unaffected area, ensuring the continuity and consistency of the overall model.

[0147] Through the above steps, the method of the embodiment effectively utilizes the visual mimicry geometric model to construct a high-precision, dynamically adjustable three-dimensional reference model, as shown by the dashed line in FIG. Figure 5 This method not only accurately locates and analyzes the state of the flat curb, but also timely discovers and corrects diseases through real-time monitoring and local optimization, greatly improving the efficiency and accuracy of road maintenance work. In addition, this technology has important significance for improving the management level of urban roads and ensuring traffic safety.

[0148] S150, comparing the spatial differences between the visual mimicry geometric model and the three-dimensional reference model, calculating and identifying potential diseases to obtain residual types, residual values and geometric parameters.

[0149] In this embodiment, the residual type refers to the type of difference identified when comparing the visual mimicry geometric model and the three-dimensional reference model. Specifically, these differences can be divided into four main types: subsidence, displacement, damage and loss:

[0150] Subsidence: manifested as negative residual in the Z-axis direction, indicating that the flat stone or curb has sunk.

[0151] Displacement: mainly reflected in the residual in the X / Y-axis direction, indicating that the flat stone or curb has shifted horizontally or vertically.

[0152] Damage: the abnormal area presents irregular distribution, and the spatial geometric continuity is destroyed, which is usually determined by the significant decrease in the number of sampling points or the significant fluctuation of residual.

[0153] Loss: there is an obvious data gap or reconstruction blank in the continuous model prediction area, which means that the flat curb has been lost or severely damaged.

[0154] Residual value refers to the shortest Euclidean distance between each sampling point and its corresponding theoretical curve (such as a, b, c, d). This distance reflects the size of the deviation between the actual detected geometric shape and the expected ideal shape. For example, for the main line of the flat stone top along the road direction (curve a), the calculation method is as follows:

[0155] where P(x, y, z) is the coordinate of the sampling point, Ca(t) represents the position of a certain point on curve a, and the parameter t is solved by numerical optimization method to ensure the calculation accuracy.

[0156] The geometric parameters include various metrics for describing the geometric characteristics of the curb, including but not limited to length, width, height, and the residual value mentioned above. They are the key basis for quantifying the disease degree. For example, when evaluating the verticality of the curb, the relationship between the auxiliary line above the junction line (curve c) and the outer edge of the curb (curve d) needs to be considered, that is, to determine whether the curb has a tilt or verticality problem by minimizing .

[0157] In an embodiment, the step S150 described above can include steps S151-S154.

[0158] S151, according to the scene and the demand, select the sampling mode and calculate the number of sampling points on each flat stone or curb.

[0159] In this embodiment, according to different application scenarios, appropriate sampling modes (fine mode, default mode or fast mode) are selected, and the number of sampling points to be set on each flat stone or curb is calculated accordingly. For example, in the scientific research verification scene, the fine mode may be used, and the sampling resolution is set to 1 cm.

[0160] Specifically, according to different scenes and requirements, the fineness, i.e. the sampling resolution (r), is dynamically adjusted.

[0161] The length (l) of the flat stone and the curb obtained by the visual mimicry geometric model can obtain the number of sampling points of each flat stone and curb, such as the reference number of sampling points of a flat stone on curve a: .

[0162] According to the detection accuracy requirement, the system provides three sampling modes as shown in Table 1.

[0163] Table 1. Three sampling modes

[0164]

[0165] S152, calculate the shortest Euclidean distance of each sampling point to the four curves to quantify the spatial difference to obtain the geometric parameters.

[0166] In this embodiment, for the selected sampling points, their shortest Euclidean distances to the four theoretical curves (a, b, c, d) are calculated one by one to quantify the spatial difference and obtain the key geometric parameters.

[0167] For each measured point P(x, y, z), the system calculates its shortest Euclidean distance to the four theoretical curves:

[0168] ;

[0169] ;

[0170] ;

[0171] ;

[0172] where parameter t is solved by numerical optimization methods such as Newton iteration method to ensure the calculation accuracy and efficiency.

[0173] S153, detecting potential disease points beyond the error range based on the geometric parameters using a dynamic threshold, wherein the dynamic threshold is adaptively adjusted according to the spatial resolution of the input data, the modeling accuracy, and the road environment conditions.

[0174] Using a dynamically adjusted threshold (e.g., 3 cm), potential disease points beyond the error range are automatically detected based on the spatial resolution of the input data, the modeling accuracy, and the road environment conditions.

[0175] Specifically, a dynamic threshold τ (e.g., 3 cm) is set, and when the residual of a certain spatial sampling point or spatial region exceeds this threshold, it is identified as a potential disease. The threshold can be adaptively adjusted according to the spatial resolution of the input data, the modeling accuracy, and the road environment conditions, ensuring that the identification results have consistent judgment sensitivity under different sampling densities and detection distances.

[0176] S154, analyzing the minimum residual between specified curves based on the geometric parameters to quantify the disease type and degree of curbstone and flat stone, to obtain residual type and residual value.

[0177] Analyzing the minimum residual between specified curves to accurately quantify the disease type and degree of curbstone and flat stone. For example, by comparing and to determine whether the curbstone has settled and determine the specific settlement depth.

[0178] Specifically, curbstone disease quantification (based on curve b-c-d analysis):

[0179] Displacement identification: , displacement amount = | |, which is the residual in the Y-axis direction;

[0180] Settlement identification: Settlement depth = | |, which is the residual in the Z-axis direction;

[0181] Damage identification: or , Anomalies were found in the samples, and the surface of the side stones showed irregular spatial distribution or some sampling points were missing.

[0182] Missing data identification: By analyzing the continuity of spatial distribution, obvious spatial data gaps are identified, along with the number of missing sampling points q (e.g., ...). ).

[0183] Quantification of diseases in flat stone (based on curve ab analysis):

[0184] Settlement identification: manifested as residuals in the Z-axis direction, settlement = | |;

[0185] Damage identification: or Anomalies exist, and the surface of the flat stone exhibits an irregular spatial distribution, or some sampling points are missing;

[0186] Missing data identification: The determination method is the same as that for side stones, that is, by analyzing the continuity of spatial distribution, there are obvious spatial data gaps, and the number of missing sampling points q (e.g., ).

[0187] Through the above steps, the system can accurately identify various types of road defects and their severity, providing a scientific basis for subsequent road maintenance.

[0188] S160. Classify the diseases according to the residual type, residual value and geometric parameters, assess the risk level and generate maintenance recommendations.

[0189] In one embodiment, step S160 described above may include steps S161 to S162.

[0190] S161. Analyze the negative distribution of residual values ​​on the Z-axis according to the residual type and the geometric parameters to identify settlement-type defects; detect residuals in the X / Y axis directions according to the residual type and the geometric parameters to determine displacement-type defects; compare the actual number of sampling points with the theoretical number of sampling points and evaluate the fluctuation of residual values ​​to determine damage-type defects; use image recognition to eliminate false alarms caused by legitimate facilities; determine missing defects by obvious data gaps or reconstruction blanks in the continuous model prediction area.

[0191] S162. Taking into account the residual type, residual value, geometric parameters, and specific location, generate a risk assessment level and propose corresponding maintenance priority recommendations.

[0192] In this embodiment, the automatic classification of diseases is based on residual type (distribution pattern on the X / Y / Z axes) and the aforementioned refined and quantified geometric parameters, such as... Figure 5As shown in the figure and implemented in combination with a preset classification rule library. The settlement-type diseases are mainly manifested as negative residuals in the Z-axis direction, that is, for the curbstone, | |≥ and for the leveling stone, | |≥ , as shown in Figure 6 . The displacement-type diseases are reflected in the residuals in the X / Y-axis direction, mainly determined by | |≥ for the curbstone, as shown in Figure 7 . For the damage-type diseases, their characteristic is that the abnormal areas are irregularly and discretely distributed, and the spatial geometric continuity is damaged. It can be specifically determined by the following method, as shown in Figure 8 : First, calculate the actual number of spatial sampling points . If <n (the reference number of sampling points), it can be judged that the spatial sampling points are missing, which means the interruption of spatial continuity, and thus the damage conclusion can be drawn. Second, calculate the residual fluctuation. If there is an abnormality in or of the leveling stone (that is, [[ID=2 ...]]​​​​​​​​​​​​​​​​​​​​​​​​​​A detailed report is generated and outputted, which includes disease images, timestamps, vehicle GPS coordinates from vehicle dynamic parameters, residual type and geometric parameters, classification information, and risk level.

[0196] After all necessary data processing and risk assessment are completed, these information are integrated into a comprehensive report. This detailed report contains multiple key elements, ensuring a comprehensive understanding of the disease and providing strong support for subsequent processing:

[0197] Disease images: Show the specific location and state of the disease, clearly present the damage of road facilities through high-resolution images, provide intuitive reference for subsequent analysis and repair work.

[0198] Timestamps: Record the exact time of data collection, which is important for tracking the development of the disease, assessing its urgency, and planning maintenance activities.

[0199] Vehicle GPS coordinates from vehicle dynamic parameters: Accurately mark the location of the detection device (usually installed on the vehicle) during data collection, which helps to determine the specific location of the disease and assist the application of geographic information system (GIS).

[0200] Residual type and geometric parameters: Include the distribution pattern and quantitative values on X / Y / Z axes, such as various values (e.g. , , etc.), which are the basis for determining the type of disease.

[0201] Classification information: According to the residual type and geometric parameters, combined with the preset rule library, automatically classify the disease as settlement type, displacement type, damage type or missing type, etc., to help quickly locate the problem nature.

[0202] Risk level: Based on all the above information, the system comprehensively assesses the risk level of each disease, divided into low, medium and high levels, and gives corresponding maintenance priority suggestions.

[0203] After the report is generated, through efficient data management and network modules, the complete report and all related data are safely uploaded to cloud storage. This not only ensures the safety and accessibility of data, but also facilitates subsequent data analysis, historical comparison and cross-department collaboration. In addition, cloud storage makes remote monitoring and management possible, greatly improving work efficiency and service quality. Finally, this systematic data processing and report generation process provides a solid technical support for ensuring the safe operation of road facilities.

[0204] The method of the embodiment aims to overcome the shortcomings of existing flat curb detection techniques in terms of accuracy and automation. Current methods rely on independent, local point clouds or features for analysis, lacking modeling and constraints on the overall linearity continuity of road facilities, which makes them susceptible to local abnormal points and unable to maintain the overall continuity and stability of the model. In addition, these methods also lack precise quantification and classification mechanisms for diseases, making it difficult to objectively assess the degree of disease and risk level.

[0205] The goal of the embodiment is to construct a flat curb reference model in three-dimensional space that conforms to the actual road linearity, is continuous and stable, and to achieve intelligent detection and classification evaluation of flat curb diseases through residual quantification analysis.

[0206] By utilizing the pre-established road facility database and spatial topology information, the flat curb area is intelligently identified and extracted, improving the extraction efficiency and accuracy. By automatically distinguishing and contouring the flat stone and curb areas, based on the continuous contour results and the spatial constraints of the road, a visual mimic geometry model with true geometric characteristics is generated (a curved surface generated by lofting the extracted contour lines), thereby realizing three-dimensional geometric reconstruction and spatial analysis. By introducing the continuity constraint of road linearity, it ensures that the constructed three-dimensional reference model can effectively suppress the influence of local outliers, improving the overall stability and geometric fitting accuracy of the model. Residual calculation is used to identify diseases and quantify geometric parameters such as deformation and damage volume, achieving automatic identification and evaluation of diseases. According to the residual distribution characteristics, the disease types are automatically classified, and the risk level of the disease is evaluated in combination with the pre-set rule base. Complete report generation and data upload functions are provided to ensure safe storage and remote access of all data.

[0207] Compared with the previous two-dimensional detection methods based on image recognition, the method of the embodiment has the following advantages:

[0208] Identifying and fitting flat stones and curbs separately avoids the problem of category confusion, greatly improving the accuracy of boundary positioning and identification. By introducing a linearity continuity constraint model, the false detection phenomenon caused by local abnormalities is effectively suppressed, improving the overall stability and geometric fitting accuracy of the model. Even in complex environments (such as shadows, changes in lighting, vehicle obstruction, etc.), it still maintains stable recognition ability for flat curb shapes. The residual calculation method is used to achieve centimeter-level geometric deviation quantification, supporting precise evaluation of multiple disease types such as displacement, damage, and settlement. According to the residual distribution characteristics, the disease types are automatically classified, and the corresponding risk level evaluation is generated. It supports unmanned inspection operations and batch road facility health state evaluation work, and can be easily extended to detection applications of other linear structure facilities (such as guardrails, isolation belts, etc.).

[0209] By acquiring road video streams and vehicle dynamic parameters, and extracting the curb and its surrounding pavement area to obtain two-dimensional image information, and then converting these information into a three-dimensional geometric profile model with spatial scale, a three-dimensional reference model conforming to the actual road alignment is constructed based on this visual mimicry geometric model, which contains key geometric curves describing the state of the curb, such as the main line, edge line, auxiliary line and outer edge line, etc., for accurately reflecting the horizontal and vertical characteristics of the curb; by monitoring the residual value between the visual mimicry geometric model and the three-dimensional reference model in real time, the local abnormal area can be dynamically adjusted and optimized, so as to compare the spatial differences between the two models, calculate and identify potential diseases, and obtain specific residual types, residual values and geometric parameters; according to the above data, the diseases are classified and their risk levels are evaluated, and maintenance suggestions are generated, realizing intelligent identification, classification and degree evaluation of curb diseases, significantly improving the accuracy and automation level of detection, and promoting the development of road maintenance towards intelligence and refinement. This process covers the entire process from data acquisition to disease analysis to maintenance suggestions, making the management of road facilities more efficient and scientific.

[0210] The above-mentioned intelligent detection method for curb diseases based on linear continuity constraint extracts two-dimensional images of the curb area by analyzing road video streams and converts them into a three-dimensional geometric model. This model contains key geometric curves that can reflect the state of the curb. By comparing the differences between the real-time data and the established three-dimensional reference model, potential diseases are automatically identified and classified, and their severity and risk level are evaluated, and finally maintenance suggestions are generated. This method improves the accuracy and automation level of curb disease detection, promoting the development of road maintenance towards intelligence and refinement.

[0211] Figure 10 is a schematic block diagram of an intelligent detection system for curb diseases based on linear continuity constraint 300 provided by an embodiment of the present application. As shown in Figure 10 According to the above-mentioned intelligent detection method for curb diseases based on linear continuity constraint, the present application also provides an intelligent detection system for curb diseases based on linear continuity constraint 300. The intelligent detection system for curb diseases based on linear continuity constraint 300 includes units for executing the above-mentioned intelligent detection method for curb diseases based on linear continuity constraint, and the system can be configured in a server. Specifically, please refer to Figure 10 The intelligent detection system for curb diseases based on linear continuity constraint 300 includes an acquisition unit 301, a semantic processing unit 302, a conversion unit 303, a reference model generation unit 304, a residual analysis unit 305 and a classification unit 306.

[0212] The acquisition unit 301 is configured to acquire a road video stream and vehicle dynamic parameters; the semantic processing unit 302 is configured to extract a curb and a region of a surrounding road surface from the road video stream to obtain two-dimensional image information; the conversion unit 303 is configured to convert the two-dimensional image information into a visual mimic geometry model representing a three-dimensional geometric profile with a spatial scale; the reference model generation unit 304 is configured to construct a three-dimensional reference model conforming to an actual road alignment based on the visual mimic geometry model; wherein the three-dimensional reference model comprises a geometric curve describing a curb state, and the geometric curve comprises at least one of a main line reflecting horizontal continuity of the curb, an edge line describing a connection relationship of the curb, an auxiliary line evaluating verticality of the curb, and an outer edge line reflecting lateral displacement and tilting trend of the curb; and the three-dimensional reference model dynamically adjusts and optimizes a local abnormal region by monitoring a residual value between the visual mimic geometry model and the three-dimensional reference model in real time; the residual analysis unit 305 is configured to compare spatial differences between the visual mimic geometry model and the three-dimensional reference model, calculate and identify potential diseases to obtain a residual type and a geometric parameter; the classification unit 306 is configured to classify the diseases according to the residual type and the geometric parameter, evaluate a risk level, and generate a maintenance suggestion.

[0213] In an embodiment, the reference model generation unit 304 comprises:

[0214] The selection subunit is configured to select a sample paragraph with complete structure and continuity as a reference based on the visual mimic geometry model, apply a fitting method with a road alignment continuity constraint to establish a smooth and continuous initial three-dimensional reference model in a three-dimensional space; the calculation subunit is configured to calculate spatial differences between the visual mimic geometry model and the initial three-dimensional reference model; the correction subunit is configured to, when an increase amplitude in the spatial differences meets a requirement, enable local dynamic optimization to correct data on both sides of a region with the increase amplitude meeting the requirement to obtain geometric data after a disease area; and the reconstruction subunit is configured to reconstruct the initial three-dimensional reference model according to the geometric data after the disease area and geometric data before the disease area in the visual mimic geometry model to obtain the three-dimensional reference model.

[0215] In an embodiment, the correction subunit is configured to update the data on both sides of the region with the increase amplitude meeting the requirement by using a sliding window or a segmented re-fitting strategy to obtain the geometric data after the disease area.

[0216] In an embodiment, the semantic processing unit 302 comprises:

[0217] The semantic segmentation subunit is configured to perform semantic segmentation on the key frame in the road video stream to obtain a semantic segmentation result; and the extraction subunit is configured to extract curbstone and the surrounding pavement area from the semantic segmentation result based on an established road facility prior knowledge base according to the spatial topological relationship that the curbstone is located between the lane line and the curb, to obtain two-dimensional image information.

[0218] In an embodiment, the conversion unit 303 comprises:

[0219] The edge determination subunit is configured to apply an edge detection technique to the two-dimensional image information to identify the boundary of the curbstone and the curbstone and to determine the geometric contour thereof, to obtain an edge; the outlining subunit is configured to use a contour tracking algorithm to continuously outline along the edge to form a closed geometric shape; the depth prediction subunit is configured to analyze the curbstone and curbstone area by using a monocular depth estimation algorithm to predict the depth information of each point; the coordinate addition subunit is configured to combine the geometric shape with the corresponding depth information and add three-dimensional coordinates to each contour point, to obtain a point set with coordinates; the point cloud construction subunit is configured to construct a point cloud model showing the three-dimensional shape of the curbstone and the curbstone based on the point set with coordinates; and the mapping subunit is configured to map the point cloud model in the image coordinate system to the world coordinate system by mathematical transformation, to obtain a visual mimicry geometric model.

[0220] In an embodiment, the residual analysis unit 305 comprises:

[0221] The number calculation subunit is configured to select a sampling mode according to the scene and the requirement and to calculate the number of sampling points on each curbstone or curbstone; the difference quantization subunit is configured to calculate the shortest Euclidean distance of each sampling point to the four curves to quantify the spatial difference, to obtain geometric parameters; the detection subunit is configured to detect potential disease points that exceed the error range based on the geometric parameters using a dynamic threshold, wherein the dynamic threshold is adaptively adjusted according to the spatial resolution of the input data, the modeling accuracy and the road environment condition; and the residual analysis subunit is configured to analyze the minimum residual between the specified curves based on the geometric parameters to quantify the disease type and degree of the curbstone and the curbstone, to obtain residual types and residual values.

[0222] In an embodiment, the classification unit 306 comprises:

[0223] The dividing sub-unit is configured to analyze the negative distribution of residual values on the Z axis according to the residual type and the geometric parameter to identify a subsidence type disease; detect the residual in the X / Y axis direction according to the residual type and the geometric parameter to determine a displacement type disease; compare the actual sampling points with the theoretical sampling points and evaluate the fluctuation of residual values to determine a damage type disease, and use image recognition to exclude false positives caused by legal facilities; determine a missing type disease by a continuous model to predict obvious data gaps or reconstruct blanks in the region; and the grade generating sub-unit is configured to comprehensively consider the residual type, residual value, geometric parameter and specific position to generate a risk assessment grade and propose a corresponding maintenance priority suggestion.

[0224] In an embodiment, the system described above further comprises:

[0225] The report generating unit is configured to generate a detailed report and output the detailed report, wherein the detailed report includes disease images, time stamps, vehicle GPS coordinates in vehicle dynamic parameters, residual types and geometric parameters, classification information and risk grades.

[0226] It should be noted that the specific implementation process of the above-mentioned linear continuity constraint-based flat side stone disease intelligent detection system 300 and each unit can be clearly understood by those skilled in the art, which can be referred to the corresponding description in the foregoing method embodiments. For the convenience and brevity of description, it will not be repeated here.

[0227] The linear continuity constraint-based flat side stone disease intelligent detection system 300 described above can be implemented in the form of a computer program, which can run on a computer device as shown in the computer device. Figure 11 .

[0228] Please refer to Figure 11 , Figure 11 is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 can be a server, wherein the server can be a stand-alone server or a server cluster composed of multiple servers.

[0229] Referring to Figure 11 , the computer device 500 includes a processor 502, a memory and a network interface 505 connected through a system bus 501, wherein the memory can include a non-volatile storage medium 503 and an internal memory 504.

[0230] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which when executed, can cause the processor 502 to perform a linear continuity constraint-based flat side stone disease intelligent detection method.

[0231] The processor 502 is configured to provide computing and control capabilities to support the operation of the entire computer device 500.

[0232] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503, and the computer program 5032, when executed by the processor 502, can cause the processor 502 to perform the linear continuity constraint-based flat side stone disease intelligent detection method.

[0233] The network interface 505 is configured to perform network communication with other devices. Those skilled in the art can understand that the network interface 505 can be implemented by a network card, a network adapter, or the like. Figure 11 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device 500 to which the scheme of the present application is applied. The specific computer device 500 can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0234] The processor 502 is configured to run the computer program 5032 stored in the memory to implement all steps of the linear continuity constraint-based flat side stone disease intelligent detection method.

[0235] It should be understood that in the embodiments of the present application, the processor 502 can be a central processing unit (CPU), and the processor 502 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), ready programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0236] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments of the method can be completed by a computer program instructing related hardware. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the above-mentioned embodiments of the method.

[0237] Therefore, the application further provides a storage medium. The storage medium can be a computer readable storage medium. The storage medium stores a computer program, wherein the computer program is executed by a processor to enable the processor to perform all steps of the intelligent flat stone disease detection method based on linear continuity constraint.

[0238] The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various computer readable storage media that can store program codes.

[0239] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0240] In several embodiments provided by the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are only illustrative. For example, the division of each unit is only a logical functional division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0241] The steps in the method embodiments of the present application can be adjusted, combined and reduced in sequence according to actual needs. The units in the system embodiments of the present application can be combined, divided and reduced according to actual needs. In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0242] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application essentially or say the parts that make contributions to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for enabling a computer device (which can be a personal computer, a terminal or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application.

[0243] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements shall be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An intelligent detection method for flat-sided rock defects based on linear continuity constraints, characterized in that, include: Acquire road video streams and vehicle dynamic parameters; Extract the area of ​​the curb stones and the surrounding road surface from the road video stream to obtain two-dimensional image information; The two-dimensional image information is converted into a visual mimicry geometric model representing a three-dimensional geometric contour with spatial proportions. A three-dimensional benchmark model conforming to the actual road alignment is constructed based on the visual mimicry geometric model. The three-dimensional benchmark model includes geometric curves describing the state of the curb stones. These geometric curves include at least one of the following: a main line reflecting the horizontal continuity of the curb stones; an edge line describing the connection relationship between the curb stones; an auxiliary line evaluating the verticality of the curb stones; and an outer edge line reflecting the lateral displacement and tilting trend of the curb stones. Furthermore, the three-dimensional benchmark model dynamically adjusts and optimizes local anomaly areas by real-time monitoring of the residual values ​​between the visual mimicry geometric model and the three-dimensional benchmark model. By comparing the spatial differences between the visual mimicry geometric model and the three-dimensional reference model, potential defects are calculated and identified to obtain residual type, residual value and geometric parameters; Based on the residual type, residual value, and geometric parameters, the disease is classified, the risk level is assessed, and maintenance recommendations are generated.

2. The intelligent detection method for flat curbstone defects based on linear continuity constraints according to claim 1, characterized in that, The construction of a three-dimensional benchmark model that conforms to the actual road alignment based on the visual mimicry geometric model includes: Based on the aforementioned visual mimicry geometric model, a sample segment with complete structure and continuity is selected as a reference, and a fitting method with road alignment continuity constraints is applied to establish a smooth and continuous initial three-dimensional benchmark model in three-dimensional space. Calculate the spatial difference between the visual mimicry geometric model and the initial three-dimensional reference model; When the increase in the spatial difference meets the requirements, local dynamic optimization is used to correct the data on both sides of the area where the increase meets the requirements, so as to obtain the geometric data after the diseased area. The initial three-dimensional reference model is reconstructed based on the geometric data behind the diseased area and the geometric data in front of the diseased area in the visual mimicry geometric model to obtain the three-dimensional reference model.

3. The intelligent detection method for flat-sided rock defects based on linear continuity constraints according to claim 2, characterized in that, The step of enabling local dynamic optimization correction to increase the data on both sides of the area that meets the requirements, in order to obtain the geometric data of the diseased area, includes: The sliding window or segmented refit strategy is used to update the data on both sides of the area where the increase in magnitude meets the requirements, so as to obtain the geometric data after the diseased area.

4. The intelligent detection method for flat-sided rock defects based on linear continuity constraints according to claim 1, characterized in that, The step of extracting the area of ​​the curbstone and its surrounding road surface from the road video stream to obtain two-dimensional image information includes: Semantic segmentation is performed on keyframes in the road video stream to obtain semantic segmentation results; Based on the established prior knowledge base of road facilities, the semantic segmentation results are optimized. According to the spatial topological relationship between the curb and the lane line and the curb, the area of ​​the curb and its surrounding road surface is extracted from the semantic segmentation results to obtain two-dimensional image information.

5. The intelligent detection method for flat curbstone defects based on linear continuity constraints according to claim 4, characterized in that, The prior knowledge base for road facilities includes information on road design specifications, typical dimensions of curb stones, and the spatial relationship between curb stones, lane lines, and curbs.

6. The intelligent detection method for flat-sided rock defects based on linear continuity constraints according to claim 1, characterized in that, The process of converting the two-dimensional image information into a visually approximate geometric model representing a three-dimensional geometric contour with spatial proportions includes: Edge detection technology is applied to the two-dimensional image information to identify the boundaries of flat stones and side stones, clarify their geometric contours, and obtain the edges; A contour tracing algorithm is used to continuously trace along the edges to form a closed geometric shape; The monocular depth estimation algorithm is used to analyze the flat-sided rock area and predict the depth information of each point; The geometry is combined with the corresponding depth information to add three-dimensional coordinates to each contour point, so as to obtain a set of points with coordinates. A point cloud model displaying the three-dimensional morphology of the flat stones and side stones is constructed based on the coordinate point set. The point cloud model in the image coordinate system is mapped to the world coordinate system through mathematical transformation to obtain a visual mimicry geometric model.

7. The intelligent detection method for flat-sided rock defects based on linear continuity constraints according to claim 1, characterized in that, The spatial differences between the visual mimicry geometric model and the three-dimensional baseline model are compared to calculate and identify potential defects, thereby obtaining residual types and geometric parameters, including: Select the sampling mode according to the scenario and requirements, and calculate the number of sampling points on each flat stone or side stone. For each sampling point, calculate its shortest Euclidean distance to the four curves to quantify spatial differences and obtain geometric parameters; Based on the geometric parameters, a dynamic threshold is used to detect potential defects that exceed the error range, wherein the dynamic threshold is adaptively adjusted according to the spatial resolution of the input data, the modeling accuracy, and the road environment conditions. Based on the geometric parameters, the minimum residual between the specified curves is analyzed to quantify the type and degree of damage to the curb stones and flat stones, so as to obtain the residual type and residual value.

8. The intelligent detection method for flat-sided rock defects based on linear continuity constraints according to claim 1, characterized in that, The process of classifying diseases based on residual type, residual value, and geometric parameters, assessing risk level, and generating maintenance recommendations includes: The negative distribution of residual values ​​on the Z-axis is analyzed based on the residual type and geometric parameters to identify settlement-type defects; residuals in the X / Y axis directions are detected based on the residual type and geometric parameters to determine displacement-type defects; the number of actual sampling points is compared with the theoretical number of sampling points, and the fluctuation of residual values ​​is evaluated to determine damage-type defects; false alarms caused by legitimate facilities are eliminated using image recognition; and missing defects are determined by obvious data gaps or reconstruction blanks in the continuous model prediction area. Taking into account the residual type, residual value, geometric parameters, and specific location, a risk assessment level is generated, and corresponding maintenance priority recommendations are proposed.

9. The intelligent detection method for flat-sided rock defects based on linear continuity constraints according to claim 1, characterized in that, After classifying diseases based on the residual type, residual value, and geometric parameters, assessing the risk level, and generating maintenance recommendations, the process further includes: Generate and output a detailed report, which includes disease images, timestamps, vehicle GPS coordinates in vehicle dynamic parameters, residual type and geometric parameters, classification information and risk level.

10. An intelligent detection system for flat-sided rock defects based on linear continuity constraints, characterized in that, include: The acquisition unit is used to acquire road video streams and vehicle dynamic parameters. A semantic processing unit is used to extract the area of ​​the curbstone and its surrounding road surface from the road video stream to obtain two-dimensional image information; The conversion unit is used to convert the two-dimensional image information into a visual mimicry geometric model representing a three-dimensional geometric contour with spatial proportions. A benchmark model generation unit is used to construct a three-dimensional benchmark model that conforms to the actual road alignment based on the visual mimicry geometric model. The three-dimensional benchmark model includes geometric curves describing the state of the curb stones. These geometric curves include at least one of the following: a main line reflecting the horizontal continuity of the curb stones; an edge line describing the connection relationship between the curb stones; an auxiliary line evaluating the verticality of the curb stones; and an outer edge line reflecting the lateral displacement and tilting trend of the curb stones. Furthermore, the three-dimensional benchmark model dynamically adjusts and optimizes local anomaly areas by real-time monitoring of the residual values ​​between the visual mimicry geometric model and the three-dimensional benchmark model. The residual analysis unit is used to compare the spatial differences between the visual mimicry geometric model and the three-dimensional reference model, calculate and identify potential defects, and obtain the residual type and geometric parameters. The classification unit is used to classify diseases according to the residual type and the geometric parameters, assess the risk level, and generate maintenance recommendations.

Images