Mobile terminal-based pavement disease intelligent collection and classification identification method

Abstract

The application discloses a mobile terminal-based intelligent collection and classification identification method for road surface diseases, and particularly relates to the technical field of intelligent identification of road surface diseases. Through joint driving of a camera device and a positioning module, the method realizes synchronous collection of continuous image frames and track data, constructs a dynamic pairing relationship between the image frames and track points in combination with a sliding time window and a minimum time difference principle, introduces a timestamp misalignment alignment abnormality coefficient, simultaneously constructs a geographic mapping error accumulation coefficient, fuses the two types of coefficients to construct a disease space-time superposition risk assessment model, and outputs a disease space-time superposition risk assessment index, thereby realizing spatial redundancy risk measurement of the identified diseases, dynamically completing intelligent marking of superposition risk points, and accurately identifying repeated disease points through geographic distance determination and image feature vector similarity calculation based on a redundancy point fusion mechanism based on spatial consistency and image feature similarity, and dynamically updating the track positions of the repeated disease points by using a weighted fusion strategy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent road disease recognition, and more particularly to a mobile terminal-based intelligent road disease collection and classification recognition method. BACKGROUND

[0002] With the increasing demand for urban road operation and digital management, mobile terminal-based road disease collection and recognition technology has gradually become an important means of infrastructure intelligent detection. This method usually uses image collection devices and positioning modules mounted on vehicles or pedestrian terminals to obtain road images and location information, and combines image recognition models to realize intelligent detection and classification of diseases such as cracks, potholes, and subsidence. However, in actual operation, the system faces a core spatio-temporal coupling failure problem, i.e., alignment abnormalities of image frames and trajectory data and accumulation of geographic mapping errors.

[0003] In continuous collection, due to problems such as sensor delay, network jitter, and inconsistent sampling frequency of mobile terminals during travel, image frame timestamps and trajectory timestamps are easily misaligned, causing disease recognition results to be unable to accurately match their true geographic locations. Meanwhile, in tunnels, urban canyons, or signal-shielded environments, GPS data is prone to jumping, drifting, or discontinuity, causing positioning errors to continuously accumulate on the path. This spatio-temporal mismatch further leads to the problem of disease spatial overlay: the same disease may be labeled as disease points at multiple spatial locations by the system, forming redundant data and false hot zones, which seriously affects the accuracy of disease distribution statistics, map visualization, and subsequent maintenance task scheduling.

[0004] Therefore, under the condition of limited mobile terminal resources and insufficient data synchronization accuracy, how to effectively coordinate high-precision synchronization of image frames and trajectories and build an anti-drift and anti-jump geographic mapping mechanism has become a key technical bottleneck that needs to be broken through in intelligent road disease recognition systems. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide a mobile terminal-based intelligent road disease collection and classification recognition method to solve the problems raised in the background art.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0007] The mobile terminal-based intelligent road disease collection and classification recognition method comprises the following steps:

[0008] Step S1, start the camera device and positioning module in the mobile terminal, synchronously collect continuous image frames and location information, and record the collection timestamps of each image frame and each trajectory point data;

[0009] Step S2, pairing the image frames with the trajectory points based on the timestamp matching of the sliding time window with the minimum time difference principle, and obtaining timestamp mismatch alignment information to calculate a timestamp mismatch alignment abnormality coefficient;

[0010] Step S3, grouping the continuous trajectory points into path segments, and obtaining an incremental residual of the difference between the front and rear positions to calculate a geographic mapping error accumulation coefficient;

[0011] Step S4, constructing a disease spatiotemporal superposition risk assessment model according to the timestamp mismatch alignment abnormality coefficient and the geographic mapping error accumulation coefficient, outputting a disease spatiotemporal superposition risk assessment index, assessing the degree of spatiotemporal superposition risk existing in the current road disease identification, and marking the identified disease points as superposition risk points;

[0012] Step S5, fusing redundant points based on the spatial consistency of the front and rear frames and the image feature similarity for the marked superposition risk points.

[0013] In a preferred embodiment, the image frames are paired with the trajectory points based on the timestamp matching of the sliding time window with the minimum time difference principle, specifically as follows:

[0014] The collection timestamp of the current image frame is defined as The search sliding time window is: , wherein is the window span of the search sliding time window; the trajectory point set is extracted within the search sliding time window: , wherein is the jth trajectory point, is the collection timestamp of the jth trajectory point; for all , the time mismatch difference is calculated : , the trajectory point is searched that makes : ; the matching pair and the time mismatch difference are recorded;

[0015] The time mismatch difference sequence is obtained: , wherein is the nth recorded time mismatch difference, and N is the total number of time mismatch difference records.

[0016] In a preferred embodiment, the acquisition logic of the timestamp mismatch alignment abnormality coefficient is as follows: the time mismatch difference sequence is calculated: The mean value of the time mismatch difference in the sequence is: , wherein is the nth recorded time mismatch difference, and N is the total number of time mismatch difference records; the standard deviation of the time mismatch difference is calculated: ; calculate misalignment abnormal factor: ;

[0017] at different scales construct time difference local smooth baseline : , where is the time misalignment difference of the kth record, M is the total number of scales, and a multiscale residual is constructed : ; then the residual stacking structure is: :

[0018] Calculate the residual fluctuation energy of each point residual stack: , where is the residual energy, :

[0019] Take the residual fluctuation energy sequence as a signal, and use the empirical mode decomposition method to separate its dominant fluctuation trend and local abnormal term to obtain the basic trend energy and the local enhancement term :

[0020] Calculate the abnormal prominence factor: , where is the abnormal prominence factor;

[0021] Calculate the timestamp misalignment alignment abnormal coefficient: , where is the timestamp misalignment alignment abnormal coefficient, respectively represent the preset proportion coefficient of the misalignment alignment abnormal factor and the abnormal prominence factor, and are both greater than 0.

[0022] In a preferred embodiment, in step S3, consecutive trajectory points in the trajectory point set: are grouped into path line segments: , j = {1, 2,..., J-1}, J is the total number of trajectory points, is the jth path line segment; calculate the theoretical increment of the front and rear positions: , where is the theoretical increment, is the geographic coordinate of the trajectory point , and is the geographic coordinate of the trajectory point ; obtain the corrected position corresponding to the real mapping position of each trajectory point from the image frame positioning, and calculate the actual increment: , where is the actual increment, is the trajectory point the modified coordinate of the trajectory point, the modified coordinate of the trajectory point , and obtain the incremental residual error of the front-back position difference: , wherein is the incremental residual error; and obtain the incremental residual error sequence:

[0023] calculate the average value of the incremental residual error in the incremental residual error sequence: , wherein is the average value of the incremental residual error;

[0024] define a time weight factor: , wherein is the time weight factor, is the collection timestamp of the jth trajectory point, is the collection timestamp of the jth trajectory point;

[0025] calculate the exponentially weighted residual error cumulative mean: , wherein is the exponentially weighted residual error cumulative mean;

[0026] calculate the geographic mapping error cumulative coefficient: , wherein is the geographic mapping error cumulative coefficient, respectively represent preset proportion coefficients of the average value of the incremental residual error and the exponentially weighted residual error cumulative mean, and are both greater than 0.

[0027] In a preferred embodiment, the disease spatiotemporal superposition risk assessment model is constructed according to the timestamp misalignment abnormality coefficient and the geographic mapping error cumulative coefficient, and a disease spatiotemporal superposition risk assessment index is output. The formula on which the disease spatiotemporal superposition risk assessment model is based is as follows In the formula, is the disease spatiotemporal superposition risk assessment index, is the timestamp misalignment abnormality coefficient, is the geographic mapping error cumulative coefficient, respectively represent preset proportion coefficients of the timestamp misalignment abnormality coefficient and the geographic mapping error cumulative coefficient, and are both greater than 0.

[0028] In a preferred embodiment, the disease spatiotemporal superposition risk assessment index is compared with a preset disease spatiotemporal superposition risk assessment index threshold value, and the identified disease points are marked as superposition risk points, as follows:

[0029] If the disease spatiotemporal superposition risk assessment index is greater than the disease spatiotemporal superposition risk assessment index threshold value, the identified disease points in the search sliding time window are marked as superposition risk points. ​

[0030] If the disease spatiotemporal superposition risk assessment index is less than or equal to the disease spatiotemporal superposition risk assessment index threshold value, no superposition risk point marking is needed.

[0031] Technical effects and advantages of the present application:

[0032] 1、The present application realizes the synchronous acquisition of continuous image frames and trajectory data by the joint driving of the camera equipment and the positioning module, and accurately records the timestamp information, laying a foundation for subsequent high-precision alignment. Combined with the sliding time window and the minimum time difference principle, a dynamic pairing relationship between image frames and trajectory points is constructed, and a timestamp misalignment alignment abnormality coefficient is introduced to quantify the severity of the time sequence drift in the terminal acquisition process, effectively depicting the influence of terminal sensor delay and synchronization error on recognition accuracy. At the same time, in the trajectory data processing process, the geographical mapping error accumulation coefficient is constructed through the spatial increment analysis of continuous trajectory points, reflecting the spatial accumulation effect of positioning errors caused by GPS jumping, signal shielding and other factors on the path, providing spatial resolution support for disease location reliability assessment. The disease spatiotemporal superposition risk assessment model is constructed by fusing the above two types of coefficients, and the disease spatiotemporal superposition risk assessment index is output, realizing the spatial redundancy risk measurement of identified diseases, and dynamically completing the intelligent marking of superposition risk points. Further, in order to suppress the redundant recognition caused by spatial superposition, based on the redundancy point fusion mechanism of spatial consistency and image feature similarity, through geographical distance judgment and image feature vector similarity calculation, repeated disease points are accurately identified, and the trajectory position is dynamically updated by using the weighted fusion strategy, thereby effectively eliminating the repeated labeling phenomenon caused by the superposition of positioning drift and recognition error. In summary, the present application realizes high-precision alignment and redundancy suppression of multi-source spatiotemporal data in a resource-limited environment, not only improves the positioning accuracy and data reliability of disease identification, but also avoids the misleading influence of disease superposition on subsequent statistical analysis and maintenance scheduling, while ensuring the calculation efficiency, significantly improves the intelligent level and engineering application value of the system. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to facilitate the understanding of those skilled in the art, the present application will be further described below in conjunction with the drawings;

[0034] Figure 1 The flowchart of the method of the embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0036] Embodiment: Figure 1 The present application provides a mobile terminal-based pavement disease intelligent collection and classification recognition method, comprising the following steps:

[0037] Step S1, start the camera device and positioning module in the mobile terminal, synchronously collect continuous image frames and position information, and record the collection time stamp of each image frame and each track point data;

[0038] Step S2, based on the time stamp matching of the sliding time window, pair the image frames and track points according to the minimum time difference principle, and obtain the time stamp misalignment information of the image frames and track data to calculate the time stamp misalignment alignment abnormality coefficient;

[0039] Step S3, group the continuous track points into path segments, and obtain the incremental residual of the difference between the front and rear positions to calculate the geographic mapping error accumulation coefficient;

[0040] Step S4, construct a disease space-time superposition risk assessment model according to the time stamp misalignment alignment abnormality coefficient and the geographic mapping error accumulation coefficient, output a disease space-time superposition risk assessment index, assess the degree of space-time superposition risk existing in the current pavement disease recognition, and mark the superposition risk points of the recognized disease points;

[0041] Step S5, for the marked superposition risk points, fuse redundant points based on the spatial consistency of the front and rear frames and the image feature similarity;

[0042] In step S1, the camera device and the positioning module in the mobile terminal are started, continuous image frames and position information are synchronously collected, and the collection time stamp of each image frame and each track point data is recorded, specifically as follows:

[0043] The road disease collection camera device and the positioning module in the mobile terminal are started, a local configuration file or a remote configuration parameter is loaded, a multi-thread collection task controller is initialized, including an image collection thread and a positioning data listening thread, to ensure that the two can run in parallel and support asynchronous data reading and writing of the buffer, a driver interface of the camera is called, and image collection parameters including resolution, sampling frequency, exposure control parameters, etc. are set; a system time stamp (based on a high-precision clock of the system, such as SystemClock.elapsedRealtimeNanos() of Android or CMTime of iOS) of each acquired image frame is recorded immediately; the image frame and its time stamp are packaged and cached to a queue; the positioning module (such as a GPS / Beidou / GLONASS module or an IMU+visual inertial fusion (VIO) positioning module) is started; position update parameters including position update frequency, etc. are set; a trajectory point (containing longitude, latitude, speed, heading, etc. information) is received immediately, and the collection time stamp of the point is recorded; the trajectory point and its time stamp are packaged and cached to a queue;

[0044] In step S2, the image frame and the trajectory point are paired based on the time stamp matching of the sliding time window with the minimum time difference principle, as follows:

[0045] The collection time stamp of the current image frame is defined as , and the search sliding time window is: , wherein is the window span of the search sliding time window; the trajectory point set is extracted in the search sliding time window: , wherein is the jth trajectory point, is the collection time stamp of the jth trajectory point; for all , the time misalignment difference is calculated: , the trajectory point that makes the time misalignment difference minimum is searched: ; the matching pair and the time misalignment difference are recorded;

[0046] The time misalignment difference sequence is obtained: , wherein is the nth recorded time misalignment difference, and N is the total number of time misalignment difference records;

[0047] The time stamp misalignment information of the image frame and the trajectory data is obtained, and the time stamp misalignment alignment abnormality coefficient is calculated;

[0048] The timestamp misalignment abnormality coefficient in the application is a key index for measuring the time synchronization accuracy and alignment accuracy between image frames and positioning track data in the process of collecting road diseases by a mobile terminal. The coefficient essentially reflects the degree of timestamp matching deviation caused by inconsistent sensor sampling frequencies, system clock offset, data cache delay and other factors in the actual operation of the image acquisition module and the positioning module. The larger the timestamp misalignment abnormality coefficient, the farther the time interval between the image frame and the corresponding track point, and there is a significant timestamp alignment abnormality, which may cause the disease content reflected by the image frame to mismatch its real collection location in space, resulting in identification position drift or even serious mislabeling. The smaller the timestamp misalignment abnormality coefficient, the smaller the time difference between the image frame and the track point, and the more accurate and reliable the corresponding relationship. It can be considered that the synchronization performance of the current collection system is good, and the coupling between the image and the track is stable and reliable. In actual road inspection tasks, the mobile terminal is often in a continuous operation and high-frequency collection state, and the system is easily disturbed by various vibrations and disturbances during operation. For example, when the terminal travels in a road section with uneven signal coverage or hardware response lag, the collection time of image frames and track data often changes unevenly, such as track delay uploading, image cache blocking, etc., causing the data items that should have been one-to-one corresponding to be misaligned in time. Once this misalignment is not found or corrected, it will directly lead to the error of disease labeling and positioning based on geographic coordinates. For example, a crack disease that actually occurs in the center of the road may be mislabeled to a non-disease area due to the late matching of the image frame to the next location point, or the same disease may be recorded repeatedly at multiple coordinates, causing the so-called "disease spatial shadowing" problem. By constructing the timestamp misalignment abnormality coefficient, the system can quantitatively evaluate the matching quality of each image frame and the corresponding track, and then infer the potential risks of the current overall identification system in the time synchronization level. In the application, the abnormality coefficient is used to construct a disease space shadowing risk assessment model. In the model, the timestamp misalignment abnormality coefficient and the geographic mapping error accumulation coefficient work together to comprehensively evaluate whether there is a potential shadowing risk in a certain segment of data. If the timestamp alignment abnormality coefficient of a certain image frame is high and is accompanied by poor track continuity (such as GPS jumping), it is very likely that there will be redundant disease labeling caused by time and space mismatch. In summary, the timestamp misalignment abnormality coefficient, as a precision monitoring index for mobile terminal disease collection system design, plays an important role in solving time and space coupling mismatch, improving disease labeling accuracy, and reducing data redundancy risk. Its introduction makes the system no longer rely on the harsh requirement of "absolute synchronization" of the collection hardware, but instead adopts an intelligent fault-tolerant mechanism integrating "relative deviation control + tolerance perception + risk assessment", which significantly improves the stability, accuracy and practicality of the disease intelligent identification system in complex environments, and has wide engineering application value.

[0049] The acquisition logic of timestamp misalignment abnormality coefficient is as follows:

[0050] Calculate the time misalignment difference sequence: The mean of the middle time misalignment difference: , wherein is the time misalignment difference of the nth record, and N is the total number of time misalignment difference records; calculate the standard deviation of the time misalignment difference: ; calculate the misalignment alignment abnormality factor: ;

[0051] Different scales ( represent the mth scale, for example, the scale can be 1, 3, 5, 9, etc., and the scale is the number of adjacent units of a certain time misalignment difference in the time misalignment difference sequence) to construct a local smooth baseline of time difference : , wherein is the time misalignment difference of the kth record, and M is the total number of scales, to construct a multi-scale residual : ; then the residual stacking structure is: ;

[0052] Residual fluctuation energy calculation is performed on each point residual stacking: , wherein is the residual energy, ;

[0053] The residual fluctuation energy sequence is taken as a signal, and an empirical mode decomposition (EMD) method is used to separate the dominant fluctuation trend and the local abnormality item, to obtain the basic trend energy and the local enhancement item ;

[0054] It should be noted that EMD is an adaptive nonlinear non-stationary signal analysis method, and its basic idea is to decompose the signal into a plurality of intrinsic mode functions (IMF) with local characteristic frequency and a residual trend item, and the specific process is as follows: EMD decomposition is performed on the residual fluctuation energy: , wherein is the fth modal function, the frequency is from high to low, is the trend item; select a preset threshold frequency band FO, and split the IMF into: local enhancement item : Basic trend energy : ;

[0055] Calculate the abnormality significant factor: , wherein is the abnormality significant factor;

[0056] The timestamp misalignment abnormality coefficient is calculated as follows: wherein is the timestamp misalignment abnormality coefficient, respectively represent the misalignment abnormality factor, the preset proportion coefficient of the abnormality significance factor, and are both greater than 0;

[0057] It should be noted that the above formulas are all dimensionless numerical calculations, and common dimensionless methods include Min-Max normalization, Z-Score standardization, etc., which are not described here; According to the actual situation, for example, the expert weighting method is adopted, that is, experts in the relevant field are invited to determine the preset proportion coefficient of each index through professional opinion investigation and comprehensive evaluation, for example, may be 0.5, 0.5;

[0058] In step S3, the continuous trajectory points in the trajectory point set: are combined to form path line segments: , j = {1, 2,..., J-1}, J is the total number of trajectory points, is the jth path line segment;

[0059] The theoretical increment of the front and rear positions is calculated as follows: wherein is the theoretical increment, is the geographic coordinate of the trajectory point , and is the geographic coordinate of the trajectory point ; the corrected position corresponding to each trajectory point is obtained from the image frame positioning, and the actual increment is calculated as follows: wherein is the actual increment, is the corrected coordinate of the trajectory point , and is the corrected coordinate of the trajectory point , and the increment residual of the difference between the front and rear positions is obtained as follows: wherein is the increment residual, and the increment residual sequence is obtained as follows:

[0060] The increment residual mean in the increment residual sequence is calculated as follows: wherein is the increment residual mean; the time weight factor is defined as follows: wherein is the time weight factor, is the collection timestamp of the Jth trajectory point, ​a collection timestamp of the jth trajectory point;

[0061] calculating an exponential weighted residual cumulative mean: wherein is the exponential weighted residual cumulative mean;

[0062] calculating a geographic mapping error cumulative coefficient: wherein is the geographic mapping error cumulative coefficient, respectively represent preset proportion coefficients of the incremental residual mean and the exponential weighted residual cumulative mean, and are both greater than 0;

[0063] It should be noted that the above formulas are all dimensionless values calculated, and common dimension removal methods include Min-Max normalization, Z-Score standardization, etc., which will not be described here; According to the actual situation, for example, an expert weighting method is adopted, that is, experts in the relevant field are invited to determine the preset proportion coefficients of each index through professional opinion investigation and comprehensive evaluation, for example, may be 0.5, 0.5;

[0064] The geographic mapping error accumulation coefficient in the application is a core index for measuring the overall deviation degree and error accumulation intensity of trajectory data in the spatial geometric mapping process. The larger the geographic mapping error accumulation coefficient is, the more the geometric shape of the trajectory segment is distorted in unit path length, the more the direction consistency of the path segment is destroyed, the more the projection accuracy in the spatial principal axis direction is significantly reduced, and the more the continuous or sudden disturbance aggregation phenomenon exists in the local area, indicating that the trajectory spatial structure is seriously disturbed, and the spatial authenticity and reproducibility of the data are poor. Conversely, if the geographic mapping error accumulation coefficient is small, it indicates that the trajectory segment maintains good spatial consistency in the mapping process, the geometric increment residual fluctuates within a reasonable range, the overall structure of the trajectory is clear, and the local disturbance is uniformly distributed, and the mapping process accuracy is controllable. Based on the coefficient, the effective early warning of the spatial coincidence failure of the "trajectory-image data" in the disease identification process can be realized, that is, whether there is a "misplacement labeling" or "redundant repeated identification" problem of the image acquisition point and the actual disease occurrence area caused by trajectory distortion and mapping deviation. In the actual disease identification task, the mobile terminal labels the position of the disease area based on the spatial alignment relationship of the image frame and the trajectory point, but if the trajectory deviates greatly in the mapping process, the "spatio-temporal shadowing" phenomenon of the disease identification result in space will occur, that is, the same disease area is redundantly labeled by multiple image frames at different positions, or the image frame of the actual disease area is not correctly covered due to trajectory misplacement. This spatio-temporal shadowing risk will directly affect the accuracy and efficiency of disease detection, causing false positives, repeated reports, and missed reports of the disease identification system, and in severe cases, it will also lead to the implementation of the engineering repair plan based on the wrong positioning, resulting in cost waste and governance risks. Through continuous evaluation and dynamic monitoring of the geographic mapping error accumulation coefficient in the application, the system can judge whether the trajectory mapping accuracy is reduced and the path structure is distorted in real time, and can significantly prompt and intervene the shadowing risk caused by spatial misplacement, thereby improving the spatial accuracy and reliability of the disease identification task, and ensuring the engineering practicability and generalization stability of the intelligent inspection and high-precision identification system in complex scenes. Therefore, the coefficient has significant engineering value and popularization prospect in the application of disease identification and accurate map projection.

[0065] In step S4, a disease spatio-temporal shadowing risk assessment model is constructed according to the timestamp misalignment abnormality coefficient and the geographic mapping error accumulation coefficient, a disease spatio-temporal shadowing risk assessment index is output, the degree of spatio-temporal shadowing risk existing in the current road disease identification is evaluated, and the identified disease points are marked as shadowing risk points.

[0066] A disease spatio-temporal shadowing risk assessment model is constructed according to the timestamp misalignment abnormality coefficient and the geographic mapping error accumulation coefficient, and a disease spatio-temporal shadowing risk assessment index is output. The formula on which the disease spatio-temporal shadowing risk assessment model is based is as follows , wherein is the disease spatio-temporal shadowing risk assessment index, a timestamp misalignment abnormality coefficient, a geographic mapping error accumulation coefficient, respectively represent preset proportion coefficients of the timestamp misalignment abnormality coefficient and the geographic mapping error accumulation coefficient, and are both greater than 0;

[0067] It should be noted that the above formulas are all dimensionless numerical calculations, and common dimensionless methods include Min-Max normalization, Z-Score standardization, etc., which will not be described here; According to the actual situation, for example, an expert weighting method is adopted, that is, experts in the relevant field are invited to determine the preset proportion coefficients of each index through professional opinion investigation and comprehensive evaluation, for example, may be 0.5, 0.5;

[0068] As can be seen from the above calculation expression, the greater the timestamp misalignment abnormality coefficient and the geographic mapping error accumulation coefficient, the greater the disease spatio-temporal superposition risk assessment index, indicating that the current road disease identification result is more significantly affected by the spatio-temporal information mismatch, and there is a high potential risk of disease image label redundant annotation, spatial misalignment mapping or omission coverage, which is easy to lead to the problem of decline in accuracy of disease identification result in actual geographic space, increase in repeated judgment probability and waste of resources in maintenance plan. On the contrary, the smaller the timestamp misalignment abnormality coefficient and the geographic mapping error accumulation coefficient, the smaller the disease spatio-temporal superposition risk assessment index, indicating that in the current road disease identification process, the spatio-temporal correlation between the image frame and the track point is more accurate, the synchronization between the positioning information and the image acquisition data is better, and the spatial drift error in the geographic mapping process is smaller. In other words, the risk of mismatch or distortion of time and space two types of key basic data in the fusion process is lower, so the final output disease identification result has higher spatio-temporal credibility and positioning accuracy;

[0069] The disease spatio-temporal superposition risk assessment index is compared with a preset disease spatio-temporal superposition risk assessment index threshold value, and the identified disease points are marked as superposition risk points, as follows:

[0070] If the disease spatio-temporal superposition risk assessment index is greater than the disease spatio-temporal superposition risk assessment index threshold value, it indicates that there is significant time sequence mismatch and geographic mapping offset in the current spatio-temporal fusion process, and there is a relatively obvious time delay or spatial position error between the collected image frame and the corresponding track point, which leads to the problem of position overlap, time drift or image mislabeling in disease identification. The disease points identified in the search sliding time window are marked as superposition risk points;

[0071] If the disease spatiotemporal superposition risk assessment index is less than or equal to the disease spatiotemporal superposition risk assessment index threshold value, it indicates that the coupling between the image acquisition and the positioning trajectory in the time and space dimensions is relatively coordinated, the timestamp and spatial mapping of the disease recognition data maintain relatively high consistency, there is basically no significant synchronization deviation or geographical error accumulation, and the confidence of the disease annotation information is within an acceptable range, without the need for superposition risk point marking;

[0072] In step S5, the marked superposition risk points are fused based on the spatial consistency of the front and rear frames and the image feature similarity;

[0073] The set of marked superposition risk points is obtained The corresponding image frame set is The corresponding trajectory point set is ;

[0074] For any two adjacent superposition risk points and , the geographical distance is calculated : wherein is the latitude coordinate value corresponding to the superposition risk point , is the longitude coordinate value corresponding to the superposition risk point ; , respectively represent the latitude coordinate value and the longitude coordinate value corresponding to the superposition risk point ; the geographical distance is compared with a preset geographical distance threshold value to determine the spatial consistency of the two points: if the geographical distance is less than or equal to the geographical distance threshold value, it is considered that the two points have spatial consistency;

[0075] Local feature vectors are extracted from the image frames and , and common methods such as SIFT, ORB, SURF, and ResNet intermediate layer features are used to represent the feature vectors of the image frames and as and ;

[0076] The image similarity is calculated: ;

[0077] The image similarity is compared with a preset image similarity threshold value to determine the image feature similarity of the two points: if the image similarity is greater than or equal to the image similarity threshold value, it is considered that the two points have image feature similarity;

[0078] If both of the above two conditions are met at the same time: the geographical distance is less than or equal to the geographical distance threshold value, and the image similarity is greater than or equal to the image similarity threshold value, it is considered that the trajectory points and are the redundant recognition results of the same disease point, and the trajectory points and need to be weighted and averaged to be fused: wherein is the fused trajectory point, is a weighted fusion function used for weighted fusion of the corresponding coordinates of the trajectory points and , is the actual coordinate of the trajectory point , is the actual coordinate of the trajectory point , , respectively represent preset proportion coefficients of the actual coordinates of the trajectory points and , , are both greater than 0;

[0079] It should be noted that the above formulas are all dimensionless numerical calculations, and common dimensionless methods include Min-Max normalization, Z-Score standardization, etc., which will not be described here; , According to the actual situation, for example, an expert weighting method is adopted, that is, experts in the relevant field are invited to determine the preset proportion coefficients of each index through professional opinion investigation and comprehensive evaluation, for example, , may be 0.5, 0.5;

[0080] The application realizes the synchronous acquisition of continuous image frames and trajectory data by the joint driving of the camera equipment and the positioning module, and accurately records the timestamp information, laying a foundation for subsequent high-precision alignment. The dynamic pairing relationship between image frames and trajectory points is constructed by combining the sliding time window and the minimum time difference principle, and the timestamp misalignment alignment abnormal coefficient is introduced to quantify the severity of the time sequence drift in the terminal acquisition process, effectively depicting the influence of terminal sensor delay and synchronization error on recognition accuracy. At the same time, in the trajectory data processing process, the geographic mapping error accumulation coefficient is constructed by analyzing the spatial increment of continuous trajectory points, reflecting the spatial accumulation effect of positioning errors caused by GPS jump, signal shielding and other factors on the path, providing spatial resolution support for disease location reliability evaluation. The disease space-time superposition risk assessment model is constructed by fusing the above two types of coefficients, and the disease space-time superposition risk assessment index is output, realizing the spatial redundancy risk measurement of the identified disease, and dynamically completing the intelligent marking of the superposition risk points. Further, in order to suppress the redundant recognition caused by spatial superposition, based on the redundancy point fusion mechanism of spatial consistency and image feature similarity, the repeated disease points are accurately identified through geographic distance judgment and image feature vector similarity calculation, and the weighted fusion strategy is used to dynamically update the trajectory position, thereby effectively eliminating the repeated labeling phenomenon caused by the superposition of positioning drift and recognition error. In summary, the application realizes high-precision alignment and redundancy suppression of multi-source spatio-temporal data in a resource-constrained environment, not only improves the positioning accuracy and data reliability of disease identification, but also avoids the misleading influence of disease superposition on subsequent statistical analysis and maintenance scheduling, while ensuring the calculation efficiency, significantly improving the intelligent level and engineering application value of the system.

[0081] The above formulas are dimensionless numerical calculations, and the formulas are obtained by software simulation of a large amount of data to obtain a formula of the latest real situation, and the preset parameters in the formula are set by a person skilled in the art according to the actual situation.

[0082] It should be understood that the size of the sequence number of each process described above in various embodiments of the present application does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0083] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for intelligent collection and classification of road surface defects based on mobile terminals, characterized in that: Includes the following steps: Step S1: Start the camera device and positioning module in the mobile terminal to synchronously collect continuous image frames and location information, and record the collection timestamp of each image frame and each trajectory point data. Step S2: Based on the timestamp matching of the sliding time window, the image frame and the trajectory point are paired according to the principle of minimum time difference, and the timestamp misalignment information of the image frame and the trajectory data is obtained to calculate the timestamp misalignment alignment anomaly coefficient. Step S3: Combine continuous trajectory points into path segments and calculate the cumulative coefficient of geographic mapping error by obtaining the incremental residual of the difference between the positions before and after. Step S4: Construct a spatiotemporal overlay risk assessment model for road surface defects based on the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, output the spatiotemporal overlay risk assessment index for road surface defects, assess the degree of spatiotemporal overlay risk in the current road surface defect identification, and mark the identified defect points as overlay risk points. Step S5: For the marked overlapping risk points, redundant points are fused based on spatial consistency between consecutive frames and image feature similarity. The logic for obtaining the timestamp misalignment anomaly coefficient is as follows: Calculate the time misalignment sequence: Mean of time misalignment: ,in Let N be the time misalignment difference for the nth record, and N be the total number of records with time misalignment differences; calculate the standard deviation of the time misalignment difference: ; Calculate the misalignment anomaly factor: ; At different scales Constructing a time-difference local smoothing baseline : ,in Let M be the temporal misalignment difference of the k-th record, and M be the total number of scales. Construct multi-scale residuals. : The residual stacking structure is then: ; Calculate the residual fluctuation energy for each point of residual stacking: ,in For residual energy, ; Residual fluctuation energy sequence As a signal, the Empirical Mode Decomposition (EMD) method is used to separate the dominant fluctuation trend from local anomalies, thus obtaining the basic trend energy. With local enhancement terms ; Calculate the significance factor of the anomaly: , where ycx is an abnormally significant factor; Calculate the timestamp misalignment anomaly coefficient: ,in The timestamp misalignment anomaly coefficient. These represent the preset proportional coefficients for the misalignment anomaly factor and the anomaly significance factor, respectively. All are greater than 0.

2. The method for intelligent collection and classification of road surface defects based on mobile terminals according to claim 1, characterized in that: Timestamp matching based on a sliding time window pairs image frames with trajectory points according to the principle of minimum time difference, as follows: Define the current image frame The collection timestamp is The search sliding time window is then: ,in To define the window span of the search sliding time window; to extract the set of trajectory points within the search sliding time window: ,in For the j-th trajectory point, Let j be the timestamp of the acquisition of the j-th trajectory point; for all Calculate the time misalignment difference : Search makes Minimum trajectory point Record matching pairs and time misalignment ; The time misalignment sequence is obtained: ,in Let N be the time misalignment difference of the nth record, and N be the total number of records with time misalignment differences.

3. The method for intelligent collection and classification of road surface defects based on mobile terminals according to claim 1, characterized in that: In step S3, the set of trajectory points is: The continuous trajectory points in the path form a path segment: j={1,2,...,J-1}, where J is the total number of trajectory points. For the j-th path segment, For the j-th trajectory point, To search for a sliding time window; Calculate the theoretical increment of the position before and after: ,in For theoretical increments, For trajectory points Geographic coordinates For trajectory points Geographic coordinates; corrected position obtained from image frame localization. For each trajectory point, calculate the actual increment based on its actual mapped position: ,in This represents the actual increase. For trajectory points Corrected coordinates For trajectory points The corrected coordinates are obtained, and the incremental residuals of the positional differences before and after are acquired. ,in The incremental residuals are used to obtain the incremental residual sequence: ; Calculate the mean of the incremental residuals in the incremental residual sequence: ,in The mean of the incremental residuals; Define the time weighting factor: ,in As a time-weighted factor, Let J be the timestamp of the acquisition of the J-th trajectory point. The timestamp for the collection of the j-th trajectory point; Calculate the cumulative mean of the exponentially weighted residuals: ,in The cumulative mean of the exponentially weighted residuals; Calculate the cumulative coefficient of geographic mapping error: ,in This is the cumulative coefficient of geographic mapping error. These represent the preset proportionality coefficients for the incremental residual mean and the exponentially weighted cumulative residual mean, respectively. All are greater than 0.

4. The method for intelligent collection and classification of road surface defects based on mobile terminals according to claim 1, characterized in that: A disease spatiotemporal superimposition risk assessment model is constructed based on the timestamp misalignment anomaly coefficient and the geographic mapping error cumulative coefficient, outputting a disease spatiotemporal superimposition risk assessment index. The formula used in the disease spatiotemporal superimposition risk assessment model is as follows: In the formula This is a risk assessment index for the spatiotemporal overlay of diseases. The timestamp misalignment anomaly coefficient. This is the cumulative coefficient of geographic mapping error. These represent the preset proportional coefficients for the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, respectively. All are greater than 0.

5. The method for intelligent collection and classification of road surface defects based on mobile terminals according to claim 4, characterized in that: The spatiotemporal overlay risk assessment index of diseases is compared with the preset threshold for spatiotemporal overlay risk assessment index of diseases, and the identified disease points are marked as overlay risk points, as follows: If the disease spatiotemporal overlay risk assessment index is greater than the disease spatiotemporal overlay risk assessment index threshold, then the disease points identified within the search sliding time window will be marked as overlay risk points. If the disease spatiotemporal overlay risk assessment index is less than or equal to the disease spatiotemporal overlay risk assessment index threshold, then there is no need to mark the overlay risk point.

Images