Road surface disease intelligent acquisition and classification identification method based on mobile terminal

Abstract

The invention discloses a pavement disease intelligent acquisition and classification identification method based on a mobile terminal, and particularly relates to the technical field of pavement disease intelligent identification, synchronous acquisition of continuous image frames and track data is realized through combined driving of camera equipment and a positioning module, and the pavement disease intelligent acquisition and classification identification method based on the mobile terminal is realized by combining a sliding time window and a minimum time difference principle. A dynamic pairing relation between image frames and track points is constructed, a timestamp dislocation alignment anomaly coefficient is introduced, a geographic mapping error accumulation coefficient is constructed, the two types of coefficients are fused to construct a disease space-time foldover risk assessment model, a disease space-time foldover risk assessment index is output, and space redundancy risk measurement of disease identification is realized. Intelligent marking of foldover risk points is dynamically completed, repeated disease points are accurately recognized through geographic distance judgment and image feature vector similarity calculation on the basis of a redundant point fusion mechanism of space consistency and image feature similarity, and track positions of the repeated disease points are dynamically updated through a weighted fusion strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent identification of pavement defects, and more specifically, to a method for intelligent collection and classification of pavement defects based on a mobile terminal. Background Art

[0002] With the increasing demand for urban road maintenance and digital management, mobile-based pavement defect collection and identification technology has become a crucial tool for intelligent infrastructure monitoring. This approach typically uses an image acquisition device and positioning module mounted on a vehicle or pedestrian terminal to acquire road surface images and location information. This is then combined with an image recognition model to intelligently detect and classify defects such as cracks, potholes, and subsidence. However, in actual operation, the system faces a core problem of spatiotemporal coupling failure: misalignment between image frames and trajectory data, and the accumulation of geographic mapping errors.

[0003] During continuous data collection, due to problems such as sensor delays, network jitter, and inconsistent sampling frequencies during the movement of mobile terminals, it is easy to cause the image frame timestamp and trajectory timestamp to be misaligned, resulting in the inability of the disease identification results to accurately match their true geographic location. At the same time, in tunnels, urban canyons, or signal-blocked environments, GPS data is prone to jumps, drifts, or interruptions, causing positioning errors to continue to accumulate along the path. This spatiotemporal mismatch further leads to the problem of spatial overlap of diseases: the same disease may be marked by the system as a disease point at multiple spatial locations, resulting in redundant data and false hot spots, seriously affecting the accuracy of disease distribution statistics, map visualization, and subsequent maintenance task scheduling.

[0004] Therefore, how to effectively coordinate the high-precision synchronization of image frames and trajectories under the conditions of limited mobile terminal resources and insufficient data synchronization accuracy, and build a geographical mapping mechanism that is resistant to drift and jump, has become a key technical bottleneck that urgently needs to be broken through in the intelligent road defect identification system. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a method for intelligent collection and classification and identification of pavement defects based on a mobile terminal to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions: The method for intelligent collection and classification of pavement defects based on mobile terminals includes the following steps: Step S1, starting the camera device and positioning module in the mobile terminal, synchronously capturing continuous image frames and position information, and recording the acquisition timestamp of each image frame and each trajectory point data; Step S2: Pair the image frame with the trajectory point based on the timestamp matching of the sliding time window with the principle of minimum time difference, obtain the timestamp misalignment information of the image frame and the trajectory data, and calculate the timestamp misalignment alignment anomaly coefficient; Step S3, combining the continuous trajectory points into path segments, and obtaining the incremental residual of the position difference before and after to calculate the geographic mapping error accumulation coefficient; Step S4: construct a spatiotemporal overlap risk assessment model for the disease based on the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, output a spatiotemporal overlap risk assessment index for the disease, evaluate the spatiotemporal overlap risk level of the current pavement disease identification, and mark the identified disease points as overlap risk points; Step S5: For the marked overlapping risk points, redundant points are fused based on the spatial consistency of the previous and next frames and the similarity of image features.

[0007] In a preferred embodiment, image frames and trajectory points are paired based on the principle of minimum time difference based on timestamp matching of a sliding time window, as follows: Define the acquisition timestamp of the current image frame as The search sliding time window is: ,in is the window span of the search sliding time window; extract the trajectory point set within the search sliding time window: ,in is the jth trajectory point, is the acquisition timestamp of the jth trajectory point; for all , calculate the time offset difference : , search for Minimum trajectory point : ; Record matching pairs and time dislocation ; Get the time offset difference sequence: ,in is the time offset difference of the nth record, and N is the total number of records of time offset difference.

[0008] In a preferred embodiment, the logic for obtaining the timestamp misalignment anomaly coefficient is as follows: Calculate the time misalignment difference sequence: The mean of the time offset difference: ,in is the time offset of the nth record, N is the total number of records of time offset; calculate the standard deviation of the time offset: ; Calculate the misalignment anomaly factor: ; At different scales Constructing a local smooth baseline of temporal difference : ,in is the time offset of the kth record, M is the total number of scales, and the multi-scale residual is constructed : ; Then the residual stacking structure is: ; Calculate the residual fluctuation energy for each residual stack: ,in is the residual energy, ; The residual fluctuation energy series As a signal, the empirical mode decomposition method is used to separate the dominant fluctuation trend and local abnormal terms to obtain the basic trend energy. With local enhancement ; Calculate the abnormal significance factor: ,in is an abnormally significant factor; Calculate the timestamp misalignment anomaly coefficient: ,in is the timestamp misalignment anomaly coefficient, They represent the preset proportional coefficients of misalignment anomaly factor and anomaly significance factor, respectively, and Both are greater than 0.

[0009] In a preferred embodiment, in step S3, the trajectory points are grouped as follows: The continuous trajectory points in form path segments: ,j={1,2,...,J-1}, J is the total number of trajectory points, is the jth path segment; calculate the theoretical increment of the front and rear positions: ,in is the theoretical increment, For trajectory points The geographical coordinates of For trajectory points Get the corrected position from the image frame positioning Corresponding to the true mapping position of each trajectory point, calculate the actual increment: ,in is the actual increment, For trajectory points The corrected coordinates of For trajectory points The corrected coordinates of , and obtain the incremental residual of the position difference before and after: ,in is the incremental residual; the incremental residual sequence is obtained: ; Calculate the mean of the incremental residuals in the incremental residual series: ,in is the mean of the incremental residuals; Define the time weighting factor: ,in is the time weight factor, is the acquisition timestamp of the J-th trajectory point, is the acquisition timestamp of the j-th trajectory point; Compute the exponentially weighted cumulative mean residuals: ,in exponentially weighted cumulative mean residuals; Calculate the geographic mapping error accumulation coefficient: ,in is the geographic mapping error accumulation coefficient, represent the preset proportional coefficients of the incremental residual mean and the exponentially weighted residual cumulative mean, respectively, and Both are greater than 0.

[0010] In a preferred embodiment, a spatiotemporal risk assessment model for disease overlap is constructed based on the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, and a spatiotemporal risk assessment index for disease overlap is output. The spatiotemporal risk assessment model for disease overlap is based on the following formula: In the formula is the risk assessment index of spatiotemporal overlap of diseases, is the timestamp misalignment anomaly coefficient, is the geographic mapping error accumulation coefficient, They represent the preset proportional coefficients of the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, respectively, and Both are greater than 0.

[0011] In a preferred embodiment, the disease spatiotemporal overlap risk assessment index is compared with a preset disease spatiotemporal overlap risk assessment index threshold, and the identified disease points are marked as overlap risk points, as follows: If the disease spatiotemporal overlap risk assessment index is greater than the disease spatiotemporal overlap risk assessment index threshold, the disease point identified within the search sliding time window is marked as an overlap risk point; If the disease spatiotemporal overlap risk assessment index is less than or equal to the disease spatiotemporal overlap risk assessment index threshold, there is no need to mark the overlap risk points.

[0012] Technical effects and advantages of the present invention: 1. The present invention realizes the synchronous acquisition of continuous image frames and trajectory data through the joint drive of the camera device and the positioning module, and accurately records the timestamp information, laying the foundation for subsequent high-precision alignment. Combining the sliding time window and the minimum time difference principle, a dynamic pairing relationship between the image frame and the trajectory point is constructed, and the timestamp misalignment alignment anomaly coefficient is introduced to quantify the severity of the timing drift in the terminal acquisition process, and effectively characterize the impact of the terminal sensor delay and synchronization error on the recognition accuracy. At the same time, in the process of trajectory data processing, the geographic mapping error accumulation coefficient is constructed through the spatial incremental analysis of continuous trajectory points to reflect the spatial accumulation effect of the positioning error on the path caused by factors such as GPS jumps and signal blocking, and provide spatial resolution support for the credibility assessment of the disease location. The above two types of coefficients are integrated to construct a spatiotemporal overlap risk assessment model for the disease, and the spatiotemporal overlap risk assessment index for the disease is output to realize the spatial redundancy risk measurement of the identified disease and dynamically complete the intelligent marking of the overlap risk points. Furthermore, in order to suppress redundant identification caused by spatial overlap, a redundant point fusion mechanism based on spatial consistency and image feature similarity is used to accurately identify repeated disease points through geographic distance judgment and image feature vector similarity calculation, and dynamically update their trajectory positions using a weighted fusion strategy, thereby effectively eliminating the repeated labeling phenomenon caused by the superposition of positioning drift and recognition errors. In summary, the present invention achieves high-precision alignment and redundancy suppression of multi-source spatiotemporal data in a resource-constrained environment, which not only improves the positioning accuracy and data reliability of disease identification, but also avoids the misleading effects of disease overlap on subsequent statistical analysis and maintenance scheduling, significantly improving the intelligence level and engineering application value of the system while ensuring computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 Flowchart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0015] Example: Figure 1 The present invention provides a method for intelligently collecting and classifying road surface defects based on a mobile terminal, which includes the following steps: Step S1, starting the camera device and positioning module in the mobile terminal, synchronously capturing continuous image frames and position information, and recording the acquisition timestamp of each image frame and each trajectory point data; Step S2: Pair the image frame with the trajectory point based on the timestamp matching of the sliding time window with the principle of minimum time difference, obtain the timestamp misalignment information of the image frame and the trajectory data, and calculate the timestamp misalignment alignment anomaly coefficient; Step S3, combining the continuous trajectory points into path segments, and obtaining the incremental residual of the position difference before and after to calculate the geographic mapping error accumulation coefficient; Step S4: construct a spatiotemporal overlap risk assessment model for the disease based on the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, output a spatiotemporal overlap risk assessment index for the disease, evaluate the spatiotemporal overlap risk level of the current pavement disease identification, and mark the identified disease points as overlap risk points; Step S5: For the marked overlapping risk points, redundant points are fused based on the spatial consistency of the previous and next frames and the similarity of image features; In step S1, the camera device and positioning module in the mobile terminal are started to synchronously capture continuous image frames and position information, and the acquisition timestamp of each image frame and each trajectory point data is recorded, as follows: Start the road surface disease collection camera device and positioning module in the mobile terminal, load the local configuration file or remote configuration parameters, initialize the multi-threaded collection task controller, including the image collection thread and the positioning data monitoring thread, ensure that the two can run in parallel and support asynchronous data reading and writing in the buffer, call the camera driver interface, and set the image collection parameters including resolution, sampling frequency, exposure control parameters, etc.; for each image frame acquired, immediately record the system timestamp of the frame acquisition (based on the system's high-precision clock, such as Android's SystemClock.elapsedRealtimeNanos() or iOS's CMTime); package the image frame and its timestamp and cache it in a queue; start the positioning module (such as the GPS / Beidou / GLONASS module or the IMU+visual inertial fusion (VIO) positioning module); set the position update parameters including the position update frequency, etc.; for each trajectory point received (including longitude, latitude, speed, heading, etc.), immediately record the acquisition timestamp of the point; package the trajectory point and its timestamp and cache it in a queue; Step S2: Pair the image frames with the trajectory points based on the principle of minimum time difference based on the timestamp matching of the sliding time window, as follows: Define the current image frame The collection timestamp is , then the search sliding time window is: ,in is the window span of the search sliding time window; extract the trajectory point set within the search sliding time window: ,in is the jth trajectory point, is the acquisition timestamp of the jth trajectory point; for all , calculate the time offset difference : , search for Minimum trajectory point : ; Record matching pairs and time dislocation ; Get the time offset difference sequence: ,in is the time offset difference of the nth record, and N is the total number of records of time offset difference; Obtain the timestamp misalignment information of the image frame and trajectory data and calculate the timestamp misalignment alignment anomaly coefficient; The timestamp misalignment anomaly coefficient in the present invention is a key indicator used to measure the time synchronization accuracy and alignment between image frames and positioning trajectory data during the mobile terminal's road defect collection process. This coefficient essentially reflects the degree of timestamp matching deviation caused by various factors such as inconsistent sensor sampling frequencies, system clock offsets, and data cache delays during actual operation between the image acquisition module and the positioning module. The larger the timestamp misalignment anomaly coefficient, the greater the time interval between the image frame and its corresponding trajectory point, and the presence of obvious timestamp alignment anomalies, which may cause the defect content reflected in the image frame to be spatially mismatched with its actual collection location, resulting in recognition position drift or even serious mislabeling problems. The smaller the timestamp misalignment anomaly coefficient, the smaller the time difference between the image frame and the trajectory point, and the more accurate and reliable the correspondence. It can be considered that the current collection system has good synchronization performance and the coupling between the image and the trajectory is stable and reliable. In actual road inspection tasks, since mobile terminals are often in a continuous operation and high-frequency collection state, the system is susceptible to various jitters and interferences during operation. For example, when the terminal is traveling on a road section with uneven signal coverage or delayed hardware response, the acquisition time of image frames and trajectory data often changes unevenly, such as delayed trajectory upload, image cache blockage, etc., causing data items that should have been one-to-one corresponding to be misaligned in time. Once this misalignment is not discovered or corrected, it will directly lead to subsequent errors in defect labeling and positioning based on geographic coordinates. For example, a crack disease that actually occurs in the center of the road may be mistakenly marked as a non-disease area due to the delayed matching of the image frame to the next location point, or the same disease may be recorded repeatedly at multiple coordinates, resulting in the so-called "spatial overlap of diseases" problem. By constructing a timestamp misalignment alignment anomaly coefficient, the system can achieve a quantitative evaluation of the matching quality of each frame of image and the corresponding trajectory, and then infer the potential risks of the current overall recognition system at the time synchronization level. In the present invention, the anomaly coefficient is used to construct a spatiotemporal overlap risk assessment model for diseases. In this model, the timestamp misalignment alignment anomaly coefficient and the geographic mapping error accumulation coefficient work together to comprehensively evaluate whether a certain section of data has a potential overlap risk. If the timestamp alignment anomaly coefficient for a frame is high and accompanied by poor trajectory continuity (e.g., GPS jumps), redundant defect annotations due to spatiotemporal mismatch are highly likely. In summary, the timestamp misalignment anomaly coefficient, as a precision monitoring metric designed for mobile terminal defect collection systems, plays a significant role in resolving spatiotemporal coupling mismatches, improving defect annotation accuracy, and reducing data redundancy risks. Its introduction frees the system from relying on the stringent requirement for "absolute synchronization" of acquisition hardware, instead incorporating an intelligent fault-tolerance mechanism based on "relative deviation control + tolerance perception + risk assessment." This significantly improves the stability, accuracy, and practicality of intelligent defect recognition systems in complex environments, and has broad engineering application value.

[0016] The logic for obtaining the timestamp misalignment anomaly coefficient is as follows: Calculate the time offset difference sequence: The mean of the time offset difference: ,in is the time offset of the nth record, N is the total number of records of time offset; calculate the standard deviation of the time offset: ; Calculate the misalignment anomaly factor: ; At different scales ( represents the mth scale, for example, the scale can be 1, 3, 5, 9, etc. The scale is the number of adjacent units of a time offset difference in the time offset difference sequence) to construct a local smoothing baseline of the time difference : ,in is the time offset of the kth record, M is the total number of scales, and the multi-scale residual is constructed : ; Then the residual stacking structure is: ; Calculate the residual fluctuation energy for each residual stack: ,in is the residual energy, ; The residual fluctuation energy series As a signal, the empirical mode decomposition (EMD) method is used to separate the dominant fluctuation trend and local abnormal terms to obtain the basic trend energy With local enhancement ; It should be noted that EMD is an adaptive nonlinear non-stationary signal analysis method. Its basic idea is to decompose the signal into several intrinsic mode functions (IMFs) with local characteristic frequencies and a residual trend term. The details are as follows: EMD decomposition of the residual fluctuation energy: ,in is the fth mode function, with frequencies from high to low, For trend items; select a preset threshold frequency band FO to split the IMF into: local enhancement items : Basic Trend Energy : ; Calculate the abnormal significance factor: ,in is an abnormally significant factor; Calculate the timestamp misalignment anomaly coefficient: ,in is the timestamp misalignment anomaly coefficient, They represent the preset proportional coefficients of misalignment anomaly factor and anomaly significance factor, respectively, and All greater than 0; It should be noted that the above formulas are all dimensionless and numerical calculations. Common dimensionless methods include Min-Max normalization and Z-Score normalization, which will not be described here. Set it according to the actual situation. For example, adopt the expert empowerment method, that is, invite experts in related fields to determine the preset proportion coefficients of various indicators through professional opinion surveys and comprehensive evaluations, for example, It can be 0.5, 0.5; In step S3, the trajectory points are gathered: The continuous trajectory points in form path segments: , j={1,2,...,J-1}, J is the total number of trajectory points, is the j-th path segment; Calculate the theoretical increment of front and back positions: ,in is the theoretical increment, For trajectory points The geographical coordinates of For trajectory points Get the corrected position from the image frame positioning Corresponding to the true mapping position of each trajectory point, calculate the actual increment: ,in is the actual increment, For trajectory points The corrected coordinates of For trajectory points The corrected coordinates of , and obtain the incremental residual of the position difference before and after: ,in is the incremental residual; the incremental residual sequence is obtained: ; Calculate the mean of the incremental residuals in the incremental residual series: ,in is the incremental residual mean; define the time weight factor: ,in is the time weight factor, is the acquisition timestamp of the J-th trajectory point, is the acquisition timestamp of the j-th trajectory point; Compute the exponentially weighted cumulative mean residuals: ,in is the cumulative mean of exponentially weighted residuals; Calculate the geographic mapping error accumulation coefficient: ,in is the geographic mapping error accumulation coefficient, represent the preset proportional coefficients of the incremental residual mean and the exponentially weighted residual cumulative mean, respectively, and All greater than 0; It should be noted that the above formulas are all dimensionless and numerical calculations. Common dimensionless methods include Min-Max normalization and Z-Score normalization, which will not be described here. Set it according to the actual situation. For example, adopt the expert empowerment method, that is, invite experts in related fields to determine the preset proportion coefficients of various indicators through professional opinion surveys and comprehensive evaluations, for example, It can be 0.5, 0.5; The geographic mapping error accumulation coefficient in the present invention is a core indicator for measuring the overall deviation degree and error accumulation intensity of trajectory data in the process of spatial geometric mapping. The larger the geographic mapping error accumulation coefficient, the more significant the geometric shape of the trajectory segment within the unit path length is, the directional consistency of the path segment is destroyed, and the projection accuracy in the direction of the spatial main axis is significantly reduced. In addition, there is a continuous or sudden disturbance aggregation phenomenon in the local area, indicating that the trajectory spatial structure is seriously disturbed, and the spatial authenticity and reproducibility of the data are poor. On the contrary, if the geographic mapping error accumulation coefficient is small, it means that the trajectory segment maintains good spatial consistency during the mapping process, the geometric incremental residual fluctuates within a reasonable range, the overall structure of the trajectory is clear, the local disturbance is evenly distributed, and the accuracy of the mapping process is controllable. Based on this coefficient, an effective early warning of the failure of spatial overlap of "trajectory-image data" in the process of disease identification can be achieved, that is, it can be judged whether there is a problem of "misplaced labeling" or "redundant repeated identification" between the image acquisition point and the actual disease occurrence area caused by trajectory distortion and mapping deviation. In actual disease identification tasks, the mobile terminal marks the location of the diseased area based on the spatial alignment relationship between the image frame and the trajectory point. However, if the trajectory deviates significantly during the mapping process, the disease identification result will appear in space. The phenomenon of "space-time overlap" is that the same diseased area is redundantly marked by multiple image frames at different positions, or the image frame of the actual diseased area is not correctly covered due to trajectory misalignment. This risk of space-time overlap will directly affect the accuracy and efficiency of disease detection, causing false alarms, repeated reports, missed reports, etc. in the disease identification system. In severe cases, it will also cause the engineering repair plan to be implemented based on incorrect positioning, resulting in cost waste and governance risks. However, through the continuous evaluation and dynamic monitoring of the geographic mapping error accumulation coefficient in the present invention, the system can determine in real time whether the trajectory mapping accuracy has decreased and whether the path structure is distorted, and provide significant prompts and interventions for the overlap risk caused by spatial misalignment, thereby improving the spatial accuracy and reliability of the disease identification task and ensuring the engineering practicality and generalization stability of the intelligent inspection and high-precision identification system in complex scenarios. Therefore, this coefficient has significant engineering value and promotion prospects in the application of combining disease identification with precise map projection.

[0017] Step S4: construct a spatiotemporal overlap risk assessment model for the disease based on the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, output a spatiotemporal overlap risk assessment index for the disease, evaluate the spatiotemporal overlap risk level of the current pavement disease identification, and mark the identified disease points as overlap risk points; A spatiotemporal overlap risk assessment model for diseases is constructed based on the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, and the spatiotemporal overlap risk assessment index for diseases is output. The spatiotemporal overlap risk assessment model for diseases is based on the following formula: , where is the risk assessment index of spatiotemporal overlap of diseases, is the timestamp misalignment anomaly coefficient, is the geographic mapping error accumulation coefficient, They represent the preset proportional coefficients of the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, respectively, and All greater than 0; It should be noted that the above formulas are all dimensionless and numerical calculations. Common dimensionless methods include Min-Max normalization and Z-Score normalization, which will not be described here. Set it according to the actual situation. For example, adopt the expert empowerment method, that is, invite experts in related fields to determine the preset proportion coefficients of various indicators through professional opinion surveys and comprehensive evaluations, for example, It can be 0.5, 0.5; From the above calculation expression, it can be seen that the larger the timestamp misalignment anomaly coefficient and the larger the geographic mapping error cumulative coefficient, the larger the spatiotemporal overlap risk assessment index of the disease. This indicates that the current pavement disease identification results are more significantly affected by the mismatch of spatiotemporal information, and there is a high potential risk of redundant labeling of disease image labels, spatial misalignment mapping, or missed coverage. This can easily lead to a decrease in the accuracy of the disease identification results in the actual geographic space, an increase in the probability of repeated judgments, and a waste of resources in the maintenance plan. Conversely, the smaller the timestamp misalignment anomaly coefficient and the smaller the geographic mapping error cumulative coefficient, the smaller the spatiotemporal overlap risk assessment index of the disease. This indicates that in the current pavement disease identification process, the spatiotemporal correlation between image frames and trajectory points is more accurate, the synchronization between positioning information and image acquisition data is better, and the spatial drift error in the geographic mapping process is smaller. In other words, the risk of mismatch or distortion of the two key basic data types of time and space during the fusion process is low, so the final output of the disease identification result has higher spatiotemporal credibility and positioning accuracy. The disease spatiotemporal overlap risk assessment index is compared with the preset disease spatiotemporal overlap risk assessment index threshold, and the identified disease points are marked as overlap risk points, as follows: If the spatiotemporal overlap risk assessment index of the disease is greater than the spatiotemporal overlap risk assessment index threshold, it indicates that there is a significant temporal mismatch and geographic mapping offset in the current spatiotemporal fusion process, and there is a significant time delay or spatial position error between the acquired image frame and the corresponding trajectory point, which may lead to position overlap, time drift or image mislabeling in the identification of the disease. The disease points identified within the search sliding time window are marked as overlap risk points; If the spatiotemporal overlap risk assessment index of the disease is less than or equal to the spatiotemporal overlap risk assessment index threshold, it indicates that the coupling between image acquisition and positioning trajectory in the time and space dimensions is relatively coordinated, the timestamp and spatial mapping of the disease identification data maintain a high degree of consistency, there is basically no significant synchronization offset or geographic error accumulation, the confidence level of the disease annotation information is within an acceptable range, and there is no need to mark overlap risk points; Step S5: For the marked overlapping risk points, redundant points are fused based on the spatial consistency of the previous and next frames and the similarity of image features; Get the set of marked overlapping risk points , then the corresponding image frame set is , the corresponding trajectory point set is ; For any two adjacent overlapping risk points and Calculating geographic distance : ,in Risk point for overlapping The corresponding latitude coordinate value, Risk point for overlapping The corresponding longitude coordinate value; 、 Represents the overlapping risk points respectively The corresponding latitude and longitude coordinate values; compare the geographic distance with the preset geographic distance threshold to determine the spatial consistency of the two points: if the geographic distance is less than or equal to the geographic distance threshold, the two points are considered to have spatial consistency; From image frame and Extract local feature vectors, commonly used methods such as SIFT, ORB, SURF, ResNet intermediate layer features, etc., and convert the image frame and The eigenvectors of are expressed as and ; Calculating image similarity : ; Compare the image similarity with the preset image similarity threshold to determine the image feature similarity of the two points: if the image similarity is greater than or equal to the image similarity threshold, the two points are considered to have image feature similarity; If the above two judgment conditions are met at the same time: the geographical distance is less than or equal to the geographical distance threshold and the image similarity is greater than or equal to the image similarity threshold, the trajectory point is considered and For redundant identification results of the same disease point, it is necessary to identify the trajectory points and Perform weighted average fusion: in is the trajectory point after fusion, is a weighted fusion function used to derive the trajectory points and The corresponding coordinates of are weighted fused, For trajectory points The actual coordinates of For trajectory points The actual coordinates of 、 Represents trajectory points and trajectory points The preset scale factor of the actual coordinates, 、 All greater than 0; It should be noted that the above formulas are all dimensionless and numerical calculations. Common dimensionless methods include Min-Max normalization and Z-Score normalization, which will not be described here. 、 Set it according to the actual situation. For example, adopt the expert empowerment method, that is, invite experts in related fields to determine the preset proportion coefficients of various indicators through professional opinion surveys and comprehensive evaluations, for example, 、 It can be 0.5, 0.5; The present invention realizes the synchronous acquisition of continuous image frames and trajectory data through the joint drive of the camera device and the positioning module, and accurately records the timestamp information, laying the foundation for subsequent high-precision alignment. Combining the sliding time window and the minimum time difference principle, a dynamic pairing relationship between the image frame and the trajectory point is constructed, and the timestamp misalignment alignment anomaly coefficient is introduced to quantify the severity of the timing drift during the terminal acquisition process, and effectively characterize the impact of the terminal sensor delay and synchronization error on the recognition accuracy. At the same time, in the process of trajectory data processing, the geographic mapping error accumulation coefficient is constructed through the spatial incremental analysis of continuous trajectory points to reflect the spatial accumulation effect of the positioning error on the path caused by factors such as GPS jumps and signal shielding, providing spatial resolution support for the credibility assessment of the disease location. The above two types of coefficients are integrated to construct a spatiotemporal overlap risk assessment model for the disease, and the spatiotemporal overlap risk assessment index for the disease is output to realize the spatial redundancy risk measurement of the identified disease and dynamically complete the intelligent marking of the overlap risk points. Furthermore, in order to suppress redundant identification caused by spatial overlap, a redundant point fusion mechanism based on spatial consistency and image feature similarity is used to accurately identify repeated disease points through geographic distance judgment and image feature vector similarity calculation, and dynamically update their trajectory positions using a weighted fusion strategy, thereby effectively eliminating the repeated labeling phenomenon caused by the superposition of positioning drift and recognition errors. In summary, the present invention achieves high-precision alignment and redundancy suppression of multi-source spatiotemporal data in a resource-constrained environment, which not only improves the positioning accuracy and data reliability of disease identification, but also avoids the misleading effects of disease overlap on subsequent statistical analysis and maintenance scheduling, significantly improving the intelligence level and engineering application value of the system while ensuring computing efficiency.

[0018] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0019] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0020] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A mobile terminal-based intelligent pavement disease collection and classification identification method, characterized by: The steps include: Step S1, starting the camera device and positioning module in the mobile terminal, synchronously capturing continuous image frames and position information, and recording the acquisition timestamp of each image frame and each trajectory point data; Step S2: Pair the image frame with the trajectory point based on the timestamp matching of the sliding time window with the principle of minimum time difference, obtain the timestamp misalignment information of the image frame and the trajectory data, and calculate the timestamp misalignment alignment anomaly coefficient; Step S3, combining the continuous trajectory points into path segments, and obtaining the incremental residual of the position difference before and after to calculate the geographic mapping error accumulation coefficient; Step S4: construct a spatiotemporal overlap risk assessment model for the disease based on the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, output a spatiotemporal overlap risk assessment index for the disease, evaluate the spatiotemporal overlap risk level of the current pavement disease identification, and mark the identified disease points as overlap risk points; Step S5: For the marked overlapping risk points, redundant points are fused based on the spatial consistency of the previous and next frames and the similarity of image features.

2. The mobile terminal-based intelligent pavement disease collection and classification identification method according to claim 1 is characterized by: The timestamp matching based on the sliding time window is used to pair the image frames with the trajectory points according to the principle of minimum time difference, as follows: Define the current image frame The collection timestamp is , then the search sliding time window is: ,in is the window span of the search sliding time window; extract the trajectory point set within the search sliding time window: ,in is the jth trajectory point, is the acquisition timestamp of the jth trajectory point; for all , calculate the time offset difference : , search for Minimum trajectory point : ; Record matching pairs and time dislocation ; Get the time offset difference sequence: ,in is the time offset difference of the nth record, and N is the total number of records of time offset difference.

3. The mobile terminal-based intelligent pavement disease collection and classification identification method according to claim 2 is characterized by: The logic for obtaining the timestamp misalignment anomaly coefficient is as follows: Calculate the time offset difference sequence: The mean of the time offset difference: , where N is the time offset of the nth record and is the total number of records of time offset. Calculate the standard deviation of the time offset: ; Calculate the misalignment anomaly factor: ; At different scales Constructing a local smooth baseline of temporal difference : ,in is the time offset of the kth record, M is the total number of scales, and the multi-scale residual is constructed : ; Then the residual stacking structure is: ; Calculate the residual fluctuation energy for each residual stack: ,in is the residual energy, ; The residual fluctuation energy series As a signal, the empirical mode decomposition method is used to separate the dominant fluctuation trend and local abnormal terms to obtain the basic trend energy. with local enhancements; Calculate the abnormal significance factor: ,in is an abnormally significant factor; Calculate the timestamp misalignment anomaly coefficient: ,in is the timestamp misalignment anomaly coefficient, They represent the preset proportional coefficients of misalignment anomaly factor and anomaly significance factor respectively. And both are greater than 0.

4. The mobile terminal-based intelligent pavement disease collection and classification identification method according to claim 1 is characterized by: In step S3, the trajectory points are gathered: The continuous trajectory points in form path segments: , j={1,2,...,J-1}, J is the total number of trajectory points, is the j-th path segment; Calculate the theoretical increment of front and back positions: ,in is the theoretical increment, For trajectory points The geographical coordinates of For trajectory points Get the corrected position from the image frame positioning Corresponding to the true mapping position of each trajectory point, calculate the actual increment: ,in is the actual increment, For trajectory points The corrected coordinates of For trajectory points The corrected coordinates of , and obtain the incremental residual of the position difference before and after: ,in is the incremental residual; the incremental residual sequence is obtained: ; Calculate the mean of the incremental residuals in the incremental residual series: ,in is the mean of the incremental residuals; Define the time weighting factor: ,in is the time weight factor, is the acquisition timestamp of the J-th trajectory point, is the acquisition timestamp of the j-th trajectory point; Compute the exponentially weighted cumulative mean residuals: ,in is the cumulative mean of exponentially weighted residuals; Calculate the geographic mapping error accumulation coefficient: ,in is the geographic mapping error accumulation coefficient, represent the preset proportional coefficients of the incremental residual mean and the exponentially weighted residual cumulative mean, respectively, and Both are greater than 0.

5. The method for intelligent collection and classification of road surface defects based on a mobile terminal according to claim 1 is characterized by: A spatiotemporal overlap risk assessment model for diseases is constructed based on the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, and the spatiotemporal overlap risk assessment index for diseases is output. The spatiotemporal overlap risk assessment model for diseases is based on the following formula: , where is the risk assessment index of spatiotemporal overlap of diseases, is the timestamp misalignment anomaly coefficient, is the geographic mapping error accumulation coefficient, They represent the preset proportional coefficients of the timestamp misalignment anomaly coefficient and the geographic mapping error accumulation coefficient, respectively, and Both are greater than 0.

6. The method for intelligent collection and classification of road surface defects based on a mobile terminal according to claim 5 is characterized by: The disease spatiotemporal overlap risk assessment index is compared with the preset disease spatiotemporal overlap risk assessment index threshold, and the identified disease points are marked as overlap risk points, as follows: If the disease spatiotemporal overlap risk assessment index is greater than the disease spatiotemporal overlap risk assessment index threshold, the disease point identified within the search sliding time window is marked as an overlap risk point; If the disease spatiotemporal overlap risk assessment index is less than or equal to the disease spatiotemporal overlap risk assessment index threshold, there is no need to mark the overlap risk points.

Images