A road anomaly detection method and system

Abstract

The present invention discloses a road anomaly detection method and system, which belongs to the technical field of road detection. The method includes: obtaining gyroscope data and accelerometer data, and performing Kalman filtering on the gyroscope data and accelerometer data; obtaining road bump anomaly detection results based on the sampling frequency of the accelerometer data and the accelerometer data or the gyroscope data after Kalman filtering; obtaining mobile video, inputting the mobile video into a trained road anomaly detection model for processing, and obtaining road anomaly detection results; wherein, the mobile video is stored through a spatiotemporal query algorithm based on trajectory segmentation to improve the query efficiency of road anomaly detection. The method can improve the accuracy and efficiency of road anomaly detection and solve the problems of low robustness and accuracy of existing road detection.

Description

Technical Field

[0001] The present invention relates to the technical field of road detection, and in particular to a road anomaly detection method and system. Background Art

[0002] The statements in this section merely mention background art related to the present invention and do not necessarily constitute prior art.

[0003] With the booming economy, the automobile industry and infrastructure construction have undergone rapid and profound changes. Significant progress has been made in the automobile industry and infrastructure, and a large increase in the number of motor vehicles and road mileage has been achieved.

[0004] With the advancement of sensor technology and the rapid development of computers, computer vision and image processing have also developed. In this context, the concepts of smart transportation and smart cities have been proposed. An important part of smart transportation and smart cities is to use computer vision and sensors to detect road anomalies. Its main purpose is to improve the safety, efficiency and sustainability of urban transportation.

[0005] At present, there are two main types of road anomaly detection methods: one is based on Radar and GPS, which is divided into 2D and 3D; the other is based on a combination of sensors and some intelligent algorithms. Most of them are based on visual information (two-dimensional road information) or trajectory information (sensor data) for processing and detection, analyzing road anomalies, and most of them are based on the YOLOv5 framework for detection, which is difficult to reflect the road surface bump anomaly and its severity, and is prone to errors when processing complex data.

[0006] In addition, due to the different image scales and the fact that the features of tiny potholes are not obvious on road images, road anomaly detection based only on visual information requires high image quality on the one hand, and high visual detection algorithms on the other hand, making it difficult to detect tiny potholes and bumps. Large and fine cracks on the road are unlikely to cause significant bumps in the vehicle, and such road anomalies are difficult to detect accurately through sensor data. Therefore, based on a single data source, it is impossible to achieve accurate and comprehensive detection of road anomalies.

[0007] With the popularity of smart phones and car dashcams, a large number of mobile videos have been generated. The trajectory anomaly algorithm based on the Bi-LSTM algorithm model, grid index tree, and R-Tree algorithm cannot complete the anomaly detection in the visible area of ​​the video, so that the mobile video data is not fully utilized. Algorithms based on video detection models, such as self-feedback optimal subclass, FCN, and LSTM algorithms ignore the mobile nature of spatial positions, which is not conducive to efficient practical applications, because they only consider accelerometer sensors, resulting in data noise affecting the robustness of the algorithm. Summary of the invention

[0008] In order to address the deficiencies of the prior art, the present invention provides a road anomaly detection method, system, electronic device and computer-readable storage medium, which adopt sensor data and computer vision processing to assist in detection, and can efficiently and practically complete anomaly detection in the video visible area, accurately and comprehensively detect uneven anomalies in the road surface, and realize reliable and efficient road anomaly detection.

[0009] In a first aspect, the present invention provides a road anomaly detection method;

[0010] A road anomaly detection method, comprising:

[0011] Acquire gyroscope data and accelerometer data, and perform Kalman filtering on the gyroscope data and accelerometer data;

[0012] Based on the sampling frequency of the accelerometer data, the road bump anomaly detection result is obtained according to the accelerometer data or the gyroscope data after the Kalman filter;

[0013] Obtain mobile video, input the mobile video into the trained road anomaly detection model for processing, and obtain road anomaly detection results;

[0014] Among them, mobile videos are stored through a spatiotemporal query algorithm based on trajectory segmentation to improve the query efficiency of road anomaly detection.

[0015] Furthermore, Kalman filtering of the accelerometer data specifically includes:

[0016] Input the accelerometer data into the Kalman filter to obtain the predicted state value;

[0017] According to the error covariance at the previous moment, the error covariance at the current moment is obtained and the Kalman gain is calculated;

[0018] The predicted state value is updated according to the Kalman gain, the accelerometer data and the predicted state value to obtain the filtered data.

[0019] Furthermore, the data sampling frequency is calculated according to the accelerometer data by generating the data sampling frequency according to the timestamp difference between adjacent data points in the accelerometer data.

[0020] Furthermore, based on the sampling frequency of the accelerometer data, according to the accelerometer data or the gyroscope data after the Kalman filter, obtaining the road bump anomaly detection result specifically includes:

[0021] If the sampling frequency of the accelerometer data is greater than a preset sampling threshold, a road bump anomaly detection result is obtained based on the accelerometer data after Kalman filtering;

[0022] If the sampling frequency of the accelerometer data is less than the preset sampling threshold, the vehicle posture estimation result is obtained based on the gyroscope data after Kalman filtering; and the road bump anomaly detection result is obtained based on the vehicle posture estimation result.

[0023] Furthermore, the storing of mobile video by using a spatiotemporal query algorithm based on trajectory segmentation specifically includes:

[0024] Extracting original trajectory data from mobile video, dividing the original trajectory data by using a trajectory division strategy based on trajectory length, and obtaining multiple trajectory segments;

[0025] The mobile video is traversed according to the trajectory segment, and the storage structure and index are determined according to the attribute information of the trajectory segment to store the corresponding mobile video.

[0026] Furthermore, the road anomaly detection model is an improved YOLOv7 model.

[0027] Furthermore, the improved YOLOv7 model includes stacked multi-layer Conv convolutional layers, multiple convolutional layers and an ELAN module.

[0028] In a second aspect, the present invention provides a road anomaly detection system;

[0029] A road anomaly detection system, comprising:

[0030] The acquisition module is configured to: acquire gyroscope data and accelerometer data, and perform Kalman filtering on the gyroscope data and accelerometer data;

[0031] The road bump anomaly detection module is configured to: obtain a road bump anomaly detection result based on the sampling frequency of the accelerometer data and the accelerometer data or the gyroscope data after the Kalman filter;

[0032] The road anomaly detection module is configured to: obtain a mobile video, input the mobile video into a trained road anomaly detection model for processing, and obtain a road anomaly detection result;

[0033] Among them, mobile videos are stored through a spatiotemporal query algorithm based on trajectory segmentation to improve the query efficiency of road anomaly detection.

[0034] In a third aspect, the present invention provides an electronic device;

[0035] An electronic device includes a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are run by the processor, the steps of the road anomaly detection method are completed.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium;

[0037] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the steps of the above-mentioned road anomaly detection method are completed.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The technical solution provided by the present invention uses sensor data to detect road bump anomalies, and can also detect tiny bump anomalies on the road; uses mobile video data to detect road anomalies, and can accurately identify the existence of large and small cracks, puddles, speed bumps and other features. By combining the above two types of data, comprehensive and accurate detection of road anomalies is achieved.

[0040] 2. The technical solution provided by the present invention selects appropriate data for road bump anomaly detection according to the vehicle driving state in order to accurately characterize the posture change information of the vehicle under different driving conditions, thereby improving the accuracy of detection, and uses the Kalman filter algorithm to smooth the sensor data, thereby greatly improving the detection accuracy and the accuracy of the results.

[0041] 3. The technical solution provided by the present invention utilizes mobile video data containing information such as location, time, visible distance, viewing angle, and visible direction to perform road anomaly detection, and combines it with geographic spatial location information to improve the accuracy of road anomaly detection.

[0042] 4. The technical solution provided by the present invention uses an improved YOLOv7 to perform frame-by-frame detection on mobile videos with information labels and visible areas. The improved YOLOv7 model increases the model depth, improves the feature extraction capability of the model, and helps to improve the accuracy of road anomaly detection.

[0043] 5. The technical solution provided by the present invention uses trajectory segmentation technology to segment the input video of road anomaly detection to improve query efficiency. When querying a specific trajectory, by dividing the long trajectory into several shorter trajectory segments, the amount of data that needs to be processed for each query can be reduced, thereby reducing the time complexity of the query and improving performance; at the same time, when it is necessary to perform road anomaly detection on a specific road section, it is necessary to first perform a trajectory query, and then input the queried video into the anomaly detection algorithm. The use of trajectory segmentation technology not only reduces unnecessary data processing during query, but also reduces the amount of data processing in anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0045] Figure 1 A schematic diagram of a process flow provided by an embodiment of the present invention;

[0046] Figure 2 An example diagram of comparison between raw data and filtered data of an accelerometer provided in an embodiment of the present invention;

[0047] Figure 3 An example diagram of comparison between raw data and filtered data of a gyroscope provided in an embodiment of the present invention;

[0048] Figure 4 An example diagram of a target detection algorithm provided by an embodiment of the present invention;

[0049] Figure 5 A network architecture diagram of an ELAN module provided in an embodiment of the present invention;

[0050] Figure 6 An example diagram of data processing based on trajectory segmentation technology provided by an embodiment of the present invention;

[0051] Figure 7 An example diagram of a database entity provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0053] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0054] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0055] Embodiment 1

[0056] Existing road anomaly detection technologies are mostly based on single data and do not consider the road trajectory information corresponding to the data; therefore, the present invention provides a road anomaly detection method.

[0057] Next, combine Figure 1-Figure 7, a road anomaly detection method disclosed in this embodiment is described in detail. The road anomaly detection method includes the following steps:

[0058] S1. Obtaining a mobile video, wherein the mobile video is collected by a vehicle-mounted driving recorder, and the mobile video contains trajectory information, time, visible distance, viewing angle, and visible direction, etc., composed of gyroscope data and accelerometer data collected by an IMU inertial measurement unit and position information corresponding to each frame of video.

[0059] In a dash cam with an IMU inertial measurement unit, the stored mobile video will also contain the raw data of the gyroscope and accelerometer collected by the IMU.

[0060] In this embodiment, an open source data set of a driving recorder published on the Internet is used.

[0061] S2. Perform Kalman filtering on the accelerometer data and the gyroscope data. The accelerometer data and the gyroscope data are processed by the Kalman filter to remove noise and data drift, making the data more usable.

[0062] Noise interferes with the observation of sensor data. Data filtering is a data processing technique that removes noise to recover real data. As an implementation, the Kalman filter algorithm is used to filter accelerometer data and gyroscope data and generate smoother data from a series of data, where the measurement noise has a known measurement variance. The specific process is as follows:

[0063] S201, inputting accelerometer data into a Kalman filter to obtain a predicted state value.

[0064] S202: According to the error covariance at the previous moment, obtain the error covariance at the current moment and calculate the Kalman gain.

[0065] S203: Update the predicted state value according to the Kalman gain, the accelerometer data and the predicted state value to obtain filtered data.

[0066] For example, the accelerometer A z [i] and gyroscope G z [i] The z-axis data is extracted from the moving video and used as the Kalman filter z i The specific calculation formula is as follows:

[0067]

[0068] In the formula, the current state is inferred based on the state at the previous moment. is the predicted state value, which is the result of the prediction equation, x i-1 is the result of the last filter (i-1 moment).

[0069]

[0070] In the formula, the error covariance at the current moment is inferred based on the error covariance at the previous moment, is the prediction error covariance, P i-1 is the previous error covariance, q is the process noise, also known as the noise generated by the prediction model itself, and δ is the systematic measurement error as a constant.

[0071]

[0072] Here we calculate the Kalman gain, which is the result of the filter intermediate calculation, also known as the Kalman coefficient. g is the Kalman gain, r represents the measurement noise covariance, which can generally be observed during filtering.

[0073]

[0074] Here is the revised update of the state prediction to obtain the optimal estimate In the formula, z i is the observed value (the measurement data of the sensor, which is the accelerometer data or the gyroscope data in this embodiment), which is the input of the Kalman filter, is the residual between the actual observation and the predicted observation, and Kalman gain K g The predicted data are corrected together to obtain updated data.

[0075] The final step is to perform an update operation for the next iteration of the estimate, that is, to update P i value.

[0076]

[0077] Equations (1) and (2) generate predictions based on the calculations at the previous moment. Equations (3), (4), and (5) correct the predictions by combining them with the new measured variables. Based on the above formulas, data filtering is performed on the accelerometer data and the gyroscope data. Figure 2 and Figure 3 The following figure shows the changes of a small part (300) of BDD100K data before and after data filtering within 600 seconds. The parameters are set as follows: q = 0.012, r = 0.001. It can be seen from the figure that the filtered data becomes smoother.

[0078] S3. Calculate the data sampling frequency according to the gyroscope data; based on the data sampling frequency, obtain the road bump abnormality detection result according to the accelerometer data or the gyroscope data after Kalman filtering.

[0079] Considering that a car may have different running speeds when driving, in this embodiment, whether to use accelerometer data or gyroscope data for road bump detection is selected in combination with the running conditions of the car to improve the accuracy of detection.

[0080] The sampling frequency of the gyroscope is high, which can provide detailed information about attitude changes and is suitable for detecting fast movements or high-frequency turbulence. The sampling frequency of the accelerometer is relatively low, but it can provide more stable information about the direction of gravity and is suitable for detecting slower movements or low-frequency turbulence. Therefore, if the sampling frequency is high, the gyroscope data is used; if the sampling frequency is low, the accelerometer data is used.

[0081] As an implementation mode, step S3 specifically includes:

[0082] S301, calculating the data sampling frequency according to the gyroscope data. Specifically, when the data sampling frequency is greater than a preset sampling threshold, S302 is executed, and when the data sampling frequency is less than the preset sampling threshold, S303 is executed.

[0083] S302, using acceleration data to perform road bump anomaly detection, specifically, the acceleration data is collected by an accelerometer, including the angle between the vehicle's tilt angle and the gravity direction, in this step, the angle between the vehicle's tilt angle and the gravity direction is calculated. According to the angle between the vehicle's tilt angle and the gravity direction, the road bump anomaly detection result of the road section is obtained.

[0084] The angle between the vehicle's tilt angle and the direction of gravity reflects the changes in the vehicle's posture in the front, back, left, and right directions. For example, if the angle between the vehicle's tilt angle and the direction of gravity is a pitch angle, it means that there are abnormal bumps on the road surface; when encountering a bump, the body posture is first tilted up and then tilted down; when encountering a depression, it is tilted down and then tilted up. This is reflected in the accelerometer data as the vehicle's tilt angle first having a positive angle with the direction of gravity, then a negative angle; first a negative angle, then a positive angle. If the angle between the vehicle's tilt angle and the direction of gravity is a yaw angle, it reflects that the road surface is bumpy from side to side. If there is a pothole on the left side, the vehicle is tilted to the left, and the vehicle's tilt angle has a right angle with the direction of gravity; if there is a pothole on the right side, the vehicle is tilted to the right, and the vehicle's tilt angle has a left angle with the direction of gravity.

[0085] S303: Integrate the angular velocity of the vehicle using the gyroscope data, that is, the angular velocity data measured by the gyroscope, to obtain the rotation angle of the vehicle relative to the reference coordinate system, that is, the angular velocity variation. Convert the angular velocity variation into a quaternion for attitude estimation, obtain the attitude estimation result, and obtain the road bump abnormality detection result according to the attitude estimation result.

[0086] For example, if the posture estimation result shows that the posture of the vehicle body is undulating, then there are bumps on the road; if the posture estimation result shows that the posture is tilted to the left, then the left side of the road surface is concave.

[0087] As an implementation method, the angular velocity change is converted into a quaternion for attitude estimation. The specific process of obtaining the attitude estimation result is as follows:

[0088] (1) Posture initialization:

[0089] Initialize a quaternion q, usually set it to the unit quaternion, that is, q = (1, 0, 0, 0).

[0090] (2) Posture update:

[0091] Update the quaternion according to the change in angular velocity. Assume that the change in angular velocity is ω=(ω x ,ω y ,ω z ), the sampling time interval is Δt, then the formula for updating the quaternion is:

[0092]

[0093] Among them, q old is the quaternion at the previous moment, Represents quaternion multiplication.

[0094] (3) Unitization:

[0095] To ensure that the quaternion satisfies the unit length constraint, the updated quaternion needs to be normalized, as shown below:

[0096]

[0097] (4) Posture conversion:

[0098] The updated quaternion is converted into a rotation matrix or Euler angle as the vehicle's attitude estimation result.

[0099] (5) Iterative processing:

[0100] Repeat steps (3)-(5) to update and estimate the vehicle's posture estimation results.

[0101] S4. Input the mobile video into the trained road anomaly detection model for processing to obtain the road anomaly detection result.

[0102] In this embodiment, the YOLO algorithm is used for target detection. The YOLO algorithm uses a regression-based method to solve the target detection problem, which is different from the traditional method based on sliding windows. The model parameters are optimized by minimizing the loss function, so that the model can more accurately predict the location and category of the target. The target detection problem is converted into a regression problem, and the optimized target is directly modeled through the loss function to achieve end-to-end training, fast detection speed, better effect and higher efficiency when facing large data sets or real-time video detection.

[0103] As an implementation method, the road anomaly detection model is an improved YOLOv7 model, including three stacked Conv convolution layers, three convolution layers, a convolution layer with a stride of 2, and an ELAN module. The Conv convolution layer is in the standard "convolution + BN + SiLU" format. Next, YOLOv7 uses another convolution layer with a stride of 2 to obtain a four-fold downsampled image, and then processes this four-fold downsampled feature image through a series of convolutions of the ELAN module. The ELAN module is an efficient network structure that controls the shortest and longest gradient paths so that the network can learn more features and has stronger robustness. ELAN has two branches. The first branch is a 1×1 convolution to change the number of channels. The second branch is more complicated. It first passes through a 1×1 convolution module to change the number of channels, and then passes through four 3×3 convolution modules to extract features; finally, the four features are superimposed together to obtain the final feature extraction result.

[0104] In this embodiment, three convolutional layers are added on the basis of the 53 convolutional layers of the YOLOv7 model, which increases the model depth and improves the feature extraction capability of the model.

[0105] As an implementation method, the specific process of training the YOLOv7 model is as follows:

[0106] (1) Data preparation

[0107] In this example, the training dataset is from the self-driving cars in the Barkley DeepDrive lab (https: / / bair.berkeley.edu / blog / 2018 / 05 / 30 / bdd / ). These data are collected from real driving platforms and have four main properties: large-scale, diverse, captured on the street, and sensor-rich gyroscope and accelerometer information. Data diversity is particularly important for testing the robustness of algorithms.

[0108] The dataset contains 100,000 data. After pre-cleaning the data, 47,495 valid data were found on the street. The visible direction of the dataset is forward (dashcam), the visible range of the data is 50 meters, and the viewing angle of the data is 120°. Since the viewing angle of the dashcam is between 90° and 170°, and the viewing angle of the mobile phone is between 65° and 78°, the viewing angle is set to 120°.

[0109] (2) Dataset annotation and generation

[0110] After the data was prepared, we used the roboflow online annotation tool to annotate it. A total of 10 categories were annotated, namely: bulge, concavity, vehicle, sundries, pedestrian, crosswalk, speed bumps, crack, seeper, and snow. Then we performed data enhancement and used the roboflow built-in tools to increase the brightness and contrast of night photos, which is conducive to YOLO detection.

[0111] After labeling and data enhancement, divide the dataset into training set, validation set and test set according to 7:2:1 and download it to the computer. Then put it into the data file in the root directory of the YOLO model, which must contain three folders, train, valid and test, and the data.yaml file. Then put the .jpg images in the images folder and the labeled txt files in the labels folder.

[0112] (3) Start training

[0113] Before starting training, you need to download the pre-trained weight file from the YOLO official website, download yolov7n.pt and put it in the weights folder in the YOLOv7 root directory, then modify the parameters to start training.

[0114] In this embodiment, the training is set to iterate 300 times. The batch-size indicates how many pictures are viewed at a time before the weight is updated. According to the computer performance, it is set to 16 here.

[0115] (4) Training results analysis

[0116] After training, the results will be stored in the runs / train / exp folder in the YOLO root directory. The folder also contains some data obtained from this training:

[0117] Box: YOLO v7 uses GIOU Loss as the loss of the bounding box. Box is estimated to be the mean of the GIoU loss function. The smaller the box, the more accurate it is. Objectness: It is estimated to be the mean of the target detection loss. The smaller the target detection, the more accurate it is. Classification: It is estimated to be the mean of the classification loss. The smaller the classification, the more accurate it is. Precision: Precision (correct positive class / all positive classes found), the formula is Precision = TP / (TP+FP), which measures the probability that the positive class classified by the classifier used in this project is indeed the positive class. Recall: Recall rate (correct positive class / all positive classes that should have been found), the formula is Recall = TP / (TP+FN), which measures the ability of a classifier to find all positive classes.

[0118] After training and obtaining the weight file best.pt, copy the best weight file to the YOLO root directory, modify the dataset file path and weight file in the detect.py file, and start detecting mobile videos according to the trained ones.

[0119] As an implementation method, mobile videos are stored through a spatiotemporal query algorithm based on trajectory segmentation to improve the query efficiency of road anomaly detection.

[0120] In order to solve the problem of low query efficiency of long-trajectory mobile video data (over 8KB), in this embodiment, the trajectory segmentation technology is used to segment the mobile video data over 8KB, and the mobile video data with a longer trajectory is divided into several mobile video data with shorter trajectories, and stored separately, which can significantly improve the performance of continuous distance query.

[0121] Since the leaf nodes store mobile video data of no more than 8K in size, according to the principle of PostpreSQL data storage per page, the mobile video data is stored in segments and segmented by trajectory, so that the long-trajectory mobile video data is segmented into several short-trajectory mobile video data, thereby significantly improving the query performance. Because in general, setting the memory capacity of the pass cache area to 25% of the physical memory can obtain the best data query processing performance. Figure 7 As shown, for the long-track mobile video data (over 8KB) generated by the driving of a certain vehicle, the track segmentation technology is used to segment the data every time it exceeds 8KB, and divide it into three short-track mobile video data. The mobile video data is stored in segments, thereby achieving the purpose of improving query efficiency.

[0122] Motion trajectory segmentation is to segment the input video of road anomaly detection, and its purpose is to improve query efficiency. When querying a specific trajectory, by segmenting the long trajectory into several shorter trajectory segments, the amount of data that needs to be processed for each query can be reduced, thereby reducing the time complexity of the query and improving performance.

[0123] Similarly, when it is necessary to perform road anomaly detection on a specific road section, it is necessary to first perform a trajectory query and then input the queried video into the anomaly detection algorithm. It can be seen that the trajectory segmentation technology not only reduces unnecessary data processing during query, but also reduces the amount of data processing in anomaly detection.

[0124] For example, the specific process is as follows:

[0125] (1) Select a partitioning strategy for trajectory segmentation.

[0126] First, the original trajectory data contained in the mobile video is sorted in chronological order to ensure that the time order of the trajectory points is continuous. Then a distance-based trajectory segmentation strategy is selected, that is, the mobile video data is divided into different trajectory segments according to the length changes between trajectory points. The trajectory segments can be divided according to the length threshold, and the size threshold of each segment is 8K.

[0127] (2) Storage of trajectory segments.

[0128] The storage structure of trajectory data storage must be determined first. This patent uses the following attributes to describe each trajectory segment: trajectory segment ID, start time, end time, area information, and trajectory point data. Then a table named trajectory_segments is created in the database to store trajectory segment data.

[0129] The structure of the table is:

[0130]

[0131] Next, to reduce storage errors, traverse each segmented trajectory data to prepare for storage in a database or file system. Then, for each trajectory segment, store its attribute information in the corresponding field of the database table. Finally, make sure to commit the data storage operation to ensure persistent storage of the data.

[0132] (3) Index creation:

[0133] First, determine the fields that need to be indexed, including track segment ID, start time, end time, region information, and track point data. Second, for track data that needs to be queried by region, consider using the spatial index type.

[0134] Next, create an index in the database management system based on the selected index field and index type. After creating the index, you can verify the index to ensure that the index is created successfully and can improve query efficiency. Use comparison query execution time: execute the same query, test it with and without an index, and compare the query execution time. The query execution time is significantly shorter with an index, indicating that the index is effective. It should be noted that as data is added, deleted, or modified, the index may no longer be valid, so it is necessary to maintain the index regularly. You can rebuild the index regularly or perform operations such as index defragmentation to maintain the efficiency of the index.

[0135] (4) Execute the query:

[0136] First, the database system receives the query request submitted by the user. The query is usually submitted in the form of SQL (Structured Query Language). The database system will parse the SQL query submitted by the user and identify the grammatical structure and semantic meaning of the query. The parsing process includes lexical analysis, grammatical analysis and semantic analysis. After parsing the query, the database system will optimize the query to improve the efficiency of query execution. The goal of query optimization is to select the optimal execution plan, that is, to use system resources most effectively to execute the query. The optimizer will consider multiple execution strategies and access paths, and evaluate the cost and effect of each strategy to select the best execution plan. After the optimizer determines the best execution plan, the database system starts to execute the query. The execution plan includes a series of operation steps, such as table scanning, index use, join operations, aggregation operations, etc. The database system executes these operations in the order and manner specified in the execution plan to obtain the query results. In the process of executing the query plan, the database system needs to access the data stored in the database. This involves reading data in tables, indexes and other storage structures, and performing operations such as filtering, sorting and aggregation according to the query conditions. The database system will access data efficiently according to the order of operations and optimization strategies in the query plan. After executing all query operations, the database system will generate query results. The way the results are generated depends on the type of query, which may be a single value, a single row of data, multiple rows of data, or aggregated results. The database system will organize the query results into a suitable data structure and prepare to return them to the user.

[0137] Embodiment 2

[0138] This embodiment discloses a road anomaly detection system, including:

[0139] The acquisition module is configured to: acquire a mobile video, perform posture analysis on the mobile video, extract accelerometer data and gyroscope data, and perform Kalman filtering; calculate a first data sampling frequency according to the accelerometer data; and calculate a second data sampling frequency according to the gyroscope data;

[0140] The road bump abnormality detection module is configured to: obtain a road bump abnormality detection result based on the first data sampling frequency and the second data sampling frequency and according to the accelerometer data or the gyroscope data after the Kalman filter;

[0141] The road anomaly detection module is configured to: input the mobile video into the trained road anomaly detection model for processing to obtain a road anomaly detection result;

[0142] Among them, mobile videos are stored through a spatiotemporal query algorithm based on trajectory segmentation to improve the query efficiency of road anomaly detection.

[0143] It should be noted that the acquisition module, the road bump anomaly detection module and the road anomaly detection module correspond to the steps in Embodiment 1, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in Embodiment 1. It should be noted that the modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0144] Embodiment 3

[0145] Embodiment 3 of the present invention provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps of the above-mentioned road anomaly detection method are completed.

[0146] Embodiment 4

[0147] Embodiment 4 of the present invention provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps of the above-mentioned road anomaly detection method are completed.

[0148] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0149] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0150] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0151] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A road anomaly detection method, characterized in that: include: Acquire gyroscope data and accelerometer data, and perform Kalman filtering on the gyroscope data and accelerometer data; Based on the sampling frequency of the accelerometer data, the road bump anomaly detection result is obtained according to the accelerometer data or the gyroscope data after the Kalman filter; Obtain mobile video, input the mobile video into the trained road anomaly detection model for processing, and obtain road anomaly detection results; Among them, mobile videos are stored through a spatiotemporal query algorithm based on trajectory segmentation to improve the query efficiency of road anomaly detection; The storing of mobile video by using a spatiotemporal query algorithm based on trajectory segmentation specifically includes: Obtaining original trajectory data corresponding to the mobile video, dividing the original trajectory data using a distance-based trajectory division strategy, and obtaining multiple trajectory segments; The mobile video is traversed according to the trajectory segment, and the storage structure and index are determined according to the attribute information of the trajectory segment to store the corresponding mobile video.

2. The road anomaly detection method according to claim 1, characterized in that: Kalman filtering of accelerometer data specifically includes: Input the accelerometer data into the Kalman filter to obtain the predicted state value; According to the error covariance at the previous moment, the error covariance at the current moment is obtained and the Kalman gain is calculated; The predicted state value is updated according to the Kalman gain, the accelerometer data and the predicted state value to obtain the filtered data.

3. The road anomaly detection method according to claim 1, characterized in that: According to the accelerometer data, the data sampling frequency is calculated specifically as follows: the data sampling frequency is generated according to the time stamp difference between adjacent data points in the accelerometer data.

4. The road anomaly detection method according to claim 1, characterized in that: Based on the sampling frequency of the accelerometer data, the road bump anomaly detection results are obtained according to the accelerometer data or gyroscope data after Kalman filtering, including: If the sampling frequency of the accelerometer data is greater than a preset sampling threshold, a road bump anomaly detection result is obtained based on the accelerometer data after Kalman filtering; If the sampling frequency of the accelerometer data is less than the preset sampling threshold, the vehicle posture estimation result is obtained based on the gyroscope data after Kalman filtering; and the road bump anomaly detection result is obtained based on the vehicle posture estimation result.

5. The road anomaly detection method according to claim 1, characterized in that: The road anomaly detection model is an improved YOLOv7 model; The improved YOLOv7 model includes stacked multi-layer Conv convolutional layers, multiple convolutional layers and an ELAN module.

6. A road anomaly detection system, characterized in that: Implementing the road anomaly detection method as described in claims 1-5, comprising: The acquisition module is configured to: acquire gyroscope data and accelerometer data, and perform Kalman filtering on the gyroscope data and accelerometer data; the road bump abnormality detection module is configured to: based on the sampling frequency of the accelerometer data, obtain the road bump abnormality detection result according to the accelerometer data or the gyroscope data after the Kalman filtering; The road anomaly detection module is configured to: obtain a mobile video, input the mobile video into a trained road anomaly detection model for processing, and obtain a road anomaly detection result; Among them, mobile videos are stored through a spatiotemporal query algorithm based on trajectory segmentation to improve the query efficiency of road anomaly detection.

7. An electronic device, characterized in that: The invention comprises a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the road anomaly detection method according to any one of claims 1 to 5 is completed.

8. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the road anomaly detection method described in any one of claims 1 to 5.

Images