Bridge head vehicle bounce detection method and system integrating multi-dimensional data

Abstract

The present invention discloses a bridge head vehicle jump detection method and system integrating multi-dimensional data. The method comprises: obtaining real-time GPS location information of a vehicle and a road surface image; determining whether the current vehicle and the bridge have an intersection; if so, inputting the road surface image into a target segmentation model for segmentation; when there is a potential bridge head vehicle jump area in the road surface image, performing image processing on the result output by the target segmentation model; continuously tracking target features and acquiring vibration data; performing feature processing on the vibration data to form a vibration feature, and calculating the similarity with the target feature, respectively predicting the probability of bridge head vehicle jump according to the target feature and the vibration feature, and comprehensively evaluating the probability of bridge head vehicle jump in combination with the similarity and the two probability values ​​and determining whether the bridge head vehicle jump occurs. By implementing the method of the present invention, it is possible to integrate data from these different sources, effectively distinguish misjudgments caused by road bumps and other factors, and accurately identify the specific road section where the bridge head vehicle jump occurs.

Description

Technical Field

[0001] The present invention relates to a bridge head vehicle jump detection method, and more specifically to a bridge head vehicle jump detection method and system integrating multi-dimensional data. Background Art

[0002] Bridge jump refers to the height difference between the road surface and the bridge deck due to ground settlement or uneven changes at the junction of the road surface and the bridge deck, which causes obvious bumps when the vehicle is driving. When the vehicle passes through this area, especially during the transition from the road surface to the bridge deck or from the bridge deck to the road surface, it often feels a large vibration. Therefore, it is crucial to identify and detect whether there is a bridge jump. The presence of a bridge jump can be comprehensively judged by combining visual detection, sensor data collection and location information, and the relevant information can be transmitted to the traffic management department so that the road section with bridge jump can be maintained in time.

[0003] At present, the detection of bridge jump usually relies on gravity sensors, which can record the vibration of the vehicle when it is driving. By measuring data such as comfort, acceleration and impact coefficient, it is judged whether there is a bridge jump phenomenon. However, the detection method of gravity sensors has certain limitations, especially when facing the driver's slight bump operation, which will lead to misjudgment. For example, the driver may have bad driving habits or over-acceleration, which may cause the system to mistakenly judge it as a bridge jump phenomenon, resulting in false alarms or missed alarms, which in turn affects the accurate positioning of the bridge jump. This not only increases the risk of misjudgment, but may also cause the traffic management department to carry out a lot of ineffective work in subsequent road maintenance. Therefore, accurately predicting and judging the occurrence area of ​​bridge jump has become an urgent problem to be solved.

[0004] Therefore, it is necessary to design a new method to integrate data from these different sources, which can effectively distinguish misjudgments caused by road bumps and other factors, and accurately identify the specific section where the bridge head jump occurs. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a bridge head vehicle jump detection method and system integrating multi-dimensional data.

[0006] To achieve the above object, the present invention adopts the following technical solution: a bridge head vehicle jump detection method integrating multi-dimensional data, comprising:

[0007] Obtain the real-time GPS location information of the vehicle and the road surface images taken by the camera installed on the vehicle;

[0008] Determine whether the current vehicle and the bridge intersect according to the real-time GPS position information;

[0009] If the current vehicle and the bridge intersect, the road image is input into the target segmentation model to segment the potential area of ​​bridge head jump, so as to determine whether the road image has the potential area of ​​bridge head jump;

[0010] When there is a potential area for vehicle jumping at a bridge head in the road image, image processing is performed on the result output by the target segmentation model to obtain target features;

[0011] Continuously tracking the target feature and acquiring vibration data collected by a vibration sensor installed on the vehicle;

[0012] After feature processing of the vibration data, a vibration feature is formed, and the similarity is calculated with the target feature. The probability of bridge head jump is predicted according to the target feature and the vibration feature respectively. The similarity and the two probability values ​​are combined to comprehensively evaluate the probability of bridge head jump and determine whether the bridge head jump occurs.

[0013] A further technical solution is: judging whether the current vehicle and the bridge intersect according to the real-time GPS position information includes:

[0014] Comparing the real-time GPS location information with a pre-built bridge GPS database to obtain a comparison result;

[0015] Determining whether the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-built bridge GPS database meets the set requirements;

[0016] If the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-built bridge GPS database meets the set requirements, it is determined that the current vehicle and the bridge have an intersection;

[0017] If the comparison result is that the distance between the real-time GPS position information and the GPS position of any bridge in the pre-built bridge GPS database does not meet the set requirements, it is determined that the current vehicle and the bridge have no intersection.

[0018] A further technical solution is: the pre-constructed bridge GPS database is a database formed by collecting and recording specific geographical location information of the bridge.

[0019] A further technical solution is: the result output from the target segmentation model is subjected to image processing to obtain target features, including:

[0020] Generating a mask image based on the output result of the target segmentation model;

[0021] Cutting out a feature map of the bridge head jump area from the mask map;

[0022] Calculating the minimum bounding rectangle of the feature map, filling the blank area in the minimum bounding rectangle using an interpolation method, and rotating the filled feature map to obtain a rotated regular rectangular feature map;

[0023] The rotated rectangular feature map is converted into a feature vector of fixed size to obtain the target feature.

[0024] A further technical solution is: the vibration data is processed to form a vibration feature, and the similarity is calculated with the target feature, the probability of bridge head jump is predicted according to the target feature and the vibration feature, the probability of bridge head jump is comprehensively evaluated by combining the similarity and the two probability values, and whether the bridge head jump occurs is determined, including:

[0025] Performing feature processing on the vibration data to obtain vibration features;

[0026] Inputting the target feature and the vibration feature into a regression model, calculating the similarity, and obtaining a calculation result;

[0027] Inputting the target feature into an image classification model to predict the probability of a vehicle jumping at a bridge head, so as to obtain a first probability value;

[0028] Inputting the vibration characteristics into a sensor classification model to predict the probability of bridge head vehicle jumping to obtain a second probability value;

[0029] The final probability of a bridge head vehicle jump is comprehensively calculated according to the calculation result, the first probability value and the second probability value, and it is determined whether a bridge head vehicle jump event occurs.

[0030] A further technical solution is: the vibration data is subjected to feature processing to obtain vibration features, including:

[0031] The vibration data is subjected to matrix transposition to obtain vibration characteristics.

[0032] A further technical solution is: inputting the target feature and the vibration feature into a regression model, calculating the similarity, and obtaining a calculation result, including:

[0033] Inputting the target feature and the vibration feature into a regression model, and using the regression model to perform deep feature extraction on the target feature to obtain a deep image feature;

[0034] The deep image feature and the vibration feature are flattened into one-dimensional vectors respectively by the regression model, and the cosine similarity of the two one-dimensional vectors is calculated to obtain a calculation result.

[0035] A further technical solution is: the final probability of a bridge head vehicle jump is calculated comprehensively according to the calculation result, the first probability value and the second probability value, and whether a bridge head vehicle jump event occurs is determined, including:

[0036] Performing a weighted summation on the first probability value and the second probability value, and multiplying the summation result by the calculation result to obtain a final probability of bridge head vehicle jumping;

[0037] The final bridge head vehicle jump occurrence probability is compared with a set occurrence threshold value, and when the final bridge head vehicle jump occurrence probability is greater than the set occurrence threshold value, it is determined that the vehicle has a bridge head vehicle jump.

[0038] A further technical solution is: performing weighted summation on the first probability value and the second probability value, and multiplying the summation result by the calculation result to obtain a final probability of bridge head vehicle jumping, including:

[0039] use = 0.5×S×( + ) calculates the final probability of bridge jumping, where is the final probability of bridge head jumping; S is the calculation result; is the first probability value; is the second probability value.

[0040] The present invention also provides a bridge head vehicle jumping detection system integrating multi-dimensional data, comprising:

[0041] An acquisition unit, used to acquire real-time GPS location information of the vehicle and road surface images taken by a camera installed on the vehicle;

[0042] A judgment unit, used for judging whether the current vehicle and the bridge intersect according to the real-time GPS position information;

[0043] The target segmentation unit is used for inputting the road surface image into the target segmentation model to segment the potential area of ​​bridge head jump if the current vehicle and the bridge intersect, so as to determine whether the road surface image has the potential area of ​​bridge head jump jump;

[0044] An image processing unit, used for performing image processing on the result output by the target segmentation model to obtain target features when there is a potential area for vehicle jumping at a bridge head in the road image;

[0045] A vibration data acquisition unit, used to continuously track the target feature and acquire vibration data collected by a vibration sensor installed on the vehicle;

[0046] The probability calculation unit is used to perform feature processing on the vibration data to form a vibration feature, calculate the similarity with the target feature, predict the probability of bridge head jump according to the target feature and the vibration feature, and comprehensively evaluate the probability of bridge head jump by combining the similarity and the two probability values ​​to determine whether the bridge head jump occurs.

[0047] The beneficial effects of the present invention compared with the prior art are as follows: the present invention can effectively judge and distinguish the difference between a bridge head vehicle jumping event and other road bump factors by integrating real-time GPS location information, road surface images and vibration data. First, the real-time GPS information is used to judge whether the vehicle enters the bridge area to ensure that only the road sections related to the bridge are focused on. Then, the road surface image is input into the target segmentation model to extract the potential area where the bridge head vehicle jumping may exist. Then, in combination with the data collected by the vibration sensor, the vibration feature is generated through feature processing, and compared with the target image feature to calculate the similarity. Finally, accurate event prediction is achieved by comprehensively analyzing the similarity of the image features and the vibration features and the predicted bridge head vehicle jumping probability of the two. In this way, the misjudgment caused by other factors can be effectively filtered out, and the road section where the bridge head vehicle jumping occurs can be accurately identified.

[0048] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0050] Figure 1 A schematic diagram of a flow chart of a bridge head vehicle bounce detection method integrating multi-dimensional data provided by an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of target segmentation provided by an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of performing image processing on the result outputted from the target segmentation model provided by an embodiment of the present invention;

[0053] Figure 4 A schematic diagram of calculating similarity provided by an embodiment of the present invention;

[0054] Figure 5 A schematic diagram of calculating the final probability of a vehicle jumping at a bridge head provided by an embodiment of the present invention;

[0055] Figure 6A schematic block diagram of a bridge head vehicle jump detection system integrating multi-dimensional data provided by an embodiment of the present invention;

[0056] Figure 7 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0059] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0060] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0061] See also Figure 1 , Figure 1A schematic flow chart of a bridge head jump detection method integrating multi-dimensional data provided by an embodiment of the present invention. The bridge head jump detection method integrating multi-dimensional data is applied to a server, and the server interacts with the vehicle-mounted GPS, camera, and vibration sensor for data. By integrating the vehicle's GPS location information, the road surface image captured by the camera, and the vibration sensor data, the system can accurately locate whether the vehicle is close to the bridge; when the vehicle is detected to be close to the bridge, the target segmentation model is used to identify the potential area of ​​the bridge head jump, and the target feature is extracted from the area; at the same time, the vibration sensor data is feature processed to form a vibration feature, and the similarity is calculated with the target feature; the probability of bridge head jump predicted by the image classification model and the sensor classification model respectively, and the above similarity is combined to comprehensively evaluate the probability of bridge head jump; finally, whether the bridge head jump occurs is determined by the set threshold, so as to effectively distinguish the misjudgment caused by road bumps or other factors, and ensure the accurate identification of the specific road section of the bridge head jump.

[0062] Figure 1 FIG. 1 is a flow chart of a bridge head vehicle jumping detection method integrating multi-dimensional data provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110 to S160.

[0063] S110, obtaining real-time GPS location information of the vehicle and a road surface image captured by a camera installed on the vehicle.

[0064] In this embodiment, real-time GPS location information refers to real-time location information obtained by the GPS device on the vehicle, which usually includes data such as the vehicle's longitude, latitude, altitude, and current speed, and is used to determine the vehicle's precise location and driving status on the road.

[0065] Road surface images refer to images of the road surface taken in real time by cameras installed on vehicles. These images can be used to detect road conditions and identify potential bridge-head jumping phenomena.

[0066] Before acquiring data, the vehicle's GPS device, vibration sensor and camera need to be debugged to ensure that the three are working properly. The GPS can accurately locate the current position, the vibration sensor can effectively collect data such as acceleration, speed and displacement, and the camera can capture clear images.

[0067] S120. Determine whether the current vehicle and the bridge intersect based on the real-time GPS position information.

[0068] In one embodiment, the above-mentioned step S120 may include steps S121 to S124.

[0069] S121, comparing the real-time GPS location information with a pre-built bridge GPS database to obtain a comparison result.

[0070] The pre-built bridge GPS database is a database formed by collecting and recording the specific geographical location information of the bridge. Specifically, the GPS location information of the bridge is obtained to form a GPS database of the bridge, which marks the area D where the bridge is located.

[0071] S122, determining whether the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-built bridge GPS database meets the set requirements;

[0072] S123, if the comparison result is that the distance between the real-time GPS position information and the GPS position of any bridge in the pre-built bridge GPS database meets the set requirements, it is determined that the current vehicle and the bridge have an intersection;

[0073] S124. If the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-built bridge GPS database does not meet the set requirements, it is determined that the current vehicle and the bridge have no intersection.

[0074] In this embodiment, by comparing the real-time GPS location information with the pre-built bridge GPS database, it is possible to accurately determine whether the vehicle is approaching a bridge or entering a bridge area. This method can efficiently and accurately identify the spatial relationship between the vehicle and the bridge, ensuring that the intersection of the vehicle and the bridge is identified in a timely manner during driving. The bridge jump detection is performed only when there is an intersection with the bridge.

[0075] S130: If the current vehicle and the bridge intersect, the road surface image is input into a target segmentation model to segment a potential area for vehicle jumping over a bridge, so as to determine whether the road surface image has a potential area for vehicle jumping over a bridge.

[0076] In this embodiment, bridge jump refers to the bumping phenomenon caused by the height difference of the road surface when the vehicle passes through the bridge joint, which may increase the risk of driving safety, especially when driving at high speed. Therefore, accurate detection of potential bridge jump areas is of great significance for preventing traffic accidents, improving driving experience and maintaining road facilities. Once it is determined that the vehicle is at or close to the bridge, a specially trained object segmentation model is used to identify the possible bridge jump areas on the road surface.

[0077] For the above target segmentation model The training process is as follows:

[0078] First, a large number of images of bridge jumping need to be collected. This is the basis for the model to learn to identify the characteristics of bridge jumping. A large number of images containing bridge jumping phenomena need to be collected under different conditions (such as weather, lighting, time, etc.).

[0079] A large number of images Images2 that are similar to bridge jumps but are not bridge jumps are collected. These images are mainly used to improve the generalization ability of the model and prevent it from mistakenly identifying other types of road defects (such as cracks, potholes, etc.) as bridge jumps. By including these images, the model can learn to distinguish between real bridge jumps and other similar visual features.

[0080] And collect normal driving images Images3. This part of data is used to provide background information to help the model understand normal road conditions and better identify abnormal conditions.

[0081] In real scenarios, bridge bumps are similar to other road defects such as cracks. Therefore, when annotating the above Image2 and Image3, we divide them into two different categories and mark which parts are bridge bumps, which parts are other types of road defects, and which parts are normal roads. The purpose of this is to enable the model to distinguish different types of road conditions during the learning process.

[0082] After the labeling is completed, a classification algorithm with category 2 is designed to shield the easily confused road diseases.

[0083] The model is trained using annotated data sets, and the model performance is evaluated through methods such as cross-validation. In this process, the model structure and parameter settings are repeatedly adjusted until satisfactory recognition accuracy is achieved.

[0084] S140: When there is a potential area for vehicle jumping at a bridge head in the road image, image processing is performed on the result output by the target segmentation model to obtain target features.

[0085] In this embodiment, the target feature refers to a fixed-size feature vector extracted from the potential area of ​​the bridge head jump, obtained after a series of image processing steps (generating a mask image, cropping, calculating the minimum bounding rectangle, interpolation filling, rotation, and feature normalization). This feature vector It not only retains the main features of the bridge jump area, but also makes it suitable for various machine learning models through standardization. Specifically, the target feature The dimension of is 9×64, which means it is a matrix with 9 rows and 64 columns, and each row represents the information on one feature dimension.

[0086] The specific target segmentation model outputs the following results: Figure 2As shown, the potential area for vehicle jumping at the bridge head is identified in the input road surface image.

[0087] Once the target segmentation model successfully identifies the potential areas of bridge jump in the road image, the next key step is to extract useful information, namely target features, from these areas. These features will be used for subsequent analysis, such as assessing the risk of jump and locating the specific jump location. To this end, a series of sophisticated image processing techniques are required to ensure that the extracted features are both accurate and representative.

[0088] In one embodiment, see Figure 3 The above-mentioned step S140 may include steps S141~S144.

[0089] S141. Generate a mask image based on the output result of the target segmentation model.

[0090] In this embodiment, a mask image is generated This is to clearly mark the specific location of the potential area of ​​bridge head jump. This step is the basis for subsequent processing. Only by accurately marking the area of ​​interest can it be further analyzed and processed.

[0091] The output of the object segmentation model is an image with labels of the bridge jump area; the process of generating the mask image is actually to convert these labels into a binary image, where the pixel value of the bridge jump area is set to 1 (or 255) and the rest of the area is set to 0.

[0092] S142: Cut out a feature map of the bridge head vehicle jumping area from the mask map.

[0093] In this embodiment, the feature map corresponding to the bridge head vehicle jumping area is accurately extracted from the entire feature map, so as to perform a more in-depth analysis later.

[0094] Specifically, using the mask image The coordinates of the region with a median value of 1 are cut out from the original feature map F. This process ensures that only the feature information related to the bridge jump is selected, and the influence of irrelevant background is excluded. It is a binary image used to identify the target area. In the mask image, the pixel value of the bridge head jump area is 1 (or 255), and the pixel value of the background area is 0. The original feature map F is the road image taken by the camera installed on the vehicle; the area It refers to the set of pixels cropped from the original feature map F and marked as the target area in the mask image, that is, the bridge jump area.

[0095] S143, calculating the minimum circumscribed rectangle of the feature map, and using an interpolation method to fill the blank area in the minimum circumscribed rectangle, and rotating the filled feature map to obtain a rotated regular rectangular feature map.

[0096] In this embodiment, by calculating the minimum circumscribed rectangle and performing appropriate filling and rotation, it can be ensured that the extracted feature map not only maintains the shape information of the original area, but also facilitates subsequent standardization processing.

[0097] Specifically, first, find a bridgehead jump area that can completely surround it. The minimum rectangle This step helps to reduce unnecessary white space in the feature map while maintaining the true shape of the region.

[0098] Next, for For the part that does not belong to the bridge jump area, an interpolation method (such as bilinear interpolation, nearest neighbor interpolation, or bicubic interpolation) is used to fill the blank area. This can ensure the continuity and integrity of the feature map.

[0099] Finally, the padded feature map Rotate it to make it a regular rectangle. This step is to standardize the shape of the feature map for subsequent processing.

[0100] S144, converting the rotated rectangular feature map into a feature vector of a fixed size to obtain a target feature.

[0101] In this embodiment, the processed feature map is converted into a feature vector of a fixed size to facilitate subsequent use or storage of the machine learning model.

[0102] Specifically, the rotated rectangular feature map is converted into a fixed-size feature map, such as 9×64 feature dimensions, using methods such as global average pooling (avgPool), maximum pooling, bilinear interpolation, nearest neighbor interpolation, or bicubic interpolation. The final feature vector Contains key feature information of the bridge head jump area.

[0103] S150: Continue to track the target feature and obtain vibration data collected by a vibration sensor installed on the vehicle.

[0104] In this embodiment, since the camera continuously collects data, this process will be repeated continuously, so the features obtained are based on the time series. , the feature dimension is (n×9×64);

[0105] The single data collected by the vibration sensor has a data length of 9 and is arranged in the order of (x speed, y speed, z speed, x acceleration, y acceleration, z acceleration, x displacement, y displacement, z displacement). After multiple collections, it is combined into a matrix of (n×9) dimensions.

[0106] S160, performing feature processing on the vibration data to form a vibration feature, and calculating similarity with the target feature, predicting the probability of bridge head vehicle jumping according to the target feature and the vibration feature respectively, and comprehensively evaluating the probability of bridge head vehicle jumping by combining the similarity and the two probability values ​​to determine whether the bridge head vehicle jumping occurs.

[0107] In one embodiment, if Figure 4 and Figure 5 As shown, the above-mentioned step S160 may include steps S161~S165.

[0108] S161. Perform feature processing on the vibration data to obtain vibration features.

[0109] In this embodiment, the vibration data is subjected to matrix transposition to obtain vibration characteristics.

[0110] Specifically, the vibration data is transposed to generate a feature vector with a dimension of (9×n), namely, the vibration feature .

[0111] S162: Input the target feature and the vibration feature into a regression model, calculate the similarity, and obtain a calculation result.

[0112] In this embodiment, the calculation result refers to the cosine similarity value S between the image feature and the vibration feature calculated by the regression model. This value reflects the similarity between the image feature and the vibration feature in the feature space.

[0113] Image features and vibration features are two important data sources in bridge health monitoring and safety assessment. Image features usually come from visual sensors (such as cameras), while vibration features come from physical sensors (such as accelerometers). In order to integrate these two different types of features, a regression model is needed to establish the association between them and calculate their similarity. This helps to assess the status of the bridge, especially the severity of the bridge jumping phenomenon.

[0114] In one embodiment, if Figure 4 As shown, the above-mentioned step S162 may include steps S1621~S1622.

[0115] S1621. Input the target feature and the vibration feature into a regression model, and use the regression model to perform deep feature extraction on the target feature to obtain deep image features.

[0116] In this embodiment, deep feature extraction is performed on image features through a regression model to capture higher-level abstract features that can better reflect the essential information of the image.

[0117] Input features: target features (image features, dimension is n×9×64) and vibration features (Vibration feature, dimension is n×9). Regression model It contains multiple layers of neural networks, which are responsible for the input image features. Through the learning of these layers, the model can capture the deep patterns and structures in the image features. After being processed by multiple layers of neural networks, the model outputs deep image features. ', the feature dimension is n×9, these features are more compact and representative.

[0118] S1622. Flatten the deep image features and the vibration features into one-dimensional vectors respectively using the regression model, and calculate the cosine similarity of the two one-dimensional vectors to obtain a calculation result.

[0119] In this embodiment, the cosine similarity between the one-dimensional vectors of the deep image feature and the vibration feature is calculated to quantify the similarity between the two. Cosine similarity is a commonly used measurement method, which is particularly suitable for vector comparison in high-dimensional space.

[0120] Flattening operation: deep image features ' and vibration characteristics are flattened into one-dimensional vectors respectively. Assuming that the deep image features The dimension of ' is n×9, and the dimension of the flattened vector is (1×9n); the dimension of the vibration feature F_s is n×9, and the dimension of the flattened vector is (1×9n), 1×9n means that the row is 1 and the column is 9n, that is, a row or column vector with a length of 9n.

[0121] Calculate the cosine similarity of two one-dimensional vectors. The formula for calculating cosine similarity is: Cosine similarity= ; where A and B are the flattened deep image feature vector and vibration feature vector, respectively, and A⋅B is the dot product of the two vectors. and are the magnitudes of the two vectors respectively.

[0122] The final cosine similarity value S represents the similarity between the image feature and the vibration feature. This value usually ranges from -1 to 1. The closer the value is to 1, the higher the similarity between the two.

[0123] S163: Input the target feature into an image classification model to predict the probability of vehicle jumping at a bridge head, so as to obtain a first probability value.

[0124] In this embodiment, the first probability value Refers to the image classification model The output probability value of the vehicle jumping at the bridge head reflects the matching degree between the image features and the vehicle jumping at the bridge head scene.

[0125] Image Classification Model The purpose is to determine whether a bridge jump has occurred through image features. Image features usually contain rich visual information and can intuitively reflect the state and environmental conditions of the bridge. By training the image classification model, the typical features of bridge jump can be effectively identified, thereby improving the accuracy and reliability of detection.

[0126] Input features are target features , with a dimension of (n×96×4), represents the feature vector extracted from the image.

[0127] Image Classification Model It is a deep learning model, which usually contains convolutional layers, pooling layers, fully connected layers, etc. These layers can perform multi-level abstraction and extraction of input image features to capture the key features of the image.

[0128] The target feature Input to the image classification model In the process, the model will propagate forward through each layer in turn and finally output a probability value .

[0129] The first probability value Representing image features The probability of a car jumping off the bridge scenario. The value range of is [0,1]. The closer the value is to 1, the more likely the image feature belongs to the bridge jumping scene.

[0130] S164: Input the vibration characteristics into a sensor classification model to predict the probability of bridge head vehicle jumping to obtain a second probability value.

[0131] In this embodiment, the second probability value Refers to the sensor classification model The output probability value of the bridge head vehicle jumping reflects the matching degree between the vibration characteristics and the bridge head vehicle jumping scenario.

[0132] Specifically, the sensor classification model The purpose is to determine whether a bridge jump has occurred by using vibration characteristics. Vibration characteristics usually come from sensors installed on the bridge, which can collect dynamic response data of the bridge in real time. By training the sensor classification model, the specific vibration mode caused by bridge jump can be effectively identified, thus providing another perspective of the detection results.

[0133] Input feature is vibration feature , with a dimension of (n×9), represents the feature vector extracted from the vibration sensor.

[0134] Sensor Classification Model It is also a deep learning model, which usually contains multi-layer perceptron (MLP), recurrent neural network (RNN), etc. These layers can abstract and extract the input vibration features at multiple levels, thereby capturing the key features in the vibration signal.

[0135] The vibration characteristics Input to sensor classification model In the process, the model will propagate forward through each layer in turn and finally output a probability value .

[0136] The second probability value Indicates vibration characteristics The probability of a car jumping off the bridge scenario. The value range of is also [0, 1]. The closer the value is to 1, the more likely the vibration feature belongs to the bridge head jumping scene.

[0137] S165, comprehensively calculating a final probability of a bridge head vehicle jump according to the calculation result, the first probability value, and the second probability value, and determining whether a bridge head vehicle jump event occurs.

[0138] In order to more accurately detect the bridge jumping event, this embodiment adopts a multimodal data fusion method. By combining image features and vibration features, the accuracy and robustness of detection can be improved. Specifically, the deep image features are extracted through the regression model, and the cosine similarity between them and the vibration features is calculated to obtain the calculation result S. At the same time, the two classification models are used to output the first probability value and the second probability value Finally, the final probability of bridge head jumping is calculated by weighted summation .

[0139] In one embodiment, the above-mentioned step S165 may include steps S1651~S1652.

[0140] S1651. Perform a weighted summation on the first probability value and the second probability value, and multiply the summation result by the calculation result to obtain a final probability of vehicle jumping at a bridge head.

[0141] In this embodiment, the = 0.5××S××( + ) calculates the final probability of bridge jumping, where is the final probability of bridge head jumping; S is the calculation result; is the first probability value; is the second probability value. This probability is determined by S and and Joint decision, if When S are all close to 1, the overall probability is close to 1, that is, the three factors jointly determine the probability of vehicle jumping at the bridge head.

[0142] Specifically, 0.5 is a weight factor used to balance and This weight factor can be adjusted according to actual conditions.

[0143] when , When and S are close to 1, It will also be close to 1, indicating that the probability of the vehicle jumping off the bridge is very high.

[0144] when , Or when S is close to 0, It will also be close to 0, indicating that the probability of jumping off the bridge is very low.

[0145] S1652: Compare the final probability of vehicle jumping at a bridge head with a set occurrence threshold, and when the final probability of vehicle jumping at a bridge head is greater than the set occurrence threshold, determine that the vehicle has experienced a vehicle jumping at a bridge head.

[0146] In this embodiment, a threshold value T is set to distinguish whether a vehicle jump at a bridge head occurs. The selection of the threshold value can be determined according to actual needs and experience.

[0147] The calculated final bridge head vehicle jumping probability Compare with the set threshold T.

[0148] if >T, it is determined that the vehicle has jumped off the bridge.

[0149] if ≤T, it is considered that no bridge jump occurs.

[0150] For example: Assume that the threshold T=0.66: If =0.7, then it is considered that a bridge jump has occurred. =0.5, it is considered that no bridge jump occurs.

[0151] Assume that the threshold T=0.4: If =0.5, it is considered that a bridge jump has occurred. =0.3, it is considered that no bridge jump occurs.

[0152] By combining image features and vibration features, the risk of bridge head jumping can be more comprehensively assessed to reduce misjudgments and missed judgments. Data from different modes can complement each other. Even if the data of one mode is abnormal, the data of other modes can still provide effective information. By adjusting the threshold, the standard of bridge head jumping can be flexibly determined according to different application scenarios and needs.

[0153] Through step S165 and its sub-steps S1651 and S1652, the multi-modal data can be effectively integrated to calculate the final probability of bridge head vehicle jumping, and determine whether a bridge head vehicle jumping event occurs according to the set threshold. This method not only improves the accuracy and robustness of detection, but also provides a scientific basis for subsequent decision-making.

[0154] In addition, if the current vehicle has no intersection with the bridge, the process proceeds to the end step.

[0155] The above-mentioned bridge head jump detection method integrating multi-dimensional data can effectively judge and distinguish the difference between bridge head jump events and other road bump factors by integrating real-time GPS location information, road surface images and vibration data. First, the real-time GPS information is used to determine whether the vehicle has entered the bridge area to ensure that only the road sections related to the bridge are focused on. Then, the road surface image is input into the target segmentation model to extract potential areas where bridge head jump may exist. Then, combined with the data collected by the vibration sensor, vibration features are generated through feature processing, and compared with the target image features to calculate their similarity. Finally, accurate event prediction is achieved by comprehensively analyzing the similarity of image features and vibration features and the predicted bridge head jump probability of the two. In this way, misjudgments caused by other factors can be effectively filtered out, and the road sections where bridge head jump occurs can be accurately identified.

[0156] Figure 6 FIG. 3 is a schematic block diagram of a bridge head vehicle jumping detection system 300 integrating multi-dimensional data provided by an embodiment of the present invention. Figure 6As shown, corresponding to the above bridge head jump detection method integrating multi-dimensional data, the present invention also provides a bridge head jump detection system 300 integrating multi-dimensional data. The bridge head jump detection system 300 integrating multi-dimensional data includes a unit for executing the above bridge head jump detection method integrating multi-dimensional data, and the system can be configured in a server. Specifically, please refer to Figure 6 The bridge head vehicle jumping detection system 300 integrating multi-dimensional data includes an acquisition unit 301, a judgment unit 302, a target segmentation unit 303, an image processing unit 304, a vibration data acquisition unit 305 and a probability calculation unit 306.

[0157] The acquisition unit 301 is used to acquire the real-time GPS position information of the vehicle and the road surface image taken by the camera installed on the vehicle; the judgment unit 302 is used to judge whether the current vehicle and the bridge have an intersection according to the real-time GPS position information; the target segmentation unit 303 is used to input the road surface image into the target segmentation model to segment the potential area of ​​bridge head jump if the current vehicle and the bridge have an intersection, so as to determine whether the road surface image has the potential area of ​​bridge head jump; the image processing unit 304 is used to perform image processing on the result output by the target segmentation model when the road surface image has the potential area of ​​bridge head jump to obtain the target feature; the vibration data acquisition unit 305 is used to continuously track the target feature and acquire the vibration data collected by the vibration sensor installed on the vehicle; the probability calculation unit 306 is used to perform feature processing on the vibration data to form a vibration feature, calculate the similarity with the target feature, predict the probability of bridge head jump according to the target feature and the vibration feature respectively, and comprehensively evaluate the probability of bridge head jump in combination with the similarity and the two probability values ​​to determine whether the bridge head jump occurs.

[0158] In one embodiment, the determining unit 302 includes:

[0159] a comparison subunit, for comparing the real-time GPS position information with a pre-constructed bridge GPS database to obtain a comparison result; a distance judgment subunit, for judging whether the comparison result is that the distance between the real-time GPS position information and the GPS position of any bridge in the pre-constructed bridge GPS database meets the set requirements; a first determination subunit, for determining that the current vehicle and the bridge have an intersection if the comparison result is that the distance between the real-time GPS position information and the GPS position of any bridge in the pre-constructed bridge GPS database meets the set requirements; and a second determination subunit, for determining that the current vehicle and the bridge have no intersection if the comparison result is that the distance between the real-time GPS position information and the GPS position of any bridge in the pre-constructed bridge GPS database does not meet the set requirements.

[0160] In one embodiment, the image processing unit 304 includes:

[0161] A mask image generation subunit is used to generate a mask image for the result output by the target segmentation model; a cropping subunit is used to crop a feature image of a bridge head jump area from the mask image; a rotation subunit is used to calculate the minimum circumscribed rectangle of the feature image, fill the blank area in the minimum circumscribed rectangle by using an interpolation method, and rotate the filled feature image to obtain a rotated regular rectangular feature image; a conversion subunit is used to convert the rotated regular rectangular feature image into a feature vector of a fixed size to obtain a target feature.

[0162] In one embodiment, the probability calculation unit 306 includes:

[0163] A processing subunit is used to perform feature processing on the vibration data to obtain a vibration feature; a similarity calculation subunit is used to input the target feature and the vibration feature into a regression model, calculate the similarity, and obtain a calculation result; a first prediction subunit is used to input the target feature into an image classification model to predict the probability of a bridge head jump to obtain a first probability value; a second prediction subunit is used to input the vibration feature into a sensor classification model to predict the probability of a bridge head jump to obtain a second probability value; a comprehensive judgment subunit is used to comprehensively calculate a final bridge head jump probability based on the calculation result, the first probability value, and the second probability value, and determine whether a bridge head jump event occurs.

[0164] In one embodiment, the processing subunit is used to perform matrix transposition on the vibration data to obtain vibration characteristics.

[0165] In one embodiment, the similarity calculation subunit includes:

[0166] A deep feature extraction module, used for inputting the target feature and the vibration feature into a regression model, and performing deep feature extraction on the target feature by the regression model to obtain a deep image feature;

[0167] The cosine similarity calculation module is used to flatten the deep image features and the vibration features into one-dimensional vectors respectively by the regression model, and calculate the cosine similarity of the two one-dimensional vectors to obtain a calculation result.

[0168] In one embodiment, the comprehensive judgment subunit includes:

[0169] The final probability determination module is used to perform weighted summation on the first probability value and the second probability value, and multiply the summation result by the calculation result to obtain the final probability of bridge head jump; specifically, using =0.5×S×( + ) calculates the final probability of bridge jumping, where is the final probability of bridge head jumping; S is the calculation result; is the first probability value; is the second probability value.

[0170] The threshold comparison module is used to compare the final bridge head vehicle jump probability with the set occurrence threshold, and when the final bridge head vehicle jump probability is greater than the set occurrence threshold, it is determined that the vehicle has a bridge head vehicle jump.

[0171] It should be noted that technicians in the relevant field can clearly understand that the specific implementation process of the bridge head vehicle jump detection system 300 integrating multi-dimensional data and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0172] The bridge head vehicle jump detection system 300 integrating multi-dimensional data can be implemented in the form of a computer program. The computer program can be used in Figure 7 Runs on the computer device shown.

[0173] See also Figure 7 , Figure 7 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0174] See also Figure 7 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0175] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a bridge head vehicle jump detection method integrating multi-dimensional data.

[0176] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0177] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a bridge head vehicle jump detection method integrating multi-dimensional data.

[0178] The network interface 505 is used to communicate with other devices over the network. Figure 7 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0179] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0180] The real-time GPS location information of the vehicle and the road surface image taken by the camera installed on the vehicle are obtained; whether the current vehicle and the bridge intersect is determined according to the real-time GPS location information; if the current vehicle and the bridge intersect, the road surface image is input into the target segmentation model to segment the potential area of ​​bridge head jump, so as to determine whether the road surface image has the potential area of ​​bridge head jump; when the road surface image has the potential area of ​​bridge head jump, image processing is performed on the result output by the target segmentation model to obtain the target feature; the target feature is continuously tracked, and vibration data collected by the vibration sensor installed on the vehicle is obtained; vibration features are formed after feature processing of the vibration data, and the similarity is calculated with the target feature, the probability of bridge head jump is predicted according to the target feature and the vibration feature respectively, and the probability of bridge head jump is comprehensively evaluated by combining the similarity and the two probability values ​​to determine whether the bridge head jump occurs.

[0181] In one embodiment, when the processor 502 implements the step of determining whether the current vehicle and the bridge intersect according to the real-time GPS position information, the processor 502 specifically implements the following steps:

[0182] The real-time GPS location information is compared with a pre-constructed bridge GPS database to obtain a comparison result; it is determined whether the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-constructed bridge GPS database meets the set requirements; if the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-constructed bridge GPS database meets the set requirements, it is determined that the current vehicle and the bridge have an intersection; if the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-constructed bridge GPS database does not meet the set requirements, it is determined that the current vehicle and the bridge do not have an intersection.

[0183] The pre-built bridge GPS database is a database formed by collecting and recording specific geographical location information of the bridge.

[0184] In one embodiment, when the processor 502 performs image processing on the result output from the target segmentation model to obtain the target feature, the processor 502 specifically implements the following steps:

[0185] A mask image is generated based on the output result of the target segmentation model; a feature image of the bridge head jump area is cropped from the mask image; a minimum bounding rectangle of the feature image is calculated, and an interpolation method is used to fill the blank area in the minimum bounding rectangle, and the filled feature image is rotated to obtain a rotated regular rectangular feature image; the rotated regular rectangular feature image is converted into a feature vector of a fixed size to obtain the target feature.

[0186] In one embodiment, the processor 502 forms a vibration feature after implementing the feature processing of the vibration data, calculates the similarity with the target feature, predicts the probability of bridge head jump according to the target feature and the vibration feature, and comprehensively evaluates the probability of bridge head jump in combination with the similarity and the two probability values ​​and determines whether the bridge head jump occurs, and specifically implements the following steps:

[0187] The vibration data is feature processed to obtain a vibration feature; the target feature and the vibration feature are input into a regression model, and the similarity is calculated to obtain a calculation result; the target feature is input into an image classification model to predict the probability of a bridge head vehicle jump to obtain a first probability value; the vibration feature is input into a sensor classification model to predict the probability of a bridge head vehicle jump to obtain a second probability value; a final bridge head vehicle jump probability is comprehensively calculated based on the calculation result, the first probability value, and the second probability value, and it is determined whether a bridge head vehicle jump event occurs.

[0188] In one embodiment, when the processor 502 implements the step of performing feature processing on the vibration data to obtain vibration features, the processor 502 specifically implements the following steps:

[0189] The vibration data is subjected to matrix transposition to obtain vibration characteristics.

[0190] In one embodiment, when the processor 502 implements the step of inputting the target feature and the vibration feature into the regression model and calculating the similarity to obtain the calculation result, the processor 502 specifically implements the following steps:

[0191] The target feature and the vibration feature are input into a regression model, and the regression model is used to extract deep features of the target feature to obtain deep image features; the regression model is used to flatten the deep image feature and the vibration feature into one-dimensional vectors respectively, and the cosine similarity of the two one-dimensional vectors is calculated to obtain a calculation result.

[0192] In one embodiment, when the processor 502 implements the step of comprehensively calculating the final probability of a bridge head vehicle jump according to the calculation result, the first probability value, and the second probability value, and determining whether a bridge head vehicle jump event occurs, the processor 502 specifically implements the following steps:

[0193] The first probability value and the second probability value are weightedly summed, and the summed result is multiplied by the calculated result to obtain a final probability of vehicle jumping at a bridge head; the final probability of vehicle jumping at a bridge head is compared with a set occurrence threshold, and when the final probability of vehicle jumping at a bridge head is greater than the set occurrence threshold, it is determined that the vehicle has a vehicle jumping at a bridge head.

[0194] In one embodiment, when the processor 502 implements the step of performing weighted summation on the first probability value and the second probability value and multiplying the summation result by the calculation result to obtain a final probability of bridge head vehicle jumping, the processor 502 specifically implements the following steps:

[0195] use = 0.5×S×( + ) calculates the final probability of bridge jumping, where is the final probability of bridge head jumping; S is the calculation result; is the first probability value; is the second probability value.

[0196] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0197] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0198] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0199] The real-time GPS location information of the vehicle and the road surface image taken by the camera installed on the vehicle are obtained; whether the current vehicle and the bridge intersect is determined according to the real-time GPS location information; if the current vehicle and the bridge intersect, the road surface image is input into the target segmentation model to segment the potential area of ​​bridge head jump, so as to determine whether the road surface image has the potential area of ​​bridge head jump; when the road surface image has the potential area of ​​bridge head jump, image processing is performed on the result output by the target segmentation model to obtain the target feature; the target feature is continuously tracked, and vibration data collected by the vibration sensor installed on the vehicle is obtained; vibration features are formed after feature processing of the vibration data, and the similarity is calculated with the target feature, the probability of bridge head jump is predicted according to the target feature and the vibration feature respectively, and the probability of bridge head jump is comprehensively evaluated by combining the similarity and the two probability values ​​to determine whether the bridge head jump occurs.

[0200] In one embodiment, when the processor executes the computer program to implement the step of determining whether the current vehicle and the bridge intersect according to the real-time GPS position information, the processor specifically implements the following steps:

[0201] The real-time GPS location information is compared with a pre-constructed bridge GPS database to obtain a comparison result; it is determined whether the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-constructed bridge GPS database meets the set requirements; if the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-constructed bridge GPS database meets the set requirements, it is determined that the current vehicle and the bridge have an intersection; if the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-constructed bridge GPS database does not meet the set requirements, it is determined that the current vehicle and the bridge do not have an intersection.

[0202] The pre-built bridge GPS database is a database formed by collecting and recording specific geographical location information of the bridge.

[0203] In one embodiment, when the processor executes the computer program to implement the step of performing image processing on the result output from the target segmentation model to obtain target features, the processor specifically implements the following steps:

[0204] A mask image is generated based on the output result of the target segmentation model; a feature image of the bridge head jump area is cropped from the mask image; a minimum bounding rectangle of the feature image is calculated, and an interpolation method is used to fill the blank area in the minimum bounding rectangle, and the filled feature image is rotated to obtain a rotated regular rectangular feature image; the rotated regular rectangular feature image is converted into a feature vector of a fixed size to obtain the target feature.

[0205] In one embodiment, when the processor executes the computer program to implement the steps of performing feature processing on the vibration data to form a vibration feature, calculating similarity with a target feature, predicting the probability of a bridge head bump according to the target feature and the vibration feature, comprehensively evaluating the probability of a bridge head bump by combining the similarity and the two probability values, and determining whether a bridge head bump occurs, the following steps are specifically implemented:

[0206] The vibration data is feature processed to obtain a vibration feature; the target feature and the vibration feature are input into a regression model, and the similarity is calculated to obtain a calculation result; the target feature is input into an image classification model to predict the probability of a bridge head vehicle jump to obtain a first probability value; the vibration feature is input into a sensor classification model to predict the probability of a bridge head vehicle jump to obtain a second probability value; a final bridge head vehicle jump probability is comprehensively calculated based on the calculation result, the first probability value, and the second probability value, and it is determined whether a bridge head vehicle jump event occurs.

[0207] In one embodiment, when the processor executes the computer program to implement the step of performing feature processing on the vibration data to obtain vibration features, the processor specifically implements the following steps:

[0208] The vibration data is subjected to matrix transposition to obtain vibration characteristics.

[0209] In one embodiment, when the processor executes the computer program to implement the step of inputting the target feature and the vibration feature into a regression model, calculating the similarity, and obtaining a calculation result, the processor specifically implements the following steps:

[0210] The target feature and the vibration feature are input into a regression model, and the regression model is used to extract deep features of the target feature to obtain deep image features; the regression model is used to flatten the deep image feature and the vibration feature into one-dimensional vectors respectively, and the cosine similarity of the two one-dimensional vectors is calculated to obtain a calculation result.

[0211] In one embodiment, when the processor executes the computer program to implement the step of comprehensively calculating the final probability of a bridge head vehicle jump according to the calculation result, the first probability value, and the second probability value, and determining whether a bridge head vehicle jump event occurs, the processor specifically implements the following steps:

[0212] The first probability value and the second probability value are weightedly summed, and the summed result is multiplied by the calculated result to obtain a final probability of vehicle jumping at a bridge head; the final probability of vehicle jumping at a bridge head is compared with a set occurrence threshold, and when the final probability of vehicle jumping at a bridge head is greater than the set occurrence threshold, it is determined that the vehicle has a vehicle jumping at a bridge head.

[0213] In one embodiment, when the processor executes the computer program to implement the step of performing weighted summation on the first probability value and the second probability value, and multiplying the summation result by the calculation result to obtain a final probability of bridge head vehicle jumping, the processor specifically implements the following steps:

[0214] use = 0.5×S×( + ) calculates the final probability of bridge jumping, where is the final probability of bridge head jumping; S is the calculation result; is the first probability value; is the second probability value.

[0215] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0216] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0217] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0218] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the system of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0219] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0220] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A bridge head vehicle jumping detection method integrating multi-dimensional data, characterized in that: include: Obtain the real-time GPS location information of the vehicle and the road surface images taken by the camera installed on the vehicle; Determine whether the current vehicle and the bridge intersect according to the real-time GPS position information; If the current vehicle and the bridge intersect, the road image is input into the target segmentation model to segment the potential area of ​​bridge head jump, so as to determine whether the road image has the potential area of ​​bridge head jump; When there is a potential area for vehicle jumping at a bridge head in the road image, image processing is performed on the result output by the target segmentation model to obtain target features; Continuously tracking the target feature and acquiring vibration data collected by a vibration sensor installed on the vehicle; After feature processing is performed on the vibration data, a vibration feature is formed, and similarity is calculated with the target feature, the probability of bridge head vehicle jumping is predicted according to the target feature and the vibration feature respectively, and the probability of bridge head vehicle jumping is comprehensively evaluated by combining the similarity and the two probability values ​​to determine whether the bridge head vehicle jumping occurs; specifically, the vibration data is feature processed to obtain the vibration feature; Inputting the target feature and the vibration feature into a regression model, calculating the similarity, and obtaining a calculation result; The result output from the target segmentation model is subjected to image processing to obtain target features, including: Generating a mask image based on the output result of the target segmentation model; Cutting out a feature map of the bridge head jump area from the mask map; Calculating the minimum bounding rectangle of the feature map, filling the blank area in the minimum bounding rectangle using an interpolation method, and rotating the filled feature map to obtain a rotated regular rectangular feature map; The rotated rectangular feature map is converted into a feature vector of fixed size to obtain the target feature.

2. The bridge head vehicle bounce detection method integrating multi-dimensional data according to claim 1 is characterized in that: The determining whether the current vehicle and the bridge intersect according to the real-time GPS position information includes: Comparing the real-time GPS location information with a pre-built bridge GPS database to obtain a comparison result; Determining whether the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-built bridge GPS database meets the set requirements; If the comparison result is that the distance between the real-time GPS location information and the GPS location of any bridge in the pre-built bridge GPS database meets the set requirements, it is determined that the current vehicle and the bridge have an intersection; If the comparison result is that the distance between the real-time GPS position information and the GPS position of any bridge in the pre-built bridge GPS database does not meet the set requirements, it is determined that the current vehicle and the bridge have no intersection.

3. The bridge head vehicle jump detection method integrating multi-dimensional data according to claim 2 is characterized in that: The pre-constructed bridge GPS database is a database formed by collecting and recording specific geographical location information of the bridge.

4. The bridge head vehicle bounce detection method integrating multi-dimensional data according to claim 1 is characterized in that: The vibration data is processed to form a vibration feature, and similarity is calculated with the target feature, the probability of bridge head jump is predicted according to the target feature and the vibration feature, and the probability of bridge head jump is comprehensively evaluated by combining the similarity and the two probability values ​​to determine whether the bridge head jump occurs, including: Inputting the target feature into an image classification model to predict the probability of a vehicle jumping at a bridge head, so as to obtain a first probability value; Inputting the vibration characteristics into a sensor classification model to predict the probability of bridge head vehicle jumping to obtain a second probability value; The final probability of a bridge head vehicle jump is comprehensively calculated according to the calculation result, the first probability value and the second probability value, and it is determined whether a bridge head vehicle jump event occurs.

5. The bridge head vehicle bounce detection method integrating multi-dimensional data according to claim 4 is characterized in that: The feature processing of the vibration data to obtain vibration features includes: The vibration data is subjected to matrix transposition to obtain vibration characteristics.

6. The bridge head vehicle bounce detection method integrating multi-dimensional data according to claim 4 is characterized in that: The step of inputting the target feature and the vibration feature into a regression model and calculating the similarity to obtain a calculation result includes: Inputting the target feature and the vibration feature into a regression model, and using the regression model to perform deep feature extraction on the target feature to obtain a deep image feature; The deep image feature and the vibration feature are flattened into one-dimensional vectors respectively by the regression model, and the cosine similarity of the two one-dimensional vectors is calculated to obtain a calculation result.

7. The bridge head vehicle bounce detection method integrating multi-dimensional data according to claim 4 is characterized in that: The step of comprehensively calculating a final probability of a bridge head vehicle jump according to the calculation result, the first probability value, and the second probability value, and determining whether a bridge head vehicle jump event occurs, includes: Performing a weighted summation on the first probability value and the second probability value, and multiplying the summation result by the calculation result to obtain a final probability of bridge head vehicle jumping; The final bridge head vehicle jump occurrence probability is compared with a set occurrence threshold value, and when the final bridge head vehicle jump occurrence probability is greater than the set occurrence threshold value, it is determined that the vehicle has a bridge head vehicle jump.

8. The bridge head vehicle bounce detection method integrating multi-dimensional data according to claim 7 is characterized in that: The step of performing a weighted summation on the first probability value and the second probability value, and multiplying the summation result by the calculation result to obtain a final probability of vehicle jumping at a bridge head includes: use = 0.5×S×( + ) calculates the final probability of bridge jumping, where is the final probability of bridge head jumping; S is the calculation result; is the first probability value; is the second probability value.

9. The bridge head vehicle jumping detection system integrating multi-dimensional data is characterized by: include: An acquisition unit, used to acquire real-time GPS location information of the vehicle and road surface images taken by a camera installed on the vehicle; A judgment unit, used for judging whether the current vehicle and the bridge intersect according to the real-time GPS position information; The target segmentation unit is used for inputting the road surface image into the target segmentation model to segment the potential area of ​​bridge head jump if the current vehicle and the bridge intersect, so as to determine whether the road surface image has the potential area of ​​bridge head jump jump; An image processing unit, used for performing image processing on the result output by the target segmentation model to obtain target features when there is a potential area for vehicle jumping at a bridge head in the road image; A vibration data acquisition unit, used to continuously track the target feature and acquire vibration data collected by a vibration sensor installed on the vehicle; A probability calculation unit is used to perform feature processing on the vibration data to form a vibration feature, calculate a similarity with a target feature, predict the probability of a bridge head vehicle jump according to the target feature and the vibration feature, and comprehensively evaluate the probability of a bridge head vehicle jump by combining the similarity and the two probability values ​​to determine whether the bridge head vehicle jump occurs; The image processing unit comprises: A mask image generation subunit is used to generate a mask image based on the output result of the target segmentation model; a cropping subunit is used to crop a feature image of the bridge head jump area from the mask image; a rotation subunit is used to calculate the minimum circumscribed rectangle of the feature image, fill the blank area in the minimum circumscribed rectangle by using an interpolation method, and rotate the filled feature image to obtain a rotated regular rectangular feature image; a conversion subunit is used to convert the rotated regular rectangular feature image into a feature vector of a fixed size to obtain a target feature; The probability calculation unit comprises: The processing subunit is used to perform feature processing on the vibration data to obtain vibration features; the similarity calculation subunit is used to input the target features and the vibration features into a regression model, calculate the similarity, and obtain a calculation result.

Images