Guardrail deformation monitoring method, device and system based on millimeter wave radar and medium

Abstract

The invention discloses a guardrail deformation monitoring method, device and system based on a millimeter-wave radar, and a medium. The method comprises the following steps: obtaining a point cloud data set containing guardrails on two sides of a road collected by the millimeter-wave radar installed on the road at at least N moments within a time period; extracting data points representing the guardrail in the point cloud data set at each moment according to the speed and the reflection intensity, and superposing the data points representing the guardrail at each moment; determining a boundary of the guardrail point cloud set and dividing the guardrail point cloud set into a first guardrail point cloud set and a second guardrail point cloud set based on the boundary; matching data points in a second guardrail point cloud set for the data points in the first guardrail point cloud set based on Euclidean distance to obtain a point cloud pair sequence and a distance sequence corresponding to the point cloud pair sequence; and inputting the distance sequence into a trained deep learning model to realize guardrail deformation monitoring judgment. Data acquisition equipment does not need to be installed again, and the deformation judgment accuracy is high.

Description

Technical Field

[0001] The present invention belongs to the technical field of guardrail deformation monitoring, and in particular relates to a guardrail deformation monitoring method, device, system and medium based on millimeter wave radar. Background Art

[0002] Highway corrugated guardrails are important safety facilities used to prevent vehicles from running out of the lane when they lose control. However, due to various reasons, corrugated guardrails may deform, affecting their protective performance. Therefore, it is necessary to monitor corrugated guardrails in a timely manner to improve road operation safety.

[0003] There are many existing methods for monitoring guardrail deformation, such as installing strain sensors, acceleration sensors, vibration sensors, etc. on the guardrail, analyzing the strain, acceleration, vibration and other parameters of the guardrail through the data collected by the sensors, and then determining the degree of deformation and position of the guardrail; or using cameras or drones to collect image data, and extracting the features of the guardrail, such as edges, contours, curvatures, etc., by analyzing image processing algorithms, and then determining the degree of deformation and position of the guardrail.

[0004] When using sensors on the guardrail to detect guardrail deformation, it is necessary to deploy corresponding sensors. The installation and maintenance of the sensors are complicated and the operation is not convenient. When using image data to detect guardrail deformation, the image acquisition quality is directly related to the accuracy of deformation judgment, and the image acquisition quality is affected by factors such as lighting, weather, and occlusion. This method not only requires a large amount of image data and computing resources, but also the accuracy and robustness of the image processing algorithm need to be improved.

[0005] That is, the existing method cannot take into account both the convenience of operation and the accuracy of deformation judgment. Summary of the invention

[0006] In order to solve the problem that the existing guardrail deformation monitoring cannot take into account both the convenience of operation and the accuracy of deformation judgment, the present invention proposes a guardrail deformation monitoring method, device, system and medium based on millimeter wave radar.

[0007] The objective of the present invention is achieved through the following technical solutions:

[0008] The first aspect of the present invention discloses a guardrail deformation monitoring method based on millimeter wave radar, comprising the following steps:

[0009] Obtain a point cloud data set containing guardrails on both sides of the road collected by a millimeter-wave radar installed on the road at least N times within a period of time, where the point cloud data set at each time includes multiple data points, each data point includes speed and reflection intensity, and N is greater than or equal to 2;

[0010] According to the speed and reflection intensity, the data points representing the guardrail in the point cloud data set at each moment are extracted and the data points representing the guardrail at each moment are superimposed to obtain a guardrail point cloud set;

[0011] Determine a dividing line of the guardrail point cloud set and divide the guardrail point cloud set into a first guardrail point cloud set and a second guardrail point cloud set based on the dividing line, wherein the data points in the first guardrail point cloud set are data points corresponding to the guardrail on one side of the road, and the data points in the second guardrail point cloud set are data points corresponding to the guardrail on the other side of the road; match a data point in the first guardrail point cloud set with a data point in the second guardrail point cloud set based on the Euclidean distance to obtain a data point pair sequence and a distance sequence corresponding to the data point pair sequence;

[0012] The distance sequence is input into the trained deep learning model to realize guardrail deformation monitoring and judgment.

[0013] The second aspect of the present invention discloses a guardrail deformation monitoring device based on millimeter wave radar, comprising a memory and a controller which are communicatively connected in sequence, wherein a computer program is stored in the memory, and the controller is used to read the computer program and execute the guardrail deformation monitoring method based on millimeter wave radar described in the first aspect.

[0014] The third aspect of the present invention discloses a computer-readable storage medium, on which instructions are stored. When the instructions are run on a computer, the guardrail deformation monitoring method based on millimeter-wave radar described in the first aspect is executed.

[0015] The third aspect of the present invention discloses a guardrail deformation monitoring system based on millimeter wave radar, comprising:

[0016] Millimeter wave radar;

[0017] A guardrail deformation monitoring device based on millimeter wave radar as described in the second aspect, wherein the guardrail deformation monitoring device based on millimeter wave radar is connected to the millimeter wave radar signal;

[0018] A business platform, wherein the business platform is signal-connected to the guardrail deformation monitoring device based on the millimeter-wave radar, and an ANSYS Twin Builder modeling unit of a guardrail physical model is constructed in the business platform, wherein the guardrail physical model includes material properties, geometric dimensions, and boundary conditions, and the ANSYS Twin Builder modeling unit is used to map the guardrail point cloud set received from the guardrail deformation monitoring device based on the millimeter-wave radar to the corresponding position of the guardrail physical model and mark the area of ​​the guardrail deformation position according to the guardrail deformation monitoring judgment result.

[0019] The beneficial effects of the present invention are:

[0020] The present invention realizes guardrail deformation detection based on data of millimeter-wave radar installed on the road, does not require a new sensor, and can improve the accuracy of guardrail deformation judgment while satisfying operational convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 The figure is a flow chart of the guardrail deformation monitoring method based on millimeter wave radar of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0026] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0027] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the invention is usually placed when in use, or the positions or positional relationships commonly understood by those skilled in the art, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0028] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0029] Millimeter-wave radars on existing roads are mainly used to monitor vehicle driving information, including speed and position, to detect traffic events based on vehicle driving data, or to integrate with video data for traffic event monitoring, such as abnormal parking, congestion, etc. Based on the data of existing millimeter-wave radars that have been installed on roads, the present invention has developed a guardrail deformation monitoring method, device, system and medium based on millimeter-wave radars.

[0030] The first aspect of the present invention discloses a guardrail deformation monitoring method based on millimeter wave radar, which can be performed by, but is not limited to, a guardrail deformation monitoring device based on millimeter wave radar. The guardrail deformation monitoring device based on millimeter wave radar can be software, or a combination of software and hardware. The guardrail deformation monitoring device based on millimeter wave radar can be integrated in smart mobile terminals, tablets, computers and other smart devices. Specifically, Figure 1 As shown, the guardrail deformation monitoring method based on millimeter wave radar includes the following steps S01 to S05. It should be noted that the step identifiers in this scheme are only for the convenience of explaining the method and do not constitute a sequence limitation. The sequence of each step is based on its language description and the sequence of each signal.

[0031] Step S01, obtaining a point cloud data set including guardrails on both sides of the road collected by a millimeter-wave radar installed on the road at least N times within a time period, wherein the point cloud data set at each time includes multiple data points, each data point includes speed and reflection intensity, and N is greater than or equal to 2.

[0032] The N moments here may be continuous data collection points of the millimeter wave radar, periodic data collection points, or random data collection points, without any special restrictions.

[0033] Only the guardrail data points are extracted from the data collected by a single-frame millimeter-wave radar, and the point cloud is sparse, and the accuracy of guardrail deformation monitoring is low. In this regard, the guardrail point cloud volume can be enhanced based on the principle of multi-frame superposition. Here, N is greater than or equal to 2. However, since the radar is installed on the pole, it will shake due to strong winds or the passage of large vehicles, and the slight shaking will be amplified after the calculation of the azimuth angle and pitch angle, which will cause the overall data to shift. Therefore, if the number of superimposed frames is too large, the waveform guardrail in the image will be too wide. In this regard, in order to further improve the accuracy of the superimposed guardrail point cloud, N is greater than or equal to 2 and less than or equal to 20.

[0034] Step S02: extracting data points representing the guardrail from the point cloud data set at each moment according to the speed and reflection intensity, and superimposing the data points representing the guardrail at each moment to obtain a guardrail point cloud set.

[0035] Specifically, this step first deletes data points with a speed greater than 0 in the point cloud data set at each moment to obtain a static point cloud set; then deletes data points with a reflection intensity less than a first threshold from the static point cloud set to obtain data points representing the guardrail.

[0036] According to the speed data in the millimeter wave radar data, the data points of moving objects such as vehicles and pedestrians can be obtained. In static data, due to the difference in the material of the guardrail and the road surface, there will be differences in reflection intensity, and the road surface reflection intensity will be much lower than the guardrail reflection intensity. In this regard, the data points of the guardrail and other data points can be further distinguished and filtered out in the static point cloud set by the reflection intensity threshold. In general, the first threshold here can preferably be set to 20, of course, it can be adjusted according to the road conditions.

[0037] After superimposing multiple frames of guardrail data points, the guardrail point cloud becomes dense.

[0038] Step S03, determine the dividing line of the guardrail point cloud set and divide the guardrail point cloud set into a first guardrail point cloud set and a second guardrail point cloud set based on the dividing line, the data points in the first guardrail point cloud set are the data points corresponding to the guardrail on one side of the road, and the data points in the second guardrail point cloud set are the data points corresponding to the guardrail on the other side of the road.

[0039] Since millimeter-wave radars are generally installed in the middle of the road, such as in the green belt in the middle of the two-way lanes of a highway, or on one side of the road, the direction of the guardrail in the field of view in the data collected by the millimeter-wave radar is not fixed. In order to avoid misclassification of the guardrail point cloud, this step preferentially uses a multi-layer perceptron to determine the starting point coordinates (x1, y1) and the end point coordinates (x2, y2) of the calculated dividing line, and then divides the guardrail point cloud set into a first guardrail point cloud set and a second guardrail point cloud set according to the dividing line.

[0040] Specifically, (x1, y1), (x2, y2) = MLP(X),

[0041] Where MLP() is a multi-layer perceptron operation, and X is a guardrail point cloud set.

[0042] The use of multi-layer perceptrons requires less computation and consumes less resources than support vector machines and statistical methods, and is more efficient, especially when data points are densely populated.

[0043] The data points in the first guardrail point cloud set and the second guardrail point cloud set are respectively marked with a first label and a second label.

[0044] Step S04: Match a data point in the first guardrail point cloud set with a data point in the second guardrail point cloud set based on the Euclidean distance to obtain a point cloud pair sequence and a distance sequence corresponding to the point cloud pair sequence.

[0045] Specifically, a data point in the first guardrail point cloud set is matched with a data point in the second guardrail point cloud set based on the shortest Euclidean distance principle.

[0046] Step S05: input the distance sequence into the trained deep learning model to realize guardrail deformation monitoring and judgment.

[0047] In this step, the sliding window average features and guardrail local features of the distance sequence are first extracted, and then the sliding window average features, guardrail local features and distance sequence are fused to obtain guardrail deformation features. Finally, the guardrail deformation probability is determined based on the guardrail deformation features.

[0048] Specifically, the calculation method for extracting the sliding window average feature of the distance sequence is:

[0049]

[0050] Among them, d smooth (i) is the average feature of the sliding window, i.e., the distance of the i-th point cloud pair in the guardrail distance sequence after smoothing, w is the sliding window size, i is the data point index, j is the count index, and d(j) is the distance between the two data points in the j-th point cloud pair.

[0051] The local feature f of the guardrail is extracted using an LSTM network with adaptive length, namely:

[0052] f = LSTM k (D)

[0053] {h}=LSTM(D),

[0054] k=MLP({h}),

[0055] Among them, {h} is the set of all hidden states generated by the LSTM network, LSTM() is the LSTM network operation, k is the adaptive length, LSTM k () is the LSTM network operation with k units, D is the distance sequence, and MLP() is the multi-layer perceptron operation.

[0056] Long Short-Term Memory (LSTM) is a time-recurrent neural network. LSTM includes three gating structures: forget gate, input gate, and output gate, as well as a cell state, which can effectively control the flow and preservation of information, thereby capturing long-term dependencies in sequence data. In the detection of waveform guardrail deformation, the length of the guardrail distance sequence may vary due to factors such as different road sections and different traffic flows; the adaptive length LSTM network can dynamically adjust its internal structure, such as the number of LSTM units, according to the length of the input sequence, so as to better adapt to sequence data of different lengths. This dynamic adjustment helps the network to more accurately capture the key information in the sequence; in addition, since the LSTM network can learn long-term dependencies in sequence data, it can ignore the noise data points in the sequence to a certain extent. In the adaptive length LSTM network, by dynamically adjusting the network structure, the impact of noise data points on the network output can be further reduced; in addition, by combining the sliding window average feature extraction, the sequence data can be further smoothed and the interference of noise can be reduced.

[0057] When performing feature fusion on the sliding window average feature, guardrail local feature and distance sequence, a deep neural network is used, and the calculation method is:

[0058] F=MLP([D,d smooth (i),[{f},A]]),

[0059] Among them, F is the guardrail deformation feature, [] is the splicing operation; A is the positioning code used to match the corresponding local position, and {f} is the set of all guardrail local features.

[0060] When calculating the deformation characteristics of the guardrail, in order to achieve accurate positioning of the deformation position, the positioning code A is introduced, which is a coding sequence corresponding to the guardrail distance sequence, in which each coding element represents a specific position on the guardrail; the positioning code A can be generated according to the position of the guardrail column, the interval of the corrugated beam and other information; each coding element is a unique identifier or coordinate value used to represent a specific position on the guardrail.

[0061] The probability p1 of guardrail deformation is:

[0062] p1=sigmoid(F),

[0063] sigmoid() is the sigmoid function operation.

[0064] The probability p2 of deformation in each part of the guardrail is:

[0065] p2 = softmax(MLP([F,A]),

[0066] softmax() is the softmax function operation.

[0067] The deformation position of the guardrail is located through positioning code A, which provides an important basis for subsequent maintenance and repair work.

[0068] The second aspect of the present invention discloses a guardrail deformation monitoring device based on millimeter wave radar, including a memory and a controller connected in sequence, wherein a computer program is stored on the memory, and the controller is used to read the computer program and execute the guardrail deformation monitoring method based on millimeter wave radar described in the first aspect. For example, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first input first output (FIFO) and or first input last output (FILO), etc.; the controller may not be limited to a microcontroller of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power supply unit, a display screen and other necessary components.

[0069] The third aspect of the present invention discloses a computer-readable storage medium, on which instructions are stored. When the instructions are run on a computer, the guardrail deformation monitoring method based on millimeter-wave radar described in the first aspect is executed.

[0070] The operating principles of the devices and media disclosed in the second and third aspects of the present invention are detailed in the first aspect and will not be elaborated here.

[0071] The fourth aspect of the present invention discloses a guardrail deformation monitoring system based on millimeter wave radar, which includes a millimeter wave radar, the guardrail deformation monitoring device based on millimeter wave radar in the second aspect, and a business platform, wherein the millimeter wave radar, the guardrail deformation monitoring device based on millimeter wave radar, and the business platform are signal-connected in sequence.

[0072] An ANSYS Twin Builder modeling unit of a guardrail physical model is constructed in the business platform. The guardrail physical model includes material properties, geometric dimensions, and boundary conditions. The ANSYS Twin Builder modeling unit is used to map the guardrail point cloud set received from the millimeter-wave radar-based guardrail deformation monitoring device to the corresponding position of the guardrail physical model and mark the area of ​​the guardrail deformation position according to the guardrail deformation monitoring judgment result.

[0073] In order to enhance the deformation prompt function, the ANSYS Twin Builder modeling unit is also used to highlight the local position of the guardrail in red when the probability of deformation of the guardrail is greater than or equal to the second threshold, so that users can intuitively identify the area where deformation exists. The second threshold can be set according to the situation, for example, it can be set to 80% to 90%.

[0074] Specifically, a data display module can also be developed on the business platform to display the status of the corrugated guardrail in a three-dimensional visual form.

[0075] By adopting the above solution, there is no need to reinstall sensors or data acquisition devices. The existing installed millimeter-wave radar can be directly used to monitor the guardrail deformation, thereby reducing the monitoring cost.

[0076] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A guardrail deformation monitoring method based on millimeter wave radar, characterized in that: The following steps are involved: Obtain a point cloud data set containing guardrails on both sides of the road collected by a millimeter-wave radar installed on the road at least N times within a period of time, where the point cloud data set at each time includes multiple data points, each data point includes speed and reflection intensity, and N is greater than or equal to 2; According to the speed and reflection intensity, the data points representing the guardrail in the point cloud data set at each moment are extracted and the data points representing the guardrail at each moment are superimposed to obtain a guardrail point cloud set; Determine a dividing line of the guardrail point cloud set and divide the guardrail point cloud set into a first guardrail point cloud set and a second guardrail point cloud set based on the dividing line, wherein the data points in the first guardrail point cloud set are data points corresponding to the guardrail on one side of the road, and the data points in the second guardrail point cloud set are data points corresponding to the guardrail on the other side of the road; match a data point in the first guardrail point cloud set with a data point in the second guardrail point cloud set based on the Euclidean distance to obtain a data point pair sequence and a distance sequence corresponding to the data point pair sequence; The distance sequence is input into the trained deep learning model to realize guardrail deformation monitoring and judgment.

2. The method for monitoring guardrail deformation based on millimeter wave radar according to claim 1, characterized in that: The N is greater than or equal to 2 and less than or equal to 20.

3. The method for monitoring guardrail deformation based on millimeter wave radar according to claim 1, characterized in that: The method of extracting data points representing the guardrail in the point cloud data set at each time according to the speed and reflection intensity includes: Delete the data points with speed greater than 0 in the point cloud data set at each moment to obtain a static point cloud set; Data points whose reflection intensity is less than a first threshold are deleted from the static point cloud set to obtain data points representing the guardrail.

4. The method for monitoring guardrail deformation based on millimeter wave radar according to claim 1, characterized in that: The step of determining a boundary line of the guardrail point cloud set and dividing the guardrail point cloud set into a first guardrail point cloud set and a second guardrail point cloud set based on the boundary line comprises: The multi-layer perceptron is used to calculate the starting point coordinates (x1, y1) and the end point coordinates (x2, y2) of the dividing line, where: (x1,y1),(x2,y2)=MLP(X), Where MLP() is a multi-layer perceptron operation, X is a guardrail point cloud set; The guardrail point cloud set is divided into a first guardrail point cloud set and a second guardrail point cloud set according to the dividing line.

5. The method for monitoring guardrail deformation based on millimeter wave radar according to claim 4 is characterized in that: The step of dividing the guardrail point cloud set into a first guardrail point cloud set and a second guardrail point cloud set according to the dividing line further includes: The data points in the first guardrail point cloud set and the second guardrail point cloud set are respectively marked with a first label and a second label.

6. The method for monitoring guardrail deformation based on millimeter wave radar according to claim 1, characterized in that: Inputting the distance sequence into the trained deep learning model to implement guardrail deformation monitoring and judgment includes: Extract the sliding window average features of the distance sequence, Among them, d smooth (i) is the average feature of the sliding window, i.e., the distance of the i-th data point in the guardrail distance sequence after smoothing, w is the sliding window size, i is the data point index, j is the count index, and d(j) is the distance between the two data points in the j-th pair of data points; Adopting the LSTM network with adaptive length, extracting the local features f of the guardrail, f=LSTM k (D), {h}=LSTM(D), k=MLP({h}), Among them, {h} is the set of all hidden states generated by the LSTM network, LSTM() is the LSTM network operation, k is the adaptive length, LSTM k () is the LSTM network operation with k units, D is the distance sequence, and MLP() is the multi-layer perceptron operation; The distance sequence, sliding window average features and guardrail local features are fused to obtain guardrail deformation features. The guardrail deformation probability is determined based on the guardrail deformation characteristics.

7. A guardrail deformation monitoring device based on millimeter wave radar, characterized in that: It comprises a memory and a controller which are communicatively connected in sequence, wherein a computer program is stored in the memory, and the controller is used to read the computer program and execute a guardrail deformation monitoring method based on millimeter wave radar as described in any one of claims 1-6.

8. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed on a computer, a guardrail deformation monitoring method based on millimeter-wave radar as described in any one of claims 1 to 6 is executed.

9. A guardrail deformation monitoring system based on millimeter wave radar, characterized in that: include: Millimeter wave radar; A guardrail deformation monitoring device based on millimeter wave radar as described in claim 7, wherein the guardrail deformation monitoring device based on millimeter wave radar is connected to the millimeter wave radar signal; A business platform, wherein the business platform is signal-connected to the guardrail deformation monitoring device based on the millimeter-wave radar, and an ANSYS Twin Builder modeling unit of a guardrail physical model is constructed in the business platform, wherein the guardrail physical model includes material properties, geometric dimensions, and boundary conditions, and the ANSYS Twin Builder modeling unit is used to map the guardrail point cloud set received from the guardrail deformation monitoring device based on the millimeter-wave radar to the corresponding position of the guardrail physical model and mark the area of ​​the guardrail deformation position according to the guardrail deformation monitoring judgment result.

10. The guardrail deformation monitoring system based on millimeter wave radar according to claim 9, characterized in that: The ANSYS Twin Builder modeling unit is also used to highlight the local position of the guardrail in red when the probability of local deformation of the guardrail is greater than or equal to a second threshold.

Images