Artificial intelligence-based highway traffic safety facility technical condition detection method and system

Abstract

The application provides a kind of highway traffic safety facilities technical condition detection method and system based on artificial intelligence, by collecting the continuous detection image sequence of highway traffic safety facilities in different time periods, the facility feature evolution model containing time sequence feature tracking module and spatial feature correlation module is constructed, the change process and spatial dependence of facility feature are modeled.The continuous detection image sequence is input into the model, the change parameter and correlation parameter of facility feature are extracted, and the facility feature coupling set is obtained by executing dynamic feature coupling processing.Based on the comparison between facility feature coupling set and preset normal feature evolution benchmark, the abnormal evolution track of facility feature is mined, and the facility technical condition detection report containing abnormal development trend description is generated, and finally transmitted to the target management terminal.The application realizes accurate detection and dynamic monitoring of the technical condition of highway traffic safety facilities, effectively improves the management level of highway traffic safety facilities.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to a method and system for detecting the technical condition of highway traffic safety facilities based on artificial intelligence. Background Technology

[0002] In the field of highway transportation, traffic safety facilities are crucial for ensuring the safety of vehicles and pedestrians. These facilities include, but are not limited to, traffic signs, guardrails, and road markings, and their technical condition directly affects the safety and smoothness of road traffic. Currently, the inspection of the technical condition of highway traffic safety facilities mainly relies on manual inspection and traditional image recognition technology. Manual inspection is not only inefficient but also easily affected by human factors, leading to inaccurate results and difficulty in detecting subtle changes in facilities. While traditional image recognition technology improves detection efficiency to some extent, it mostly only performs static analysis on single frames of images, failing to capture the changing process of facility features over different time periods or consider the spatial dependencies between features of different parts of the facility. For example, for detecting the fading of traffic signs, traditional methods may only identify the degree of fading at the current moment but cannot know its fading rate or the mutual influence of fading in different parts, making it difficult to accurately judge the abnormal development trend of the facility. Therefore, existing technologies are insufficient to meet the needs for comprehensive, accurate, and dynamic inspection of the technical condition of highway traffic safety facilities. Summary of the Invention

[0003] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide an artificial intelligence-based method for detecting the technical condition of highway traffic safety facilities, the method comprising: Collect a continuous image sequence of highway traffic safety facilities at different time periods, wherein the continuous image sequence contains multiple frames of the same facility under different environmental conditions; A facility feature evolution model is constructed, which includes a temporal feature tracking module and a spatial feature association module. The temporal feature tracking module models the change process of facility features in a continuous detection image sequence, and the spatial feature association module models the spatial dependency relationship of features in different parts of the facility. The continuous detection image sequence is input into the facility feature evolution model. The temporal feature tracking module extracts the change parameters of facility features at different time periods, and the spatial feature association module extracts the association parameters of features of different parts of the facility. The dynamic feature coupling processing is performed by combining the change parameters and the association parameters to obtain the facility feature coupling set. The abnormal evolution trajectory of facility features is mined based on the facility feature coupling set, which is obtained by comparing the facility feature coupling set with a preset normal feature evolution benchmark. Based on the abnormal evolution trajectory, a facility technical condition inspection report containing a description of the abnormal development trend is generated, and the facility technical condition inspection report is transmitted to the target management terminal.

[0004] In another aspect, embodiments of the present invention also provide an artificial intelligence-based highway traffic safety facility technical condition detection system, including a processor and a machine-readable storage medium connected to the processor. The machine-readable storage medium is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the machine-readable storage medium to implement the above-described method.

[0005] Based on the above, this embodiment of the invention collects continuous detection image sequences of highway traffic safety facilities at different time periods, covering multiple frames of images of the facilities under different environmental conditions. A facility feature evolution model is constructed, including a temporal feature tracking module and a spatial feature association module, to deeply model the facility features from both temporal and spatial dimensions. The temporal feature tracking module can accurately capture the change process of facility features at different time periods, reflecting the gradual evolution of facility features; the spatial feature association module fully considers the spatial dependencies of features in different parts of the facility, reflecting the degree of mutual influence of features in each part. After inputting the continuous detection image sequence into the model, dynamic feature coupling processing is performed by combining the changing parameters and association parameters to obtain a facility feature coupling set, realizing the organic integration of spatiotemporal features. Based on this set, the abnormal evolution trajectory of facility features is mined, which can accurately identify abnormal conditions of the facilities and generate a facility technical condition inspection report containing a description of the abnormal development trend based on the abnormal evolution trajectory. This provides a scientific and accurate decision-making basis for the maintenance and management of highway traffic safety facilities, effectively improving the comprehensiveness, accuracy, and dynamism of highway traffic safety facility inspection, and ensuring the safety and smooth flow of road traffic. Attached Figure Description

[0006] Figure 1 This is a schematic diagram of the execution flow of the artificial intelligence-based highway traffic safety facility technical condition detection method provided in the embodiments of the present invention.

[0007] Figure 2 This is a schematic diagram of exemplary hardware and software components of the artificial intelligence-based highway traffic safety facility technical condition detection system provided in an embodiment of the present invention. Detailed Implementation

[0008] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating an artificial intelligence-based method for detecting the technical condition of highway traffic safety facilities, provided in one embodiment of the present invention. The following is a detailed description of this artificial intelligence-based method for detecting the technical condition of highway traffic safety facilities.

[0009] Step S110: Collect a continuous detection image sequence of highway traffic safety facilities at different time periods. The continuous detection image sequence contains multiple frames of the same facility under different environmental conditions.

[0010] In this embodiment, a corrugated beam guardrail is used as an example of a highway traffic safety facility. This guardrail is located on a section of a highway and its structure includes corrugated beams, posts, anti-blocking blocks, end caps, bolts, etc. To collect continuous image sequences of the corrugated beam guardrail at different times, high-definition industrial cameras mounted on both sides of the highway are used for image acquisition. The cameras are installed at the same height as the top of the guardrail, and at an angle perpendicular to the extension direction of the guardrail, ensuring that each frame completely covers a 20-meter-long section of the guardrail. The acquisition periods include four different times: early morning, noon, evening, and night. Each time period lasts for 30 minutes, with one frame acquired every second. 1800 frames are acquired in each time period, for a total of 7200 frames across the four time periods. The environmental conditions during the early morning are cloudy with weak light and no obvious shadows; the environmental conditions during midday are sunny with strong light and clear shadows; the environmental conditions during the evening are partly cloudy turning sunny with gradually decreasing light and varying shadow lengths; the environmental conditions during the night are cloudless, relying on streetlights along the roadside for illumination, with uniform light intensity and no natural shadows. During image acquisition, to protect potentially sensitive privacy data (such as vehicle license plate information), privacy protection processing is performed on each frame of the acquired image. Specifically, an image blurring algorithm is used to Gaussian blur the license plate area in the image, with the blur radius set at the pixel level to ensure that the license plate information cannot be identified, while ensuring that the features of the corrugated beam guardrail are not blurred and thus do not affect subsequent detection. The acquired continuous detection image sequences are classified and stored according to the acquisition time period. Each time period image sequence is named with a timestamp, which includes the acquisition year, month, date, hour, minute, and second. For example, the first frame of the early morning time period is named "20240520060000", which means that the image was acquired at 6:00:00 AM on May 20, 2024.

[0011] Step S120: Construct a facility feature evolution model. The facility feature evolution model includes a temporal feature tracking module and a spatial feature association module. The temporal feature tracking module models the change process of facility features in a continuous detection image sequence, and the spatial feature association module models the spatial dependency relationship of features in different parts of the facility.

[0012] In this embodiment, the facility feature evolution model is constructed based on a deep learning framework. The input of the model is the continuous detection image sequence acquired in step S110, and the output is a facility feature coupling set that integrates temporal variation parameters and spatial correlation parameters.

[0013] Step S121: Determine the input dimension of the facility feature evolution model, matching the image resolution and number of frames of the continuous detection image sequence.

[0014] In this embodiment, the resolution of each detection image frame acquired in step S110 is 2,000 pixels wide and 1,000 pixels high, therefore the number of pixels in each frame is 2 million pixels. The total number of frames in the continuous detection image sequence is 7,200 frames, therefore the input dimension of the facility feature evolution model is set to 7,200 x 2,000 x 1,000 x 3, where "3" represents the three RGB color channels of the image. Determining the input dimension ensures that the model can completely receive all pixel information in the continuous detection image sequence.

[0015] Step S122: Construct the network structure of the temporal feature tracking module. The temporal feature tracking module contains multiple layers of recurrent processing units. The input of the recurrent processing unit is the facility feature parameters of the previous frame image, and the output is the difference in facility feature changes between the current frame and the previous frame image.

[0016] In this embodiment, the network structure of the temporal feature tracking module is built based on a recurrent neural network framework. Its core function is to model the temporal change process of the waveform beam guardrail features in a continuously detected image sequence.

[0017] Step S1221: Determine the number of loop processing unit layers in the temporal feature tracking module. The number of loop processing unit layers corresponds to the number of frames in the continuously detected image sequence.

[0018] In this embodiment, the total number of frames in the continuously detected image sequence is 7,200. Therefore, the number of layers in the loop processing unit is set to 7,199. Each loop processing unit is responsible for processing the feature change difference between two adjacent frames in the continuously detected image sequence.

[0019] Step S1222: Construct the first layer of loop processing unit. The input of the first layer of loop processing unit is the facility feature parameters of the first frame image in the continuous detection image sequence. The facility feature change difference between the first frame and the second frame image is generated through feature difference operation.

[0020] In this embodiment, the first frame image is the first frame image captured in the early morning. Its facility feature parameters include the grayscale feature parameters of the corrugated beam slab, the edge feature parameters of the column, and the shape feature parameters of the anti-blocking block, etc. Each feature parameter is in the form of a multi-dimensional vector. For example, the grayscale feature parameter of the corrugated beam slab is a vector of length 2,000, and each element in the vector corresponds to the grayscale value of the corrugated beam slab at different positions in the image. After receiving the facility feature parameters of the first frame image as input, the first layer loop processing unit stores them in the feature memory unit. Then, it receives the facility feature parameters of the second frame image and performs element-wise subtraction on the facility feature parameters of the first and second frames images through feature difference operation to generate the facility feature change difference value between the first and second frames. This change difference value is also in the form of a multi-dimensional vector, and each element in the vector corresponds to the change in the same feature position in the two adjacent frames.

[0021] Step S1223: Construct a second-layer loop processing unit. The input of the second-layer loop processing unit is the facility feature change difference output by the first-layer loop processing unit and the facility feature parameters of the third frame image. The facility feature change difference between the second frame and the third frame image is generated through feature superposition operation.

[0022] In this embodiment, the input of the second-layer loop processing unit first receives the facility feature change difference output by the first-layer loop processing unit, and then superimposes it with the facility feature parameters of the second frame image stored in the feature memory unit to obtain the superimposed feature parameters. Subsequently, it receives the facility feature parameters of the third frame image, and performs element-wise subtraction between the superimposed feature parameters and the facility feature parameters of the third frame image through feature difference operation to generate the facility feature change difference between the second and third frames.

[0023] Step S1224: Construct the remaining loop processing units in sequence according to the method of constructing the second layer of loop processing units. The input of each of the remaining loop processing units includes the facility feature change difference output by the previous loop processing unit and the facility feature parameters of the corresponding frame image.

[0024] In this embodiment, the input of the third-layer loop processing unit is the facility feature change difference output by the second-layer loop processing unit and the facility feature parameters of the fourth frame image. The facility feature change difference between the third and fourth frames is generated through feature superposition and feature difference operations. The input of the fourth-layer loop processing unit is the facility feature change difference output by the third-layer loop processing unit and the facility feature parameters of the fifth frame image. The facility feature change difference between the fourth and fifth frames is generated through the same operation. This process continues until the 7,199th loop processing unit is completed.

[0025] Step S1225: Configure a feature memory unit for the loop processing unit. The feature memory unit is used to store the facility feature parameters of the previous frame image for the current loop processing unit to perform feature comparison calculations.

[0026] In this embodiment, the feature memory unit adopts a dynamic storage structure, which can update the stored facility feature parameters in real time. When the current loop processing unit receives the facility feature change difference output by the previous loop processing unit and the facility feature parameters of the corresponding frame image, the feature memory unit replaces the facility feature parameters of the previous frame image with the facility feature parameters of the current frame image.

[0027] Step S1226: Set a difference integration unit at the output of all loop processing units. The difference integration unit is used to arrange the facility characteristic change difference output by each loop processing unit in time order to form a time-seriesd change parameter sequence.

[0028] In this embodiment, the difference integration unit receives the facility feature change difference values ​​output by all 7,199 layers of the loop processing unit, and arranges these change difference values ​​according to the layer number order of the loop processing unit (i.e., from the first layer to the 7,199th layer) to form a time-series change parameter sequence of length 7,199. Each element in this sequence corresponds to the facility feature change difference value between two adjacent frames in the continuously detected image sequence.

[0029] Step S123: Configure the cyclic update parameters of the temporal feature tracking module. The cyclic update parameters are used to control the feature transfer weights between multi-layer cyclic processing units, so that the feature change difference can reflect the gradual change of facility features in continuous frame images.

[0030] In this embodiment, the cyclic update parameters include a weighting coefficient and a bias coefficient. The weighting coefficient is used to adjust the influence of the facility feature change difference output by the previous cyclic processing unit in the current cyclic processing unit, and the bias coefficient is used to adjust the basic influence of the facility feature parameters of the current frame image in the current cyclic processing unit. The weighting coefficient ranges from zero to one, and the bias coefficient ranges from zero to 0.5. For the early morning cyclic processing unit, due to weak light intensity and no obvious shadows, the changes in facility features are relatively slow; therefore, the weighting coefficient is set to a larger value, and the bias coefficient is set to a smaller value. For the midday cyclic processing unit, due to strong light intensity and clear shadows, the changes in facility features are more obvious; therefore, the weighting coefficient is set to a smaller value, and the bias coefficient is set to a larger value. For the evening cyclic processing unit, due to gradually weakening light intensity and gradually changing shadow length, the changes in facility features are relatively stable; therefore, both the weighting coefficient and the bias coefficient are set to medium values. For the nighttime cyclic processing unit, due to uniform light intensity and no natural shadows, the changes in facility features are relatively slow; therefore, the weighting coefficient is set to a larger value, and the bias coefficient is set to a smaller value. By configuring the cyclic update parameters, it is ensured that the feature transfer between multiple cyclic processing units can accurately reflect the gradual changes in the wave-beam guardrail features in continuous frame images.

[0031] Step S124: Construct the network structure of the spatial feature association module. The spatial feature association module contains multiple sets of feature interaction units. Each feature interaction unit corresponds to a feature of a facility. Association transmission channels are set between different feature interaction units.

[0032] In this embodiment, the network structure of the spatial feature association module is built based on a graph neural network framework, and its core function is to model the spatial dependency relationship of features in different parts of the corrugated beam guardrail.

[0033] Step S1241: Classify the parts of highway traffic safety facilities, determine the number of parts based on the structural composition of the facilities, and each part type corresponds to a set of feature interaction units.

[0034] In this embodiment, based on the structural composition of the corrugated beam guardrail, its parts are divided into five types: corrugated beam plate, post, anti-blocking block, end, and bolt. Therefore, there are five parts types, and five sets of feature interaction units are set accordingly, namely, corrugated beam plate feature interaction unit, post feature interaction unit, anti-blocking block feature interaction unit, end feature interaction unit, and bolt feature interaction unit.

[0035] Step S1242: Construct the internal processing structure of the feature interaction unit. The feature interaction unit includes a part feature extraction subunit and a feature transformation subunit. The part feature extraction subunit is used to extract the original features of the corresponding part from the detection image. The feature transformation subunit is used to convert the original features into standardized part feature parameters.

[0036] In this embodiment, taking the corrugated beam plate feature interaction unit as an example, its internal processing structure includes a corrugated beam plate feature extraction subunit and a corrugated beam plate feature transformation subunit. The corrugated beam plate feature extraction subunit uses a convolutional neural network to process the detected image, extracting the original features of the corrugated beam plate through multi-layer convolution operations. These original features include edge features, texture features, and shape features, each of which is a multi-dimensional feature map. The corrugated beam plate feature transformation subunit uses a standardization method to transform the original features, uniformly converting the value range of the original features to zero to one, generating standardized corrugated beam plate feature parameters, which are in multi-dimensional vector form. The internal processing structures of the column feature interaction unit, the anti-blocking block feature interaction unit, the end feature interaction unit, and the bolt feature interaction unit are the same as those of the corrugated beam plate feature interaction unit, respectively extracting the original features of their corresponding parts and converting them into standardized part feature parameters.

[0037] Step S1243: Set up association transmission channels between different feature interaction units. The association transmission channel corresponds to the information transmission path between two feature interaction units. The number of association transmission channels matches the number of combinations of part types.

[0038] In this embodiment, there are five part types, resulting in ten possible combinations of part types. This corresponds to ten associated transmission channels: the transmission channel between the corrugated beam and the column, the transmission channel between the corrugated beam and the anti-blocking block, the transmission channel between the corrugated beam and the end, the transmission channel between the corrugated beam and the bolt, the transmission channel between the column and the anti-blocking block, the transmission channel between the column and the end, the transmission channel between the column and the bolt, the transmission channel between the anti-blocking block and the end, the transmission channel between the anti-blocking block and the bolt, and the transmission channel between the end and the bolt. Each associated transmission channel is a unidirectional information transmission path, with the information transmission direction from one feature interaction unit to another.

[0039] Step S1244: Configure an information filtering subunit for the association transmission channel. The information filtering subunit is used to filter the feature information transmitted in the association transmission channel and retain information related to the feature association between the two parts.

[0040] In this embodiment, taking the association transmission channel between the corrugated beam and the column as an example, its information filtering subunit uses an attention mechanism to filter the transmitted feature information. The standardized feature parameters of the corrugated beam section output by the corrugated beam feature interaction unit and the standardized feature parameters of the column section output by the column feature interaction unit are input to the information filtering subunit. The information filtering subunit calculates the attention weight between the feature parameters of the corrugated beam section and the feature parameters of the column section. The attention weight ranges from zero to one; a larger value indicates a higher correlation between the feature information and the features of the two sections. The information filtering subunit filters out the feature information with high correlation based on the attention weight, retains this feature information, and transmits it to the next feature interaction unit, while filtering out the feature information with low correlation. The information filtering subunits of other association transmission channels all use the same attention mechanism for information filtering.

[0041] Step S1245: Set up an association parameter integration unit at the output end of the spatial feature association module. The association parameter integration unit is used to collect the association information output by all association transmission channels and generate an association parameter set containing the degree of association of features of each part.

[0042] In this embodiment, the association parameter integration unit receives association information output from ten association transmission channels. Each piece of association information is in the form of a multi-dimensional vector, and each element in the vector corresponds to the degree of association between two part features. The association parameter integration unit concatenates these association information to generate an association parameter set containing the degree of association of all part features. This set is in the form of a multi-dimensional vector, and the length of the vector is the sum of the vector lengths of the association information output from the ten association transmission channels.

[0043] Step S1246: Configure a feature weight allocation subunit for the associated parameter integration unit. The feature weight allocation subunit is used to assign corresponding weight values ​​to each associated parameter in the associated parameter set according to the importance of different parts of the facility.

[0044] In this embodiment, based on the importance of different parts of the corrugated beam guardrail, the corrugated beam plate is the most important, followed by the posts, anti-blocking blocks, and end caps, with bolts being the least important. Therefore, the feature weight allocation subunit assigns higher weight values ​​to the associated parameters related to the corrugated beam plate, the second highest weight values ​​to the associated parameters related to the posts, medium weight values ​​to the associated parameters related to the anti-blocking blocks, lower weight values ​​to the associated parameters related to the end caps, and the lowest weight values ​​to the associated parameters related to the bolts. The weight values ​​range from zero to one; the weight value for the associated parameters related to the corrugated beam plate is set to 0.8, the weight value for the associated parameters related to the posts is set to 0.7, the weight value for the associated parameters related to the anti-blocking blocks is set to 0.6, the weight value for the associated parameters related to the end caps is set to 0.5, and the weight value for the associated parameters related to the bolts is set to 0.4. By configuring the feature weight allocation subunit, it is ensured that the weight of each associated parameter in the associated parameter set accurately reflects the importance of different parts of the facility.

[0045] Step S125: Configure the correlation coefficient of the spatial feature association module. The correlation coefficient is used to define the information transmission strength between different feature interaction units, so that the association parameters can reflect the degree of mutual influence of features in different parts of the facility.

[0046] In this embodiment, the correlation coefficient ranges from zero to one. A larger value indicates a higher information transmission strength between the two feature interaction units and a greater degree of mutual influence between the two feature components. For the correlation transmission channel between the corrugated beam plate and the column, since the corrugated beam plate and the column are the main load-bearing structures of the corrugated beam guardrail, the degree of mutual influence between them is relatively large, so the correlation coefficient is set to a larger value. For the correlation transmission channel between the corrugated beam plate and the bolt, since the bolt is an auxiliary structure connecting the corrugated beam plate and the column, the degree of mutual influence between them is relatively small, so the correlation coefficient is set to a smaller value. For the correlation transmission channel between the column and the anti-blocking block, since the anti-blocking block is a transition structure connecting the column and the corrugated beam plate, the degree of mutual influence between them is moderate, so the correlation coefficient is set to a moderate value. For the correlation transmission channel between the anti-blocking block and the end, since the end is the end structure of the corrugated beam guardrail, the degree of mutual influence between them is relatively small, so the correlation coefficient is set to a smaller value. For the correlation transmission channel between the end and the bolt, since the bolt is an auxiliary structure connecting the end and other parts, the degree of mutual influence between them is relatively small, so the correlation coefficient is set to a smaller value. By configuring correlation coefficients, it is ensured that the information transmission strength between different feature interaction units can accurately reflect the degree of mutual influence between the features of different parts of the facility.

[0047] Step S126: Connect the temporal feature tracking module and the spatial feature association module through the feature fusion interface. The feature fusion interface is used to unify the format of the changing parameters and the associated parameters, and complete the construction of the facility feature evolution model.

[0048] In this embodiment, the feature fusion interface uses tensor concatenation to unify the format of the changing parameters and the associated parameters. The temporal changing parameter sequence output by the temporal feature tracking module is in multi-dimensional tensor form, with a dimension of 7,199 multiplied by the vector length of the feature change difference; the associated parameter set output by the spatial feature association module is also in multi-dimensional tensor form, with a dimension equal to the vector length of the associated parameter set. The feature fusion interface concatenates the tensor of the temporal changing parameter sequence and the tensor of the associated parameter set. The concatenated tensor has a dimension of 7,199 multiplied by (the vector length of the feature change difference plus the vector length of the associated parameter set), thus unifying the format of the changing parameters and the associated parameters. By connecting the temporal feature tracking module and the spatial feature association module through the feature fusion interface, the construction of the facility feature evolution model is completed.

[0049] Step S130: Input the continuous detection image sequence into the facility feature evolution model, extract the change parameters of facility features at different time periods through the temporal feature tracking module, extract the correlation parameters of features of different parts of the facility through the spatial feature association module, and perform dynamic feature coupling processing by combining the change parameters and the correlation parameters to obtain the facility feature coupling set.

[0050] In this embodiment, the continuous detection image sequence is input into the facility feature evolution model in the order of the acquisition time period (i.e., early morning, noon, evening, and night). The model first preprocesses the continuous detection image sequence, including image denoising, image enhancement, and image normalization, to ensure that the quality and format of the input image meet the requirements of the model.

[0051] Step S131: Input the continuous detection image sequence into the temporal feature tracking module of the facility feature evolution model in chronological order, and process each frame of image sequentially through a multi-layer loop processing unit to generate a sequence of changing parameters of facility features at different time periods.

[0052] In this embodiment, the continuously detected image sequence is sequentially input into the multi-layered loop processing unit of the temporal feature tracking module. The first-layer loop processing unit processes the first and second frames of images, generating facility feature change differences between the first and second frames. The second-layer loop processing unit processes the second and third frames of images, generating facility feature change differences between the second and third frames. This process continues until the 7,199th layer loop processing unit processes the 7,199th and 7,200th frames of images, generating facility feature change differences between the 7,199th and 7,200th frames. The difference integration unit arranges these facility feature change differences in chronological order to form a temporally sequenced sequence of change parameters, which includes facility feature change parameters for four time periods: morning, noon, evening, and night.

[0053] Step S132: Input the facility location features of each frame in the continuously detected image sequence into the spatial feature association module. Through multiple sets of feature interaction units and association transmission channels, generate a set of association parameters for the features of different parts of the facility.

[0054] In this embodiment, the facility location features of each frame in the continuously detected image sequence are input to five sets of feature interaction units. The corrugated beam feature interaction unit extracts the original features of the corrugated beam and converts them into standardized corrugated beam location feature parameters. The column feature interaction unit extracts the original features of the column and converts them into standardized column location feature parameters. The anti-blocking block feature interaction unit extracts the original features of the anti-blocking block and converts them into standardized anti-blocking block location feature parameters. The end feature interaction unit extracts the original features of the end and converts them into standardized end location feature parameters. The bolt feature interaction unit extracts the original features of the bolt and converts them into standardized bolt location feature parameters. These standardized location feature parameters are input to ten association transmission channels. The information filtering subunit filters out information related to the association between two location features. The association parameter integration unit collects the association information output from all association transmission channels. The feature weight allocation subunit assigns corresponding weight values ​​to each association parameter in the association parameter set, ultimately generating an association parameter set containing the degree of association between the features of each location.

[0055] Step S133: Input the changing parameter sequence and the associated parameter set into the feature coupling processing module. The feature coupling processing module includes a parameter alignment subunit, which is used to match the time dimension of the changing parameter sequence with the spatial dimension of the associated parameter set.

[0056] In this embodiment, the time dimension of the changing parameter sequence is 7,199, representing the time interval between two adjacent frames in a continuously detected image sequence; the spatial dimension of the associated parameter set is five, representing the five part types of the corrugated beam guardrail. The parameter alignment subunit uses time slicing to match the time dimension of the changing parameter sequence with the spatial dimension of the associated parameter set, dividing the changing parameter sequence into time slices with the same number of spatial dimensions as the associated parameter set. Each time slice corresponds to an associated parameter for one part type. For example, the changing parameter sequence is divided into five time slices: the first time slice corresponds to the associated parameters of the corrugated beam plate, the second time slice corresponds to the associated parameters of the column, the third time slice corresponds to the associated parameters of the anti-blocking block, the fourth time slice corresponds to the associated parameters of the end, and the fifth time slice corresponds to the associated parameters of the bolt. Through the parameter alignment subunit, it is ensured that the time dimension of the changing parameter sequence and the spatial dimension of the associated parameter set can be accurately matched.

[0057] Step S134: The interactive operation subunit of the feature coupling processing module performs interactive operations on the aligned change parameters and associated parameters to generate an intermediate feature set containing spatiotemporal coupling information.

[0058] In this embodiment, the core function of the interactive operation subunit is to perform interactive processing on the aligned change parameters and associated parameters to generate an intermediate feature set that can reflect the interaction of spatiotemporal features.

[0059] Step S1341: Determine the operation mode of the interactive operation subunit. The operation mode is set based on the temporal characteristics of the changing parameters and the spatial characteristics of the associated parameters, so that the operation results can reflect the interaction of spatiotemporal features.

[0060] In this embodiment, the temporal characteristics of the changing parameters represent the changing process of facility features in consecutive frame images, while the spatial characteristics of the associated parameters represent the degree of correlation between features of different parts of the facility. Therefore, the operation mode is set to perform element-wise multiplication of the temporal characteristics of the changing parameters and the spatial characteristics of the associated parameters. The result of the multiplication can reflect the interaction between spatiotemporal features.

[0061] Step S1342: Extract the feature change amount of each time period from the aligned change parameters, and extract the feature correlation degree of each part from the aligned correlation parameters.

[0062] In this embodiment, the aligned change parameters are divided into five time slices, each time slice corresponding to a correlation parameter for a location type. The feature change amount for each time slice is extracted; this feature change amount is in the form of a multi-dimensional vector, where each element corresponds to the degree of change in the facility feature within that time slice. The feature correlation degree for each location is extracted from the aligned correlation parameters; this feature correlation degree is also in the form of a multi-dimensional vector, where each element corresponds to the degree of correlation between features of two locations.

[0063] Step S1343: Perform element-wise interactive processing on the characteristic change of each time period and the characteristic correlation degree of each part within the corresponding time period to generate the spatiotemporal interactive characteristic value of each time period.

[0064] In this embodiment, for the first time slice (corresponding to the associated parameters of the corrugated beam slab), the feature change amount of this time period is multiplied element-wise with the feature correlation degree of the corrugated beam slab to generate the spatiotemporal interaction feature value of this time period; for the second time slice (corresponding to the associated parameters of the column), the feature change amount of this time period is multiplied element-wise with the feature correlation degree of the column to generate the spatiotemporal interaction feature value of this time period; and so on, until the fifth time slice (corresponding to the associated parameters of the bolt) is processed. The spatiotemporal interaction feature value of each time period is in the form of a multi-dimensional vector, and each element in the vector corresponds to the degree of interaction of the spatiotemporal features within the time period.

[0065] Step S1344: Perform dimensional expansion processing on the spatiotemporal interaction feature values ​​of each time period, converting the single numerical form of the spatiotemporal interaction feature values ​​into multi-dimensional feature vectors.

[0066] In this embodiment, a feature mapping method is used to expand the dimensionality of the spatiotemporal interaction feature values ​​for each time period. Each spatiotemporal interaction feature value is mapped to a feature vector with a length equal to the feature mapping dimension. The feature mapping dimension is set according to the complexity and accuracy requirements of the model. For example, if the feature mapping dimension is set to one hundred, each spatiotemporal interaction feature value is mapped to a feature vector of length one hundred. By expanding the dimensionality, the dimensionality of the spatiotemporal interaction feature values ​​is increased, thereby improving the expressive power of the intermediate feature set.

[0067] Step S1345: Arrange the multi-dimensional feature vectors of each time period in chronological order to form a time-series feature vector sequence.

[0068] In this embodiment, the multi-dimensional feature vectors of the morning, noon, evening and night periods are arranged in chronological order to form a time-series feature vector sequence with a length of four periods. Each element in the sequence corresponds to a multi-dimensional feature vector of a period.

[0069] Step S1346: Smooth the temporally sequenced feature vector sequence to eliminate abrupt interference between feature vectors in adjacent time periods, and obtain an intermediate feature set containing spatiotemporal coupling information.

[0070] In this embodiment, a moving average method is used to smooth the time-series feature vector sequence. The size of the moving window is set to three time periods, meaning that the feature vector of each time period is composed of the average of the feature vectors of that time period, the preceding time period, and the following time period. For the feature vectors of the morning period, since there is no preceding time period, the size of the moving window is set to two time periods, i.e., composed of the average of the feature vectors of the morning and noon periods. For the feature vectors of the night period, since there is no following time period, the size of the moving window is set to two time periods, i.e., composed of the average of the feature vectors of the evening and night periods. Through smoothing, the abrupt interference between feature vectors of adjacent time periods is eliminated, ensuring the stability and accuracy of the intermediate feature set.

[0071] Step S135: Perform feature enhancement processing on the intermediate feature set, and amplify the feature signals related to the facility's technical condition in the intermediate feature set through the feature enhancement sub-unit.

[0072] In this embodiment, the feature enhancement subunit uses a nonlinear activation function to process the intermediate feature set. The value of the nonlinear activation function ranges from zero to one; a larger value indicates a higher degree of amplification of the feature signal. For feature signals related to the facility's technical condition (such as deformation features of corrugated beams, tilt features of columns, and damage features of anti-blocking blocks), the nonlinear activation function has a larger value, amplifying these feature signals. For feature signals unrelated to the facility's technical condition (such as background noise features and illumination change features), the nonlinear activation function has a smaller value, suppressing these feature signals. Through feature enhancement processing, the intensity of feature signals related to the facility's technical condition in the intermediate feature set is increased, while interference from irrelevant feature signals is reduced.

[0073] Step S136: Perform feature classification processing on the enhanced intermediate feature set, dividing it into different feature subsets according to the type of facility feature. All feature subsets together constitute the facility feature coupling set.

[0074] In this embodiment, the types of facility features include deformation features, damage features, tilting features, and loosening features. Feature classification processing employs a clustering algorithm to process the enhanced intermediate feature set, grouping feature vectors with similar features into the same feature subset. For example, the deformation feature vector of the corrugated beam slab is grouped into a deformation feature subset, the damage feature vector of the column is grouped into a damage feature subset, the tilting feature vector of the guardrail block is grouped into a tilting feature subset, and the loosening feature vector of the bolt is grouped into a loosening feature subset. All feature subsets together constitute a facility feature coupling set, which contains all technical condition features of the corrugated beam guardrail at different times and locations.

[0075] Step S140: Mining abnormal evolution trajectories of facility features based on facility feature coupling sets, wherein the abnormal evolution trajectories are obtained by comparing the facility feature coupling sets with preset normal feature evolution benchmarks.

[0076] In this embodiment, the preset normal feature evolution benchmark is the feature coupling sample sequence of the corrugated beam guardrail under normal technical conditions. This sample sequence is obtained by collecting a large number of continuous detection image sequences of the corrugated beam guardrail under normal technical conditions and inputting them into the facility feature evolution model.

[0077] Step S141: Obtain a preset normal feature evolution benchmark, which includes a feature coupling sample sequence of the facility under normal technical conditions.

[0078] In this embodiment, the preset normal feature evolution benchmark is obtained as follows: One hundred continuous detection image sequences of corrugated beam guardrails under normal technical conditions are collected. Each corrugated beam guardrail is 20 meters long. The collection periods include four different time periods: early morning, noon, evening, and night. Each time period lasts for 30 minutes, with one frame captured every second, resulting in 1800 frames per time period and a total of 7200 frames across the four time periods. These continuous detection image sequences are input into the facility feature evolution model to obtain one hundred feature coupling sample sequences under normal technical conditions. Statistical analysis is performed on these one hundred feature coupling sample sequences to calculate the mean and standard deviation of each feature subset, generating a normal feature evolution benchmark that includes the mean and standard deviation of each feature subset.

[0079] Step S142: Input the facility feature coupling set and the normal feature evolution benchmark into the trajectory comparison module. The trajectory comparison module includes a feature difference calculation subunit, which is used to calculate the degree of difference between each feature subset in the facility feature coupling set and the corresponding sample subset in the normal feature evolution benchmark.

[0080] In this embodiment, the trajectory comparison module uses distance calculation to compare the facility feature coupling set with the normal feature evolution benchmark. The feature difference calculation subunit calculates the distance between each feature subset in the facility feature coupling set and the corresponding sample subset in the normal feature evolution benchmark. The distance value ranges from zero to one, with a larger value indicating a higher degree of difference. For example, it calculates the distance between the deformation feature subset and the deformation feature sample subset in the normal feature evolution benchmark, the damage feature subset and the damage feature sample subset in the normal feature evolution benchmark, the tilting feature subset and the tilting feature sample subset in the normal feature evolution benchmark, and the loosening feature subset and the loosening feature sample subset in the normal feature evolution benchmark.

[0081] Step S143: Select feature subsets that exceed the preset difference threshold based on the degree of difference, and mark the selected feature subsets as abnormal feature subsets.

[0082] In this embodiment, the preset difference threshold ranges from 0.5 to 0.8, with a larger value indicating a stricter screening standard for abnormal features. Based on the importance and technical requirements of the corrugated beam guardrail, the preset difference threshold is set to 0.6. For a subset of deformed features, if its distance from the subset of deformed features in the normal feature evolution benchmark exceeds 0.6, then the subset of deformed features is marked as an abnormal feature subset. For a subset of damaged features, if its distance from the subset of damaged features in the normal feature evolution benchmark exceeds 0.6, then the subset of damaged features is marked as an abnormal feature subset. For a subset of tilted features, if its distance from the subset of tilted features in the normal feature evolution benchmark exceeds 0.6, then the subset of tilted features is marked as an abnormal feature subset. For a subset of loosened features, if its distance from the subset of loosened features in the normal feature evolution benchmark exceeds 0.6, then the subset of loosened features is marked as an abnormal feature subset.

[0083] Step S144: Perform time-series tracing processing on the abnormal feature subset to trace the feature change process of the abnormal feature subset in different time periods and determine the start time and development time of the abnormal features.

[0084] In this embodiment, taking the deformable feature subset as an example, the deformable feature subset is marked as an abnormal feature subset and subjected to time-series tracing processing.

[0085] Step S1441: The abnormal feature subset is split into feature segments of multiple time periods in chronological order, and the feature segments correspond to a time period in the continuous detection image sequence.

[0086] In this embodiment, the deformation feature subset is divided into four feature segments in chronological order: early morning feature segment, noon feature segment, evening feature segment, and nighttime feature segment. Each feature segment corresponds to a time period in the continuously detected image sequence.

[0087] Step S1442: Perform feature intensity analysis on the feature segments of each time period and extract the intensity parameters of abnormal features in each feature segment.

[0088] In this embodiment, the feature intensity analysis uses eigenvalue summation to process the feature segments of each time period. The eigenvalues ​​of all feature vectors in each feature segment are summed to obtain the abnormal feature intensity parameter for that time period. For example, the abnormal feature intensity parameter for the feature segment in the early morning is the sum of the eigenvalues ​​of all deformed feature vectors in that time period; the abnormal feature intensity parameter for the feature segment in the midday period is the sum of the eigenvalues ​​of all deformed feature vectors in that time period; the abnormal feature intensity parameter for the feature segment in the evening period is the sum of the eigenvalues ​​of all deformed feature vectors in that time period; and the abnormal feature intensity parameter for the feature segment in the nighttime period is the sum of the eigenvalues ​​of all deformed feature vectors in that time period.

[0089] Step S1443: Arrange the intensity parameters of each time period in chronological order to form an intensity change sequence.

[0090] In this embodiment, the anomalous feature intensity parameters of the early morning, noon, evening, and nighttime periods are arranged in chronological order to form an intensity change sequence with a length of four time periods. Each element in the sequence corresponds to the anomalous feature intensity parameter of one time period.

[0091] Step S1444: Analyze the trend of intensity parameter changes in the intensity change sequence, identify the time period when the intensity parameter first exceeds the preset intensity threshold, and determine the time period as the starting time of the abnormal feature.

[0092] In this embodiment, the preset intensity threshold is set to the average value of the corresponding feature subset in the normal feature evolution benchmark plus twice the standard deviation. A larger value indicates a more stringent standard for identifying the starting time of the abnormal feature. Based on the importance and technical requirements of the corrugated beam guardrail, the preset intensity threshold is set to the average value of the deformable feature sample subset in the normal feature evolution benchmark plus twice the standard deviation. The trend of intensity parameter changes in the intensity change sequence is analyzed. If the intensity parameter in the early morning does not exceed the preset intensity threshold, but the intensity parameter in the midday period exceeds the preset intensity threshold, then the midday period is determined as the starting time of the abnormal feature; if the intensity parameters in both the early morning and midday periods do not exceed the preset intensity threshold, but the intensity parameter in the evening period exceeds the preset intensity threshold, then the evening period is determined as the starting time of the abnormal feature; and so on.

[0093] Step S1445: Identify the period after the initial period in which the intensity parameters continue to change, and determine this period as the development period of the anomalous feature.

[0094] In this embodiment, if the starting time is noon, and the intensity parameters of the evening and nighttime periods after noon continue to increase, then the evening and nighttime periods are determined as the development period of the anomalous feature; if the starting time is evening, and the intensity parameters of the nighttime periods after evening continue to increase, then the nighttime period is determined as the development period of the anomalous feature; and so on.

[0095] Step S1446: Record the details of the intensity parameter changes of the abnormal features during the initial and development periods to form descriptive information about the abnormal feature change process.

[0096] In this embodiment, the specific values, amplitudes, rates of change, and other details of the intensity parameters of the abnormal features during the initial and development periods are recorded to form descriptive information about the change process of the abnormal features. For example, if the initial period is midday, the intensity parameter is the average of the deformed feature sample subset in the normal feature evolution benchmark plus three standard deviations; the magnitude of change is twice the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark; and the rate of change is 0.5 times the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark per hour. If the development period is evening, the intensity parameter is the average of the deformed feature sample subset in the normal feature evolution benchmark plus four standard deviations; the magnitude of change is one time the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark; and the rate of change is 0.3 times the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark per hour. If the development period is nighttime, the intensity parameter is the average of the deformed feature sample subset in the normal feature evolution benchmark plus five standard deviations; the magnitude of change is one time the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark; and the rate of change is 0.2 times the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark per hour. By recording these details of change, descriptive information about the abnormal feature change process is formed.

[0097] Step S145: Extract the feature parameters of the abnormal feature subset during the initial and development periods, and construct the change path of the abnormal features.

[0098] In this embodiment, feature parameters of the deformation feature subset are extracted during the initial period (midday) and the development period (evening and night). These parameters include deformation location, deformation degree, and deformation direction. The deformation location is the middle position of the corrugated beam, the deformation degree is the maximum deformation of the corrugated beam, and the deformation direction is perpendicular to the extension direction of the corrugated beam. These feature parameters are arranged in chronological order to construct the change path of the abnormal feature. This path includes the changes of all feature parameters of the abnormal feature during the initial and development periods.

[0099] Step S146: Integrate the change paths of different subsets of abnormal features according to their temporal relationships to form the abnormal evolution trajectory of facility features.

[0100] In this embodiment, if the damage feature subset is also marked as an anomalous feature subset in addition to the deformation feature subset, the same time-series tracing processing is performed on the damage feature subset to extract its feature parameters during the initial and development periods, thus constructing the change path of the anomalous features. The change paths of the deformation feature subset and the damage feature subset are integrated according to their temporal correlation to form the anomalous evolution trajectory of the facility features. This trajectory includes all anomalous feature changes of the corrugated beam guardrail at different times and locations.

[0101] Step S150: Generate a facility technical condition inspection report containing a description of the abnormal development trend based on the abnormal evolution trajectory, and transmit the facility technical condition inspection report to the target management terminal.

[0102] In this embodiment, the target management terminal is the monitoring center terminal of the highway management department, which is used to receive and process technical condition inspection reports of highway traffic safety facilities.

[0103] Step S151: Analyze the abnormal evolution trajectory and extract the abnormal feature types, starting time, development time and feature change details in the abnormal evolution trajectory.

[0104] In this embodiment, the abnormal feature types in the abnormal evolution trajectory are deformation features and damage features. The starting time is noon, the development time is evening and night. The feature change details are the deformation location, deformation degree, deformation direction of the deformation features, and the damage location, damage degree, and damage type of the damage features.

[0105] Step S152: Query the preset trend prediction rules based on the abnormal feature types. The trend prediction rules include descriptions of the development patterns of different abnormal feature types.

[0106] In this embodiment, the preset trend prediction rules are stored in a trend prediction database, which contains descriptions of the development patterns of different abnormal feature types. For deformation feature types, the development pattern is described as the degree of deformation gradually increases with time, the deformation direction remains unchanged, and the deformation location gradually expands; for damage feature types, the development pattern is described as the degree of damage gradually increases with time, the damage type gradually develops from local damage to overall damage, and the damage location gradually expands.

[0107] Step S153: Based on the trend prediction rules and the details of the changes in abnormal features, predict the direction and speed of change of the abnormal features during the prediction period to form a description of the abnormal development trend.

[0108] In this embodiment, taking deformation features as an example, the direction and speed of change of the features during the prediction period are predicted based on the trend prediction rules and the details of the changes in abnormal features.

[0109] For example, step S1531: Extract the trend prediction model corresponding to the current abnormal feature type from the trend prediction rules. The trend prediction model includes the related factors and change patterns of the abnormal feature changes.

[0110] In this embodiment, the current abnormal feature type is deformation feature. The trend prediction model corresponding to the deformation feature is extracted from the trend prediction rules. The model includes the related factors of deformation feature change (such as the number of vehicle collisions, vehicle load, ambient temperature change, etc.) and the description of the change law (such as the degree of deformation increases linearly with the increase of the number of vehicle collisions, increases exponentially with the increase of vehicle load, and increases slowly with the increase of ambient temperature change, etc.).

[0111] Step S1532: Input the change details of the abnormal features into the trend prediction model. The change details include the intensity change parameters and change cycle parameters of the abnormal features during the development period.

[0112] In this embodiment, the intensity change parameter of the abnormal feature during the development period is that the degree of deformation increases from the average value of the deformed feature sample subset in the normal feature evolution benchmark plus three times the standard deviation to the average value of the deformed feature sample subset in the normal feature evolution benchmark plus five times the standard deviation, and the change period parameter is that it increases by 0.3 times the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark every hour.

[0113] Step S1533: Calculate the prediction intensity parameter and prediction direction of change of the abnormal features in the first prediction period using the trend prediction model.

[0114] In this embodiment, the first prediction period is the first hour of the future. The predicted intensity parameter of the deformation feature in the first prediction period is calculated by the trend prediction model. It is the average value of the deformation feature sample subset in the normal feature evolution benchmark plus five and a half standard deviations. The predicted change direction is perpendicular to the extension direction of the corrugated beam plate.

[0115] Step S1534: Following the method of calculating the prediction intensity parameters and prediction change direction of the abnormal features in the first prediction period, calculate the prediction intensity parameters and prediction change direction of multiple prediction periods after the first prediction period in sequence to form a multi-period prediction parameter sequence.

[0116] In this embodiment, multiple prediction periods following the first prediction period are the second, third, fourth, and fifth hours in the future. A trend prediction model sequentially calculates the predicted intensity parameters and predicted direction of change for the deformation features during these prediction periods. For example, the predicted intensity parameter for the second hour is the average of the deformation feature sample subset in the normal feature evolution benchmark plus 5.6 times the standard deviation, and the predicted direction of change is perpendicular to the extension direction of the corrugated beam. The predicted intensity parameter for the third hour is the average of the deformation feature sample subset in the normal feature evolution benchmark plus 5.9 times the standard deviation, and the predicted direction of change is perpendicular to the extension direction of the corrugated beam. The predicted intensity parameter for the fourth hour is the average of the deformation feature sample subset in the normal feature evolution benchmark plus 6.2 times the standard deviation, and the predicted direction of change is perpendicular to the extension direction of the corrugated beam. The predicted intensity parameter for the fifth hour is the average of the deformation feature sample subset in the normal feature evolution benchmark plus 6.5 times the standard deviation, and the predicted direction of change is perpendicular to the extension direction of the corrugated beam. These predicted intensity parameters and predicted directions of change are arranged chronologically to form a multi-period prediction parameter sequence.

[0117] Step S1535: Analyze the overall rate of change of abnormal features based on the predicted parameter sequence. The rate of change is obtained by the difference between the predicted intensity parameters of adjacent time periods.

[0118] In this embodiment, the difference in predicted intensity parameters between adjacent time periods is the difference between the predicted intensity parameters of the second and first hours, the third and second hours, and so on. For example, the difference between the predicted intensity parameters of the second and first hours is 0.3 times the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark; the difference between the third and second hours is 0.3 times the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark; the difference between the fourth and third hours is 0.3 times the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark; and the difference between the fifth and fourth hours is 0.3 times the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark. By calculating these differences, the overall rate of change of the anomalous features is found to be 0.3 times the standard deviation of the deformed feature sample subset in the normal feature evolution benchmark per hour.

[0119] Step S1536: Integrate the predicted change direction and rate of change from multiple time periods to form an abnormal development trend description that includes time nodes and predicted characteristic states.

[0120] In this embodiment, the predicted direction and rate of change for the first, second, third, fourth, and fifth hours in the future are integrated to form an abnormal development trend description that includes time nodes and predicted characteristic states. For example, for the first hour in the future, the predicted characteristic state is that the deformation degree reaches the average value of the deformed characteristic sample subset in the normal characteristic evolution benchmark plus 5.3 times the standard deviation, and the deformation direction is perpendicular to the extension direction of the corrugated beam plate; for the second hour in the future, the predicted characteristic state is that the deformation degree reaches the average value of the deformed characteristic sample subset in the normal characteristic evolution benchmark plus 5.6 times the standard deviation, and the deformation direction is perpendicular to the extension direction of the corrugated beam plate; and so on, until the fifth hour in the future, when the predicted characteristic state is that the deformation degree reaches the average value of the deformed characteristic sample subset in the normal characteristic evolution benchmark plus 6.5 times the standard deviation, and the deformation direction is perpendicular to the extension direction of the corrugated beam plate.

[0121] Step S154: Collect the processing parameters of the facility feature evolution model, the key feature parameters of the facility feature coupling set, and the core information of the abnormal evolution trajectory, and integrate the processing parameters of the facility feature evolution model, the key feature parameters of the facility feature coupling set, and the core information of the abnormal evolution trajectory with the description of the abnormal development trend.

[0122] In this embodiment, the processing parameters of the facility feature evolution model include the model's input dimension, the number of iterative processing unit layers, and correlation coefficients; the key feature parameters of the facility feature coupling set include the feature parameters of the deformation feature subset and the feature parameters of the damage feature subset; the core information of the abnormal evolution trajectory includes the abnormal feature type, start time, development time, and details of feature changes. Integrating this information with the description of the abnormal development trend forms a complete facility technical condition monitoring report.

[0123] Step S155: Arrange the integrated information according to the preset report structure, which includes basic facility information, abnormal feature analysis, abnormal development trend, and detection conclusion.

[0124] In this embodiment, basic facility information includes the location, length, and structural composition of the corrugated beam guardrail; anomaly analysis includes the type of anomaly, its start time, development time, and details of changes in the anomaly; anomaly development trends include the predicted time period and predicted characteristic state; and detection conclusions include an evaluation of the technical condition of the corrugated beam guardrail and maintenance recommendations. The integrated information is formatted according to a pre-defined report structure to ensure the report content is clear, logically sound, and well-organized.

[0125] Step S156: After generating the facility technical condition inspection report, transmit the facility technical condition inspection report to the target management terminal through an encrypted transmission channel.

[0126] In this embodiment, the encrypted transmission channel uses the SSL encryption protocol for data transmission, ensuring the security and integrity of the facility technical condition inspection report during transmission. The generated facility technical condition inspection report is transmitted to the monitoring center terminal of the highway management department through the encrypted transmission channel. The monitoring center terminal receives and processes the report, and takes appropriate maintenance measures in a timely manner to ensure the normal operation of highway traffic safety facilities.

[0127] Figure 2 The illustration shows exemplary hardware and software components of an AI-based highway traffic safety facility technical condition detection system 100 that can implement the ideas of this application, according to some embodiments of this application. For example, a processor 120 can be used in the AI-based highway traffic safety facility technical condition detection system 100 and to perform the functions described in this application.

[0128] The AI-based highway traffic safety facility technical condition detection system 100 can be a general-purpose server or a special-purpose server; both can be used to implement the AI-based highway traffic safety facility technical condition detection method of this application. Although only one server is shown in this application, for convenience, the functions described in this application can be implemented in a distributed manner on multiple similar platforms to balance the load.

[0129] For example, the AI-based highway traffic safety facility technical condition detection system 100 may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the AI-based highway traffic safety facility technical condition detection system 100 may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The methods of this application can be implemented according to these program instructions. The AI-based highway traffic safety facility technical condition detection system 100 also includes an I / O interface 150 between the computer and other input / output devices.

[0130] For ease of explanation, only one processor is described in the AI-based highway traffic safety facility technical condition detection system 100. However, it should be noted that the AI-based highway traffic safety facility technical condition detection system 100 of this application may also include multiple processors. Therefore, the steps performed by one processor described in this application may also be performed jointly or individually by multiple processors. For example, if the processor of the AI-based highway traffic safety facility technical condition detection system 100 performs steps A and B, it should be understood that steps A and B may also be performed jointly by two different processors or individually by one processor. For example, the first processor performs step A, the second processor performs step B, or the first processor and the second processor jointly perform steps A and B.

[0131] Furthermore, this embodiment of the invention also provides a readable storage medium, wherein computer-executable instructions are preset in the readable storage medium, and when the processor executes the computer-executable instructions, the above-mentioned artificial intelligence-based method for detecting the technical condition of highway traffic safety facilities is implemented.

[0132] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A method for detecting the technical condition of highway traffic safety facilities based on artificial intelligence, characterized in that, The method includes: Collect a continuous image sequence of highway traffic safety facilities at different time periods, wherein the continuous image sequence contains multiple frames of the same facility under different environmental conditions; A facility feature evolution model is constructed, which includes a temporal feature tracking module and a spatial feature association module. The temporal feature tracking module models the change process of facility features in a continuous detection image sequence, and the spatial feature association module models the spatial dependency relationship of features in different parts of the facility. The continuous detection image sequence is input into the facility feature evolution model. The temporal feature tracking module extracts the change parameters of facility features at different time periods, and the spatial feature association module extracts the association parameters of features of different parts of the facility. The dynamic feature coupling processing is performed by combining the change parameters and the association parameters to obtain the facility feature coupling set. The abnormal evolution trajectory of facility features is mined based on the facility feature coupling set, which is obtained by comparing the facility feature coupling set with a preset normal feature evolution benchmark. Based on the abnormal evolution trajectory, a facility technical condition inspection report containing a description of the abnormal development trend is generated, and the facility technical condition inspection report is transmitted to the target management terminal.

2. The method for detecting the technical condition of highway traffic safety facilities based on artificial intelligence according to claim 1, characterized in that, The facility feature evolution model is constructed, which includes a temporal feature tracking module and a spatial feature association module. The temporal feature tracking module models the change process of facility features in a continuously detected image sequence, and the spatial feature association module models the spatial dependencies of features in different parts of the facility, including: Determine the input dimension of the facility feature evolution model, and match the input dimension with the image resolution and number of frames of the continuous detection image sequence; The network structure of the temporal feature tracking module is constructed. The temporal feature tracking module contains multiple layers of recurrent processing units. The input of the recurrent processing unit is the facility feature parameters of the previous frame image, and the output is the difference in facility feature changes between the current frame and the previous frame image. Configure the cyclic update parameters of the temporal feature tracking module. The cyclic update parameters are used to control the feature transfer weights between multi-layer cyclic processing units, so that the feature change difference can reflect the gradual changes of facility features in continuous frame images. A network structure for a spatial feature association module is constructed. The spatial feature association module contains multiple sets of feature interaction units. Each feature interaction unit corresponds to a feature of a part of the facility. An association transmission channel is set between different feature interaction units. Configure the correlation coefficient of the spatial feature association module. The correlation coefficient is used to define the information transmission strength between different feature interaction units, so that the correlation parameters can reflect the degree of mutual influence of features in different parts of the facility. The temporal feature tracking module and the spatial feature association module are connected through a feature fusion interface. The feature fusion interface is used to unify the format of the changing parameters and the associated parameters, thereby completing the construction of the facility feature evolution model.

3. The method for detecting the technical condition of highway traffic safety facilities based on artificial intelligence according to claim 2, characterized in that, The network structure for building the temporal feature tracking module includes multiple layers of recurrent processing units. The input to each recurrent processing unit is the facility feature parameters of the previous frame image, and the output is the difference in facility feature changes between the current frame and the previous frame image, including: The number of loop processing unit layers in the temporal feature tracking module is determined, and the number of loop processing unit layers corresponds to the number of frames in the continuously detected image sequence; The first-layer loop processing unit is constructed. The input of the first-layer loop processing unit is the facility feature parameters of the first frame image in the continuously detected image sequence. The facility feature change difference between the first frame and the second frame image is generated through feature difference operation. A second-layer loop processing unit is constructed. The input of the second-layer loop processing unit is the facility feature change difference output by the first-layer loop processing unit and the facility feature parameters of the third frame image. The facility feature change difference between the second frame and the third frame image is generated through feature superposition operation. The remaining loop processing units are constructed sequentially in the same way as the second-layer loop processing unit. The input of each of the remaining loop processing units includes the facility feature change difference output by the previous loop processing unit and the facility feature parameters of the corresponding frame image. A feature memory unit is configured for the loop processing unit. The feature memory unit is used to store the facility feature parameters of the previous frame image for the current loop processing unit to perform feature comparison calculations. A difference integration unit is set at the output of all loop processing units. The difference integration unit is used to arrange the facility characteristic change difference output by each loop processing unit in time order to form a time-seriesd change parameter sequence.

4. The method for detecting the technical condition of highway traffic safety facilities based on artificial intelligence according to claim 2, characterized in that, The network structure for building the spatial feature association module includes multiple sets of feature interaction units. Each feature interaction unit corresponds to a feature of a part of a facility. Association transmission channels are established between different feature interaction units, including: Classify the parts of highway traffic safety facilities, determine the number of parts based on the structural composition of the facilities, and each part type corresponds to a set of feature interaction units; The internal processing structure of the feature interaction unit is constructed. The feature interaction unit includes a part feature extraction subunit and a feature transformation subunit. The part feature extraction subunit is used to extract the original features of the corresponding part from the detection image, and the feature transformation subunit is used to convert the original features into standardized part feature parameters. A correlation transmission channel is set between different feature interaction units. The correlation transmission channel corresponds to the information transmission path between two feature interaction units. The number of correlation transmission channels matches the number of combinations of part types. An information filtering subunit is configured for the association transmission channel. The information filtering subunit is used to filter the feature information transmitted in the association transmission channel and retain the information related to the association between the two parts. A correlation parameter integration unit is set at the output end of the spatial feature correlation module. The correlation parameter integration unit is used to collect the correlation information output by all correlation transmission channels and generate a set of correlation parameters containing the correlation degree of each part's features. Configure a feature weight allocation subunit for the associated parameter integration unit. The feature weight allocation subunit is used to assign corresponding weight values ​​to each associated parameter in the associated parameter set according to the importance of different parts of the facility.

5. The method for detecting the technical condition of highway traffic safety facilities based on artificial intelligence according to claim 1, characterized in that, The process involves inputting a continuously detected image sequence into a facility feature evolution model, extracting change parameters of facility features at different time periods through a temporal feature tracking module, extracting correlation parameters of features from different parts of the facility through a spatial feature association module, and performing dynamic feature coupling processing by combining the change parameters and correlation parameters to obtain a facility feature coupling set, including: The continuous detection image sequence is input into the temporal feature tracking module of the facility feature evolution model in chronological order. Each frame of the image is processed sequentially through a multi-layer loop processing unit to generate a sequence of changing parameters of facility features at different time periods. The facility location features of each frame in the continuously detected image sequence are input into the spatial feature association module. Through multiple sets of feature interaction units and association transmission channels, a set of association parameters for the features of different parts of the facility is generated. The changing parameter sequence and the associated parameter set are input into the feature coupling processing module. The feature coupling processing module contains a parameter alignment subunit, which is used to match the time dimension of the changing parameter sequence with the spatial dimension of the associated parameter set. The interactive operation subunit of the feature coupling processing module performs interactive operations on the aligned changed parameters and associated parameters to generate an intermediate feature set containing spatiotemporal coupling information. The intermediate feature set is subjected to feature enhancement processing, which amplifies the feature signals related to the facility's technical condition in the intermediate feature set through the feature enhancement sub-unit; The enhanced intermediate feature set is then classified into different feature subsets according to the type of facility feature. All feature subsets together constitute the facility feature coupling set.

6. The method for detecting the technical condition of highway traffic safety facilities based on artificial intelligence according to claim 5, characterized in that, The interactive operation subunit of the feature coupling processing module performs interactive operations on the aligned changed parameters and associated parameters to generate an intermediate feature set containing spatiotemporal coupling information, including: The operation mode of the interactive operation subunit is determined. The operation mode is set based on the temporal characteristics of the changing parameters and the spatial characteristics of the associated parameters, so that the operation results can reflect the interaction of spatiotemporal features. Extract the feature changes of each time period from the aligned change parameters, and extract the feature correlation degree of each part from the aligned correlation parameters; The spatiotemporal interaction feature values ​​of each time period are generated by performing element-by-element interactive processing on the characteristic changes of each time period and the characteristic correlation of each part within the corresponding time period. The spatiotemporal interaction feature values ​​of each time period are subjected to dimensional expansion processing, transforming the single numerical form of the spatiotemporal interaction feature values ​​into multi-dimensional feature vectors; The multi-dimensional feature vectors of each time period are arranged in chronological order to form a time-series feature vector sequence. The temporal feature vector sequence is smoothed to eliminate abrupt interference between feature vectors in adjacent time periods, resulting in an intermediate feature set containing spatiotemporal coupling information.

7. The method for detecting the technical condition of highway traffic safety facilities based on artificial intelligence according to claim 1, characterized in that, The method for mining abnormal evolution trajectories of facility features based on facility feature coupling sets, wherein the abnormal evolution trajectory is obtained by comparing the facility feature coupling set with a preset normal feature evolution benchmark, includes: Obtain a preset normal feature evolution benchmark, which includes a feature coupling sample sequence of the facility under normal technical conditions; The facility feature coupling set and the normal feature evolution benchmark are input into the trajectory comparison module. The trajectory comparison module includes a feature difference calculation subunit, which is used to calculate the degree of difference between each feature subset in the facility feature coupling set and the corresponding sample subset in the normal feature evolution benchmark. Based on the degree of difference, feature subsets exceeding a preset difference threshold are selected and marked as abnormal feature subsets. A time-series tracing process is performed on the subset of abnormal features to trace the feature changes of the subset at different time periods and determine the start and development time periods of the abnormal features. Extract feature parameters of an abnormal feature subset during the initial and development periods, and construct the change path of the abnormal features; By integrating the change paths of different subsets of abnormal features according to their temporal correlation, an abnormal evolution trajectory of facility features is formed.

8. The method for detecting the technical condition of highway traffic safety facilities based on artificial intelligence according to claim 7, characterized in that, The step of performing time-series tracing processing on the subset of abnormal features, tracing the feature changes of the subset of abnormal features in different time periods, and determining the start and development periods of the abnormal features, includes: The abnormal feature subset is split into feature segments of multiple time periods in chronological order, and each feature segment corresponds to a time period in a continuous detection image sequence. Feature intensity analysis is performed on feature segments of each time period to extract the intensity parameters of abnormal features in each feature segment; The intensity parameters for each time period are arranged in chronological order to form an intensity change sequence; Analyze the trend of intensity parameter changes in the intensity change sequence, identify the time period when the intensity parameter first exceeds the preset intensity threshold, and determine the time period as the starting time of the abnormal feature; Identify the period after the initial period when the intensity parameters continue to change, and determine this period as the development period of the anomalous features; Record the details of the intensity parameter changes of the abnormal features during the initial and development periods to form descriptive information about the process of abnormal feature changes.

9. The method for detecting the technical condition of highway traffic safety facilities based on artificial intelligence according to claim 1, characterized in that, The process of generating a facility technical condition inspection report containing a description of the abnormal development trend based on the abnormal evolution trajectory, and transmitting the facility technical condition inspection report to the target management terminal, includes: Analyze the abnormal evolution trajectory and extract the abnormal feature types, starting time, development time and feature change details from the abnormal evolution trajectory; Based on the query of abnormal feature types, a preset trend prediction rule is established, and the trend prediction rule includes a description of the development pattern of different abnormal feature types. Based on the trend prediction rules and the details of the changes in abnormal features, predict the direction and speed of change of the abnormal features during the prediction period to form a description of the abnormal development trend. Collect the processing parameters of the facility feature evolution model, the key feature parameters of the facility feature coupling set, and the core information of the abnormal evolution trajectory, and integrate the processing parameters of the facility feature evolution model, the key feature parameters of the facility feature coupling set, and the core information of the abnormal evolution trajectory with the description of the abnormal development trend; The integrated information is formatted according to a preset report structure, which includes basic facility information, anomaly feature analysis, anomaly development trend, and detection conclusions. After the facility technical condition inspection report is generated, it is transmitted to the target management terminal through an encrypted transmission channel.

10. A highway traffic safety facility technical condition detection system based on artificial intelligence, characterized in that, The AI-based highway traffic safety facility technical condition detection system includes a processor and a memory, the memory and the processor being connected. The memory is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the memory to implement the AI-based highway traffic safety facility technical condition detection method according to any one of claims 1-9.

Images