Highway electromechanical operation and maintenance management method based on internet of things time series data

Abstract

The application relates to the field of Internet of Things terminal management, in particular to a highway electromechanical operation and maintenance management method based on Internet of Things time sequence data. The application selectsively decodes coded data streams, marks background interference frames according to a plurality of decoded frames, analyzes the influence of background interference on coded data, sets interference identification for coded data in an observation time domain section, identifies abnormal coded data for unmarked coded data, synchronously determines a target reference time domain section where the abnormal coded data is located, decodes coded data in the target reference time domain section, obtains a plurality of reference frames, and processes the reference frames. The application considers the difference of coded data collected by Internet of Things terminals, constructs a regular constraint image for different Internet of Things terminals, adaptively monitors multi-dimensional coded data, only partially decodes under the premise of ensuring reliability, reduces the consumption of computing resources when facing a large range of Internet of Things terminals, and improves the monitoring efficiency.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) terminal management, and in particular to a method for the operation and maintenance management of highway electromechanical systems based on IoT time-series data. Background Technology

[0002] Currently, the scale of highway electromechanical systems is growing rapidly, and their stable operation relies heavily on a large number of IoT monitoring terminals deployed along the route. These IoT monitoring terminals continuously generate massive video encoding data streams to provide data support. At present, traditional operation and maintenance methods mostly adopt back-end full decoding analysis to analyze anomalies along the highway by decoding images.

[0003] For example, Chinese Patent Publication No. CN118714267A discloses a method and system for optimizing the transmission of highway video surveillance videos. This invention obtains main message data and idle message data with different transmission priorities based on the highway surveillance videos captured by each highway camera during the monitoring period. It uses the main message data as probe messages to predict the transmission link bandwidth and combines it with the road condition information of each highway camera's monitored section to obtain the corresponding transmission rate for each highway camera during each transmission. Thus, it calculates whether to continue transmission or repeat the transmission rate based on the channel utilization during the transmission process according to each transmission rate, until all main message data and idle message data are transmitted. This results in higher accuracy and completeness of highway surveillance video transmission, as well as higher transmission efficiency.

[0004] However, the following problems still exist in the existing technology.

[0005] The data transmitted by a large number of IoT monitoring terminals that operate routinely is massive, and different road sections are often affected by background interference such as changes in lighting and weather. At the same time, the dynamic characteristics of different road sections vary significantly, all of which affect the analysis process. Using a unified standard for analysis consumes huge computing resources and results in poor detection efficiency and reliability. Summary of the Invention

[0006] To address this, the present invention provides a highway electromechanical operation and maintenance management method based on IoT time-series data. This method solves the problems in the existing technology where different road sections are often affected by background interference such as changes in lighting and weather. At the same time, the dynamic characteristics of different road sections vary significantly, all of which affect the analysis process. Using a unified standard for analysis results in huge computational resource consumption and poor detection efficiency and reliability.

[0007] To achieve the above objectives, the present invention provides a method for the operation and maintenance management of electromechanical systems on highways based on Internet of Things (IoT) time-series data, comprising:

[0008] Acquire coded data streams from several IoT terminals deployed along the highway;

[0009] Selective decoding is performed on the encoded data stream at each observation time segment to synchronously determine the temporal mapping relationship between the obtained decoded frames and the encoded data;

[0010] Background interference frames are marked according to several of the aforementioned decoding frames. The anomalies of the corresponding encoded data of the background interference frames relative to the non-background interference frames are compared in combination with the temporal mapping relationship. The influence of background interference on the encoded data is analyzed, and interference markers are set for the encoded data of the observation time domain segment.

[0011] For unidentified coded data, the coding indicators for the corresponding verification dimensions are determined. The verification dimensions are selected based on the pattern constraint profile, which is constructed based on the historical coded data stream of the IoT terminal and includes at least one verification dimension and corresponding coding indicator constraints.

[0012] Based on the determined coding index and corresponding coding index constraints, abnormal coding data is identified, and the target reference time domain segment in which the abnormal coding data is located is determined simultaneously.

[0013] The encoded data within the target reference time domain segment is decoded to obtain several reference frames, which are then processed, including...

[0014] Extract image parameters from the reference frame, construct the temporal variation curve of the image parameters, analyze the representation of the temporal variation curve of the image parameters, select the corresponding constraint conditions based on the representation to identify the target temporal segment, and extract the decoded frame within the target temporal segment.

[0015] The extracted decoded frames are analyzed to determine whether there are any abnormal targets.

[0016] Furthermore, the process of selectively decoding based on the encoded data stream and simultaneously determining the temporal mapping relationship between the resulting decoded frames and the encoded data includes:

[0017] A portion of the encoded data in the encoded data stream within the observation time domain is selected at a predetermined ratio for decoding, and the timestamp information corresponding to the encoded data is recorded synchronously.

[0018] A mapping relationship is established between the obtained decoded frames and the corresponding timestamp information.

[0019] Furthermore, the process of setting interferometric markers for the coded data in the observation time domain includes,

[0020] Mark the background region in the decoded frame and extract the chromaticity index of the background region;

[0021] Based on the offset ratio of the chromaticity index relative to the standard sample value, the background interference frames are marked, and the first timestamp information corresponding to each of the background interference frames and the second timestamp information corresponding to the non-background interference frames are marked simultaneously.

[0022] The first encoded data corresponding to the first timestamp information and the second encoded data corresponding to the second timestamp information are extracted respectively to compare the encoding indicators and verify the variability of the encoding indicators.

[0023] If the mutation condition is met, an interference flag is set for the coded data within the observation time domain segment;

[0024] Among them, decoded frames with an offset ratio greater than a predetermined offset threshold are determined as background interference frames, and the encoding indicators include frame size, motion vector magnitude, and DC coefficient.

[0025] Furthermore, the process of verifying the variability of coding metrics includes,

[0026] Compare the coding metrics corresponding to the first and second coded data respectively;

[0027] If the mean difference ratio of the coding index is greater than the predetermined difference ratio threshold, it is determined that there is coding index variability.

[0028] Furthermore, the process of constructing a corresponding pattern constraint profile based on the historical encoded data stream of the IoT terminal includes,

[0029] Extract the historical encoded data stream of the IoT terminal under normal conditions, and simultaneously extract the encoding indicators of each dimension;

[0030] The coefficient of variation corresponding to the coding index of each dimension is determined respectively. Dimensions that are less than or equal to the predetermined coefficient of variation threshold are determined as verification dimensions, and the corresponding coding index constraints are determined based on the normal distribution of the coding index.

[0031] Record all verification dimensions and coding indicator constraints of the IoT terminal to obtain a pattern constraint profile.

[0032] Furthermore, the process of identifying anomalous coded data based on the determined coding indices and corresponding coding index constraints includes,

[0033] Determine the coded data corresponding to different times, and compare the coded indicators of the verification dimension extracted from the profile with the corresponding coded indicator constraints based on the aforementioned rules.

[0034] If a verification dimension's coding metric is not subject to coding metric constraints, the coded data is determined to be abnormal coded data.

[0035] Furthermore, the process of analyzing the temporal variation curves of image parameters includes,

[0036] Extract image parameters from the reference frame;

[0037] Construct time-domain variation curves corresponding to image parameters, and distinguish the performance characteristics of each curve segment, including the standard deviation of the peak within the curve segment and the deviation of the average amplitude of the curve segment relative to the standard value.

[0038] The form of the curve segment is determined based on the standard deviation and the range of the deviation.

[0039] The representation forms include discrete deviation representation forms, steady-state deviation representation forms, and steady-state representation forms, and the image parameters are either chroma or grayscale values.

[0040] Furthermore, the process of selecting corresponding constraints based on the representation to identify the target time domain segment includes:

[0041] For the curve segments exhibiting discrete deviation, the difference ratio between adjacent peaks is verified. Adjacent peaks that do not meet the difference ratio constraint are identified as anomalous peaks, and the time domain segment where the anomalous peaks are located is determined as the target time domain segment.

[0042] For the curve segment exhibiting steady-state deviation, verify the abrupt change in amplitude relative to the average amplitude at a single moment, identify the moment that does not meet the abrupt change constraint as an abnormal moment, and identify the time domain segment where the abnormal moment is located as the target time domain segment;

[0043] For steady-state behavior, the abrupt change in amplitude relative to average amplitude at a single moment is verified. Moments that do not meet steady-state constraints are identified as anomalous moments, and the time domain segment where the anomalous moment is located is identified as the target time domain segment.

[0044] Furthermore, the difference ratio constraint condition is that the difference ratio must be less than a predetermined difference ratio threshold, the mutation constraint condition is that the mutation amount must be less than a predetermined mutation threshold, and the steady-state constraint condition is that the mutation amount must be less than a predetermined steady-state mutation threshold.

[0045] Wherein, the steady-state mutation threshold is less than the mutation threshold.

[0046] Furthermore, it also includes decoding all the identified encoded data and analyzing the extracted decoded frames to determine whether there are any abnormal targets.

[0047] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention selectively decodes the encoded data stream to determine the temporal mapping relationship between the decoded frames and the encoded data. Subsequently, background interference frames are marked based on several decoded frames to analyze the impact of background interference on the encoded data. Interference markers are set for the encoded data in the observation time domain segment. For unmarked encoded data, encoding indicators corresponding to the verification dimensions are determined. Abnormal encoded data is identified based on the determined encoding indicators and corresponding encoding indicator constraints, and the target reference time domain segment where the abnormal encoded data is located is simultaneously determined. Encoded data within the target reference time domain segment is decoded to obtain several reference frames, which are then processed. This invention considers the impact of background interference on encoded data collected by different IoT terminals, while also taking into account the differences in encoded data collected by IoT terminals. It constructs a pattern constraint profile for different IoT terminals, adaptively monitoring multi-dimensional encoded data. While ensuring reliability, only partial decoding is required, reducing computational resource consumption and improving monitoring efficiency when targeting a large number of IoT terminals.

[0048] In particular, this invention selectively decodes the encoded data stream, extracts a portion of the encoded data to obtain decoded frames, and then sets interference labels on the encoded data. The purpose is to consider the actual monitoring area of ​​the IoT terminal. Due to the differences in the monitoring area, some IoT terminals may have strong background interference in the images they collect. This background interference will also be reflected in the encoded data, causing changes in the encoded data, which in turn mask the changes caused by the dynamic behavior of vehicle features on the road, affecting subsequent judgments. Therefore, this invention uses periodic marking of background interference frames to observe the situation where the encoded data changes significantly compared to the normal state under strong background interference, and marks it to provide preliminary data support for subsequent selective analysis and improve the accuracy of subsequent analysis.

[0049] In particular, this invention constructs a pattern constraint profile for IoT terminals, taking into account the deployment environment of the IoT terminals. In monitoring data transmission, the IoT terminals encode image data and transmit it to other terminals, which then decode and obtain the image data. The content of the image data affects the encoding indicators of various dimensions of the encoded data. For example, due to differences in the deployment environment, the confidence levels of different dimensions of the encoding indicators will vary. For straight road sections, the traffic flow pattern is simple, and the stability of the encoding indicators of various dimensions of the encoded data transmitted by the IoT terminals is relatively strong. Abnormal dynamics of vehicles are more easily captured in the encoding indicators. For areas where vehicles merge or exit, the traffic situation is complex, and some dimensions of the encoding indicators of the encoded data transmitted by the IoT terminals may be in a normal discrete state. This discrete state will mask the characteristics reflected in the encoding indicators when there are actual anomalies in the monitored area, resulting in lower confidence levels of the corresponding dimension of the encoding indicators. Based on this, the situation in different areas also varies. Therefore, this invention constructs a pattern constraint profile for IoT terminals and subsequently adaptively considers encoding indicators with higher confidence levels, thereby reducing computational resource consumption and improving monitoring efficiency.

[0050] In particular, this invention extracts coding indicators based on pattern constraint profiles and identifies coding data according to the corresponding coding indicator constraints. The coding indicators of the verification dimension within the pattern constraint profile have high confidence. Therefore, coding indicator constraints are summarized based on historical coding data streams. When a coding indicator deviates from its corresponding coding indicator constraint, an alert is triggered, and further in-depth analysis is carried out. The coding data can be used to preliminarily analyze the situation of the actual monitoring area with low computing power consumption. Furthermore, under the premise of ensuring reliability, only partial decoding is required. When facing a large number of IoT terminals, this reduces the consumption of computing resources and improves monitoring efficiency.

[0051] In particular, this invention decodes the encoded data within the target reference time domain segment to obtain several reference frames. These reference frames are then processed to extract basic image parameters. Considering the changes in image parameters during actual traffic flow, a time-domain variation curve of the image parameters is constructed to analyze its representation. Image parameters are inherent attributes of the reference frames and can be directly extracted. Compared to reading image content for analysis, this method is computationally efficient and reflects changes in image content. Therefore, the invention differentiates between different representations. In practice, when traffic flow is low, the image is relatively simple, for example, a single vehicle moving in the image. While its image parameters may slightly deviate from normal values, they tend to remain stable, exhibiting a steady-state deviation. Conversely, when traffic flow is high, the image parameters become discrete, exhibiting a discrete deviation. Finally, when traffic is stationary or nonexistent, the image parameters are in a steady-state state. Different methods are used to analyze the image parameters under different representations. For example, for discrete deviation representations... In the curve segment of the pattern, the image parameters fluctuate, but the amplitude of the fluctuation remains within a certain range. Based on this, instantaneous detection is not adopted to avoid the image parameter fluctuations masking instantaneous changes. Instead, the difference ratio between adjacent peaks is verified. Adjacent peaks are combined for verification, considering relative differences. For steady-state deviations, the image parameters are relatively stable, so instantaneous fluctuations are considered to capture changes in a timely manner. For steady-state deviations, the image is normally stable, so a more sensitive threshold is set when capturing anomalies to facilitate timely detection. Through the above process, this invention uses easily extractable image parameters to quickly identify target time-domain segments with potential problems. Subsequently, the encoded data in the target time-domain segment is decoded, decoded frames are extracted, and then targeted image processing models are used for detailed analysis to identify abnormal targets. Furthermore, multi-dimensional encoded data is adaptively monitored. Under the premise of ensuring reliability, only partial decoding is required. When targeting a wide range of IoT terminals, this reduces the consumption of computing resources and improves monitoring efficiency. Attached Figure Description

[0052] Figure 1 This is a schematic diagram illustrating the steps of a highway electromechanical operation and maintenance management method based on Internet of Things time-series data, as an embodiment of the invention.

[0053] Figure 2 A logic block diagram illustrating the setting of interference flags in an embodiment of the invention;

[0054] Figure 3 This is a logic block diagram illustrating the verification of the variability of the coding index in an embodiment of the invention.

[0055] Figure 4 This is a logic block diagram for identifying abnormal encoded data according to an embodiment of the invention. Detailed Implementation

[0056] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0057] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0058] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0059] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0060] Please see Figure 1 The diagram illustrates the steps of a highway electromechanical maintenance management method based on IoT time-series data, according to an embodiment of the invention. The highway electromechanical maintenance management method based on IoT time-series data includes:

[0061] Step S1: Obtain the encoded data streams sent by several IoT terminals deployed along the highway;

[0062] Step S2: Selectively decode the encoded data stream at each observation time segment, and synchronously determine the temporal mapping relationship between the obtained decoded frame and the encoded data;

[0063] Step S3: Mark background interference frames according to several decoded frames, and compare the anomalies of the corresponding encoded data of background interference frames relative to non-background interference frames in combination with the temporal mapping relationship, so as to analyze the influence of background interference on encoded data and set interference labels for the encoded data of the observation time domain segment;

[0064] Step S4: For unidentified coded data, determine the coding index of the corresponding verification dimension. The verification dimension is selected based on the pattern constraint profile. The pattern constraint profile is constructed based on the historical coded data stream of the IoT terminal and includes at least one verification dimension and corresponding coding index constraints.

[0065] Step S5: Identify abnormal encoded data based on the determined encoding index and corresponding encoding index constraints, and simultaneously determine the target reference time domain segment where the abnormal encoded data is located.

[0066] Step S6: Decode the coded data within the target reference time domain segment to obtain several reference frames, and process the reference frames, including...

[0067] Extract image parameters from the reference frame, construct the temporal variation curve of the image parameters, analyze the representation of the temporal variation curve of the image parameters, select the corresponding constraint conditions based on the representation to identify the target temporal segment, and extract the decoded frame within the target temporal segment.

[0068] Step S7: Analyze the extracted decoded frames to determine whether there are any abnormal targets.

[0069] Specifically, there are no restrictions on the method for identifying whether there are abnormal targets in the decoded frame. An open-source target detection model or a self-trained image processing model that can identify target objects in the image can be used. Abnormal targets can be set according to the needs of those skilled in the art, such as human bodies, animals, smoke, explosions, flames, etc., which will not be elaborated here.

[0070] Specifically, IoT terminals involve deploying various surveillance cameras along highways.

[0071] Specifically, the process of selectively decoding based on the encoded data stream and synchronously determining the temporal mapping relationship between the resulting decoded frames and the encoded data includes:

[0072] A portion of the encoded data in the encoded data stream within the observation time domain is selected at a predetermined ratio for decoding, and the timestamp information corresponding to the encoded data is recorded synchronously.

[0073] A mapping relationship is established between the obtained decoded frames and the corresponding timestamp information.

[0074] In practice, the purpose of setting a predetermined proportion is to conduct random checks on the coded data stream. To ensure sufficient samples, the predetermined proportion is set to 10% of the total amount of coded data in the observation time domain. The observation time domain is set within the interval [5s, 10s] to ensure a shorter observation interval and timely detection of background interference.

[0075] Specifically, please refer to Figure 2As shown, it is a logic block diagram of setting interference markers according to an embodiment of the invention. The process of setting interference markers for the coded data of the observation time domain segment includes:

[0076] Mark the background region in the decoded frame and extract the chromaticity index of the background region;

[0077] Based on the offset ratio of the chromaticity index relative to the standard sample value, the background interference frames are marked, and the first timestamp information corresponding to each of the background interference frames and the second timestamp information corresponding to the non-background interference frames are marked simultaneously.

[0078] The first encoded data corresponding to the first timestamp information and the second encoded data corresponding to the second timestamp information are extracted respectively to compare the encoding indicators and verify the variability of the encoding indicators.

[0079] If the mutation condition is met, an interference flag is set for the coded data within the observation time domain segment;

[0080] Among them, decoded frames with an offset ratio greater than a predetermined offset threshold are determined as background interference frames, and the encoding indicators include frame size, motion vector magnitude, and DC coefficient.

[0081] In practice, if there are no non-background interferometric frames within the observation time domain, the corresponding non-background interferometric frames can be determined through the historical encoded data stream of the IoT terminal for subsequent analysis, which will not be elaborated further.

[0082] The frame size is the amount of data occupied by a single video frame (I-frame, P-frame, or B-frame) corresponding to the encoded data after compression. The frame size is the metadata of the encoded data and can be read directly.

[0083] The motion vector magnitude is obtained by reconstructing the syntax elements of the coded block from the coded data through entropy decoding, resulting in the motion vector difference; then, combined with the motion vector prediction values ​​derived from adjacent coded blocks, the complete motion vector is calculated by vector addition. The motion vector contains horizontal and vertical components, and the geometric length of the motion vector is taken as the motion vector magnitude.

[0084] When extracting DC coefficients, the encoded data is first entropy-decoded and parsed to obtain the quantized transform coefficient array corresponding to the target image block. In the standard zigzag scanning sequence, the first coefficient in the array is the DC coefficient of that block. A single video frame corresponds to multiple image blocks. In practice, the average of the DC coefficients corresponding to the image blocks can be used as the DC coefficient of a single video frame. The trend of the DC coefficients is consistent with the average brightness of the image blocks, which can indirectly reflect the brightness of the decoded image.

[0085] Specifically, the offset ratio is the ratio of the absolute difference between the chromaticity index and the standard sample value to the standard sample value. It aims to reflect the offset of the chromaticity index relative to the standard sample value. The standard sample value is predetermined. For different IoT terminals, the collected image data is acquired, and the image data is filtered to select image data with no background interference features. The average chromaticity index in the background area corresponding to the image data is statistically analyzed to reflect the chromaticity index in the background area under normal conditions. The average chromaticity index is set as the standard sample value.

[0086] Specifically, the chromaticity index is the average chromaticity value of each pixel in the background area.

[0087] Specifically, since the monitoring perspective of IoT terminals is mostly fixed, the corresponding background area can be pre-labeled for different IoT terminals. The background area is the non-road area in the image.

[0088] Specifically, the offset threshold value is predetermined. The image data collected by the corresponding IoT terminal is recorded, and the image data with interference features in the background area is filtered out. The mean value of the chromaticity index in the background area is determined, and the ratio of the mean value of the chromaticity index to the standard sample value is calculated to obtain the sample offset ratio, which reflects the offset ratio under the condition of background interference. In actual implementation, considering that there is a certain error, an error offset index is set. The product of the sample offset ratio and the error offset index is determined as the offset threshold value. The error offset index is selected in the interval [1.2, 1.25], and 1.2 is preferred in implementation.

[0089] Specifically, please refer to Figure 3 As shown, it is a logic block diagram for verifying the variability of coding indicators according to an embodiment of the invention. The process of verifying the variability of coding indicators includes,

[0090] Compare the coding metrics corresponding to the first and second coded data respectively;

[0091] If the mean difference ratio of the coding index is greater than the predetermined difference ratio threshold, it is determined that there is coding index variability.

[0092] It is understandable that there are multiple dimensions of coding indicators. The mean difference ratio is obtained by calculating the mean of the difference ratios of each dimension of coding indicators between the first and second coded data.

[0093] The purpose of setting the difference ratio threshold is to distinguish when the coding index of the coded data deviates from the normal state. In practice, the coded data under the condition of no abnormality in the area monitored by the IoT terminal is recorded, and the average difference ratio between the coding index and the mean of the coding index at several times is calculated. The mean of the average difference ratio is solved to reflect the deviation of the coding index under normal conditions. This mean can be appropriately used as the difference ratio threshold to reflect the deviation from the normal state. In practice, 1.25 times the mean of the difference ratio is set as the difference ratio threshold.

[0094] This invention selectively decodes the encoded data stream, extracting a portion of the encoded data to obtain decoded frames. Subsequently, interference labels are set for the encoded data. The purpose is to consider the actual monitoring area of ​​IoT terminals. Due to differences in monitoring areas, some IoT terminals may have strong background interference in their captured images. This background interference will also be reflected in the encoded data, causing changes in the encoded data and thus masking changes caused by the dynamic behavior of vehicles on the road, affecting subsequent judgments. Therefore, this invention uses periodic marking of background interference frames to observe situations where the encoded data changes significantly compared to the normal state under strong background interference, and then marks these changes to provide preliminary data support for subsequent selective analysis, improving the accuracy of subsequent analysis.

[0095] Specifically, the process of constructing a corresponding pattern constraint profile based on the historical encoded data stream of the IoT terminal includes,

[0096] Extract the historical encoded data stream of the IoT terminal under normal conditions, and simultaneously extract the encoding indicators of each dimension;

[0097] The coefficient of variation corresponding to the coding index of each dimension is determined respectively. Dimensions that are less than or equal to the predetermined coefficient of variation threshold are determined as verification dimensions, and the corresponding coding index constraints are determined based on the normal distribution of the coding index.

[0098] Record all verification dimensions and coding indicator constraints of the IoT terminal to obtain a pattern constraint profile.

[0099] In implementation, the coding index constraint is a closed interval. First, a normal distribution of the coding index is constructed, and a 95% confidence interval is determined. The 95% confidence interval is then used as the coding index constraint corresponding to the coding index.

[0100] The coefficient of variation is a relative indicator used in statistics to measure the dispersion of data. Generally, when the coefficient is less than 0.25, the data is considered to have strong consistency and statistical observation value. Based on this, in order to reflect the situation of poor consistency and strong dispersion of data indicators, the threshold of the coefficient of variation is set to 0.25.

[0101] This invention constructs a pattern constraint profile for IoT terminals, taking into account the deployment environment of the IoT terminals. In monitoring data transmission, the IoT terminals encode image data and transmit it to other terminals, which then decode and obtain the image data. The content of the image data affects the encoding indicators of various dimensions of the encoded data. For example, due to differences in the deployment environment, the confidence levels of different dimensions of the encoding indicators will vary. For straight road sections, the traffic flow pattern is simple, and the stability of the encoding indicators of various dimensions of the encoded data transmitted by the IoT terminals is relatively strong. Abnormal dynamics of vehicles are more easily captured in the encoding indicators. For areas where vehicles merge or exit, the traffic situation is complex, and some dimensions of the encoding indicators of the encoded data transmitted by the IoT terminals may be in a normal discrete state. This discrete state can mask the characteristics reflected in the encoding indicators when there are actual anomalies in the monitored area, resulting in lower confidence levels of the corresponding dimensions of the encoding indicators. Based on this, the situation varies from region to region. Therefore, this invention constructs a pattern constraint profile for IoT terminals and then adaptively considers the encoding indicators with higher confidence levels, thereby reducing computational resource consumption and improving monitoring efficiency.

[0102] Specifically, please refer to Figure 4 The diagram shown is a logical block diagram for identifying anomalous coded data according to an embodiment of the invention. The process of identifying anomalous coded data based on the determined coding index and corresponding coding index constraints includes:

[0103] Determine the coded data corresponding to different times, and compare the coded indicators of the verification dimension extracted from the profile with the corresponding coded indicator constraints based on the aforementioned rules.

[0104] If a verification dimension's coding metric is not subject to coding metric constraints, the coded data is determined to be abnormal coded data.

[0105] This invention extracts coded indicators based on pattern constraint profiles and identifies coded data according to the corresponding coded indicator constraints. The coded indicators of the verification dimensions within the pattern constraint profiles have high confidence. Therefore, based on historical coded data streams, coded indicator constraints are summarized. When a coded indicator deviates from its corresponding coded indicator constraint, an alert is triggered, and further in-depth analysis is conducted. The coded data can be used to preliminarily analyze the situation in the actual monitoring area with low computational power consumption. Furthermore, while ensuring reliability, only partial decoding is required. When targeting a large number of IoT terminals, this reduces computational resource consumption and improves monitoring efficiency.

[0106] Specifically, the process of analyzing the representation of the temporal variation curve of image parameters includes,

[0107] Extract image parameters from the reference frame;

[0108] Construct time-domain variation curves corresponding to image parameters, and distinguish the performance characteristics of each curve segment, including the standard deviation of the peak within the curve segment and the deviation of the average amplitude of the curve segment relative to the standard value.

[0109] The form of the curve segment is determined based on the standard deviation and the range of the deviation.

[0110] The representation forms include discrete deviation representation forms, steady-state deviation representation forms, and steady-state representation forms, and the image parameters are either chroma or grayscale values.

[0111] During implementation,

[0112] If the standard deviation is not within the predetermined standard deviation range, the judgment curve will show a discrete deviation.

[0113] If the standard deviation is within the predetermined standard deviation range but not within the predetermined deviation range, then the judgment curve represents a steady-state deviation.

[0114] If the standard deviation is within the predetermined standard deviation range and within the predetermined deviation range, then the judgment curve is in steady-state form.

[0115] The predetermined standard deviation range is determined in advance. Image data corresponding to situations where the proportion of vehicle outlines is less than 30% under normal conditions without abnormalities are recorded within a certain time period. Time-domain variation curves of image parameters are constructed within a certain time period. The mean standard deviation of the corresponding peaks in each time period is determined to reflect the performance of image parameters under conditions of low traffic flow. The standard deviation range is set as a closed interval, with the upper limit being 1.25 times the mean standard deviation and the lower limit being 0.75 times the mean standard deviation.

[0116] The deviation range is predetermined. Image data corresponding to the absence of vehicles under no abnormal conditions are recorded within a certain time period. Time-domain variation curves of image parameters are constructed within a certain time period. The mean deviation value corresponding to each time period is determined. The deviation range is set as a closed interval, with the upper limit being 1.25 times the mean deviation value and the lower limit being 0.75 times the mean deviation value.

[0117] In practice, the standard value is the average value of image parameters in the image data collected by IoT terminals within a historical period when there are no vehicles.

[0118] This invention decodes coded data within a target reference time domain segment to obtain several reference frames. These frames are then processed to extract basic image parameters. Considering the changes in image parameters during actual traffic flow, a time-domain variation curve of the image parameters is constructed to analyze its representation. Image parameters are inherent attributes of the reference frames and can be directly extracted. Compared to reading and analyzing image content, this method is computationally efficient and reflects changes in image content. Therefore, the invention differentiates between different representations. In practice, when traffic is light, the image is relatively simple (e.g., a single vehicle moving in the image). While the image parameters may slightly deviate from normal values, they tend to remain stable, exhibiting a steady-state deviation. Conversely, when traffic is heavy, the image parameters become discrete, exhibiting a discrete deviation. Finally, when traffic is stationary or nonexistent, the image parameters are in a steady-state state. Different methods are used to analyze the image parameters under different representations. For example, for discrete deviation representations... In the curve segment, image parameters fluctuate, but the amplitude of the fluctuation remains within a certain range. Based on this, instantaneous detection is not used to avoid the image parameter fluctuations masking instantaneous changes. Instead, the difference ratio between adjacent peaks is verified. Adjacent peaks are combined for verification, considering relative differences. For steady-state deviations, the image parameters are relatively stable, so instantaneous fluctuations are considered to capture changes in a timely manner. For steady-state deviations, the image is normally stable, so a more sensitive threshold is set when capturing anomalies to facilitate timely detection. Through the above process, this invention uses easily extractable image parameters to quickly identify target time-domain segments with potential problems. Subsequently, the encoded data in the target time-domain segment is decoded, decoded frames are extracted, and then targeted image processing models are used for detailed analysis to identify abnormal targets. Furthermore, multi-dimensional encoded data is adaptively monitored. Under the premise of ensuring reliability, only partial decoding is required. When targeting a wide range of IoT terminals, this reduces the consumption of computing resources and improves monitoring efficiency.

[0119] Specifically, the process of selecting corresponding constraints based on the form of representation to identify the target time domain segment includes,

[0120] For the curve segments exhibiting discrete deviation, the difference ratio between adjacent peaks is verified. Adjacent peaks that do not meet the difference ratio constraint are identified as anomalous peaks, and the time domain segment where the anomalous peaks are located is determined as the target time domain segment.

[0121] For the curve segment exhibiting steady-state deviation, verify the abrupt change in amplitude relative to the average amplitude at a single moment, identify the moment that does not meet the abrupt change constraint as an abnormal moment, and identify the time domain segment where the abnormal moment is located as the target time domain segment;

[0122] For steady-state behavior, the abrupt change in amplitude relative to average amplitude at a single moment is verified. Moments that do not meet steady-state constraints are identified as anomalous moments, and the time domain segment where the anomalous moment is located is identified as the target time domain segment.

[0123] Specifically, the difference ratio constraint condition is that the difference ratio must be less than a predetermined difference ratio threshold, the mutation constraint condition is that the mutation amount must be less than a predetermined mutation threshold, and the steady-state constraint condition is that the mutation amount must be less than a predetermined steady-state mutation threshold.

[0124] Wherein, the steady-state mutation threshold is less than the mutation threshold.

[0125] Specifically, the difference ratio threshold is predetermined. The image parameters in the image data collected by the IoT terminal under normal conditions are statistically analyzed in advance. The time-domain change curve of the image parameters is constructed, the curve segments that belong to the discrete deviation form are screened out, the difference ratio of several adjacent peaks in the curve segments is determined, the normal distribution of the difference ratio is constructed, the upper limit of the 95% confidence interval is determined, and the upper limit is determined as the difference ratio threshold.

[0126] The mutation threshold is predetermined. The image parameters in the image data collected by the IoT terminal under normal conditions are statistically analyzed in advance. The time-domain change curve of the image parameters is constructed, the curve segments that belong to the steady-state deviation are screened out, the mutation amount at a certain time is determined, the normal distribution of the mutation amount is constructed, the upper limit of the 95% confidence interval is determined, and the upper limit is determined as the mutation threshold.

[0127] The purpose of setting a steady-state mutation threshold is to assign a smaller threshold value to capture subtle changes. The steady-state mutation threshold is determined based on the mutation threshold, and in practice it is set to 0.5 times the mutation threshold. Those skilled in the art can also adjust it, which will not be elaborated here.

[0128] Specifically, this also includes decoding all the identified encoded data and analyzing the extracted decoded frames to determine whether there are any abnormal targets.

[0129] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for electromechanical operation and maintenance management of highways based on Internet of Things (IoT) time-series data, characterized in that, include: Acquire coded data streams from several IoT terminals deployed along the highway; Selective decoding is performed on the encoded data stream at each observation time segment to synchronously determine the temporal mapping relationship between the obtained decoded frames and the encoded data; Background interference frames are labeled based on several decoded frames. By combining the temporal mapping relationship, the anomalies of the corresponding encoded data of the background interference frames relative to the non-background interference frames are compared. The influence of background interference on the encoded data is analyzed, and interference labels are set for the encoded data of the observation time domain segment. For unidentified decoded data, the corresponding encoding indicators for verification dimensions are determined. The verification dimensions are selected based on the pattern constraint profile, which is constructed based on the historical encoded data stream of the IoT terminal and includes at least one verification dimension and corresponding encoding indicator constraints. Based on the determined coding index and corresponding coding index constraints, abnormal coding data is identified, and the target reference time domain segment in which the abnormal coding data is located is determined simultaneously. The encoded data within the target reference time domain segment is decoded to obtain several reference frames, which are then processed, including... Extract image parameters from the reference frame, construct the temporal variation curve of the image parameters, analyze the representation of the temporal variation curve of the image parameters, select the corresponding constraint conditions based on the representation to identify the target temporal segment, and extract the decoded frame within the target temporal segment. The extracted decoded frames are analyzed to determine whether any abnormal targets exist; The process of constructing a corresponding pattern constraint profile based on the historical encoded data stream of the IoT terminal includes: Extract the historical encoded data stream of the IoT terminal under normal conditions, and simultaneously extract the encoding indicators of each dimension; The coefficient of variation for each dimension of the coding index is determined, and the dimensions that are less than or equal to the predetermined coefficient of variation threshold are determined as the verification dimensions. The corresponding coding index constraints are determined based on the normal distribution of the coding index. Record all verification dimensions and coding indicator constraints of the IoT terminal to obtain a pattern constraint profile; The encoding metrics include frame size, motion vector direction entropy, motion vector magnitude, and DC coefficient.

2. The method for highway electromechanical operation and maintenance management based on IoT time-series data according to claim 1, characterized in that, The process of selectively decoding based on the encoded data stream and synchronously determining the temporal mapping relationship between the resulting decoded frames and the encoded data includes: A portion of the encoded data in the encoded data stream within the observation time domain is selected at a predetermined ratio for decoding, and the timestamp information corresponding to the encoded data is recorded synchronously. A mapping relationship is established between the obtained decoded frames and the corresponding timestamp information.

3. The highway electromechanical operation and maintenance management method based on IoT time-series data according to claim 2, characterized in that, The process of setting interferometric markers for coded data in the observation time domain includes, Annotate the background region in the decoded frame and extract the chromaticity index of the background region; Based on the offset ratio of the chromaticity index relative to the standard sample value, the background interference frames are labeled, and the first timestamp information corresponding to each background interference frame and the second timestamp information corresponding to the non-background interference frames are simultaneously labeled. The first encoded data corresponding to the first timestamp information and the second encoded data corresponding to the second timestamp information are extracted respectively to compare the encoding indicators and verify the variability of the encoding indicators. If the mutation condition is met, an interference flag is set for the coded data within the observation time domain segment; Among them, decoded frames with an offset ratio greater than a predetermined offset threshold are determined as background interference frames.

4. The method for highway electromechanical operation and maintenance management based on IoT time-series data according to claim 1, characterized in that, The process of verifying the variability of coding metrics includes, Compare the coding metrics corresponding to the first and second coded data respectively; If the mean difference ratio of the coding index is greater than the predetermined difference ratio threshold, it is determined that there is coding index variability.

5. The method for highway electromechanical operation and maintenance management based on IoT time-series data according to claim 1, characterized in that, The process of identifying anomalous coded data based on the determined coding indices and corresponding coding index constraints includes, Determine the coded data corresponding to different times, and compare the coded indicators of the verification dimension extracted from the profile with the corresponding coded indicator constraints based on the aforementioned rules. If a verification dimension's coding metric is not subject to coding metric constraints, the coded data is determined to be abnormal coded data.

6. The method for highway electromechanical operation and maintenance management based on IoT time-series data according to claim 1, characterized in that, The process of analyzing the temporal variation curves of image parameters includes: Extract image parameters from the reference frame; Construct time-domain variation curves corresponding to image parameters, and distinguish the performance characteristics of each curve segment, including the standard deviation of the peak within the curve segment and the deviation of the average amplitude of the curve segment relative to the standard value. The form of the curve segment is determined based on the range of standard deviation and average amplitude. The representation forms include discrete deviation representation forms, steady-state deviation representation forms, and steady-state representation forms, and the image parameters are either chroma or grayscale values.

7. The method for highway electromechanical operation and maintenance management based on IoT time-series data according to claim 1, characterized in that, The process of selecting corresponding constraints based on the representation to identify the target time domain segment includes: For the curve segments exhibiting discrete deviation, the difference ratio between adjacent peaks is verified. Adjacent peaks that do not meet the difference ratio constraint are identified as anomalous peaks, and the time domain segment where the anomalous peaks are located is determined as the target time domain segment. For the curve segment exhibiting steady-state deviation, verify the abrupt change in amplitude relative to the average amplitude at a single moment, identify the moment that does not meet the abrupt change constraint as an abnormal moment, and determine the time domain segment where the abnormal moment is located as the target time domain segment; For steady-state behavior, the abrupt change in amplitude relative to average amplitude at a single moment is verified. Moments that do not meet steady-state constraints are identified as anomalous moments, and the time domain segment where the anomalous moment is located is identified as the target time domain segment.

8. The method for highway electromechanical operation and maintenance management based on Internet of Things time-series data according to claim 7, characterized in that, The difference ratio constraint condition is that the difference ratio must be greater than a predetermined difference ratio threshold; the mutation constraint condition is that the mutation amount must be greater than a predetermined mutation threshold; and the steady-state constraint condition is that the mutation amount must be greater than a predetermined steady-state mutation threshold. Wherein, the steady-state mutation threshold is less than the mutation threshold.

9. The method for highway electromechanical operation and maintenance management based on IoT time-series data according to claim 1, characterized in that, It also includes decoding all the identified decoded data and analyzing the extracted decoded frames to determine whether there are any abnormal targets.

Images