Novel road area traffic safety facility intelligent patrol terminal

Abstract

The invention discloses a novel road area traffic safety facility intelligent patrol terminal, and relates to the technical field of road traffic safety monitoring, and the terminal comprises a facility distribution determination module which is used for determining a traffic safety facility distribution network; the image acquisition module is used for performing real-time image acquisition on the first traffic facility to generate a first facility image time sequence; the abnormity identification module is used for carrying out state abnormity identification on the first traffic facility to generate a first abnormity identification result; and the abnormity reminding module is used for sending the first abnormity recognition result to associated maintenance personnel for reminding. The method is used for solving the technical problem that road potential safety hazards cannot be eliminated in time due to the fact that abnormity of traffic safety facilities is difficult to find rapidly and accurately in a traditional patrol mode, intelligent and real-time patrol of the road area traffic safety facilities is achieved, facility abnormity is rapidly and accurately positioned, maintenance personnel are reminded to deal with the abnormity in time, and the maintenance efficiency is improved. And potential safety hazards of roads are effectively eliminated.

Description

Technical Field

[0001] The present invention relates to the technical field of road traffic safety monitoring, and particularly to an intelligent inspection terminal for a new type of road area traffic safety facility. Background Art

[0002] The stable operation of traffic safety facilities plays a key role in ensuring road safety. At present, for the inspection of road area traffic safety facilities, it mainly relies on manual inspection or simple equipment assistance. Manual inspection has low efficiency, is greatly affected by human factors, and is prone to omissions; although simple equipment assistance can collect some data, its functions are single.

[0003] With the continuous expansion of the road network and the continuous growth of traffic flow, the traditional inspection methods are difficult to meet the requirements. It is impossible to grasp the distribution and status of traffic safety facilities in real time and comprehensively, and it is impossible to detect damages, anomalies, etc. of the facilities in time, making it difficult to eliminate road safety hazards in time. This not only affects the normal passage of the road, but also poses a threat to the life and property safety of road users. Summary of the Invention

[0004] This application provides an intelligent inspection terminal for a new type of road area traffic safety facility, which is used to solve the technical problem that the traditional inspection method is difficult to quickly and accurately detect anomalies of traffic safety facilities, thereby resulting in the inability to promptly eliminate road safety hazards.

[0005] In view of the above problems, this application provides an intelligent inspection terminal for a new type of road area traffic safety facility.

[0006] This application provides an intelligent inspection terminal for a new type of road area traffic safety facility. The terminal includes: a facility distribution determination module, configured to determine the traffic safety facility distribution network within a preset road area; an image acquisition module, configured to perform real-time image acquisition on a first traffic facility in the traffic safety facility distribution network through a pre-configured intelligent camera to generate a first facility image time series; an anomaly recognition module, configured to perform status anomaly recognition on the first traffic facility based on the first facility image time series to generate a first anomaly recognition result; and an anomaly reminder module, configured to send the first anomaly recognition result to associated maintenance personnel for reminder.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0008] The facility distribution determination module constructs a traffic safety facility distribution network, uses the image acquisition module to collect facility images to generate time-series data, the anomaly recognition module performs anomaly recognition for different types of facilities based on the image time series, and the anomaly reminder module analyzes the impact of the abnormal facilities and sorts them by priority and then reminds the maintenance personnel, so as to realize the intelligent inspection of the road area traffic safety facilities, timely and accurately discover facility anomalies, meet the requirements of efficient management of the road area traffic safety facilities, improve the road traffic safety guarantee level, achieve the intelligent and real-time inspection of the road area traffic safety facilities, quickly and accurately locate facility anomalies and timely remind the maintenance personnel to handle them, and effectively eliminate road safety hazards. Brief Description of the Drawings

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for description in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0010] Figure 1 It is a schematic structural diagram of a new type of intelligent inspection terminal for road area traffic safety facilities provided by an embodiment of the present application.

[0011] Figure 2 It is a schematic structural diagram of the anomaly recognition module in a new type of intelligent inspection terminal for road area traffic safety facilities provided by an embodiment of the present application.

[0012] Description of the reference numerals: The facility distribution determination module 10, the image acquisition module 20, the anomaly recognition module 30, the anomaly reminder module 40. Detailed Embodiments

[0013] The present application provides a new type of intelligent inspection terminal for road area traffic safety facilities, which is used to solve the technical problem that traditional inspection methods are difficult to quickly and accurately discover anomalies in traffic safety facilities, thus resulting in the inability to timely eliminate road safety hazards.

[0014] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0015] Embodiment, as Figure 1 shown, the present application provides a new type of intelligent inspection terminal for road area traffic safety facilities, and the terminal includes:

[0016] The facility distribution determination module 10 is used to determine the traffic safety facility distribution network within a preset road area.

[0017] In the embodiment of the present application, the traffic safety facility distribution network is a network used to comprehensively present the distribution of traffic safety facilities in a preset road area.

[0018] Specifically, the distribution locations of static traffic safety facilities in a preset road area are collected to generate a static distribution network, and then the distribution locations of dynamic traffic safety facilities are collected to generate a dynamic distribution network. Finally, a two-layer facility network is constructed to combine the two to generate the entire traffic safety facility distribution network. The specific steps are described in detail in the static, dynamic and traffic network generation units.

[0019] The image acquisition module 20 is used to acquire real-time images of the first traffic facility in the traffic safety facility distribution network through a pre-configured intelligent camera to generate a first facility image time sequence.

[0020] In the embodiment of the present application, the first traffic facility is the image acquisition object of the intelligent patrol terminal, including static traffic safety facilities and dynamic traffic safety facilities. The first facility image sequence is a series of image sets formed by chronologically arranging the first traffic facility image in real time through the intelligent camera.

[0021] Specifically, first, based on the traffic safety facility distribution network constructed by the facility distribution determination module 10, the specific location information of the first traffic facility is clarified and accurate to the specific longitude and latitude coordinates (such as a traffic sign is located at [X] degrees north latitude and [Y] degrees east longitude), which facilitates the deployment and precise positioning of smart cameras.

[0022] The layout of the smart camera is determined based on the specific latitude and longitude coordinates of the first traffic facility. In terms of location selection, it is necessary to ensure that the first traffic facility can be clearly photographed. For example, for traffic signs, they may be arranged at a suitable distance in front of or to the side of them; for dynamic traffic facilities, such as variable information signs, technicians in this field can set them up at reasonable locations around them according to their range of activities. In terms of angle, adjust to an angle that can fully display the entire facility without obstruction. For flat facilities, the lens should be as perpendicular to the surface of the facility as possible. For facilities with a certain height and three-dimensional structure, choose a suitable elevation or depression angle. In terms of quantity, technicians in this field will determine it based on the complexity of the facility and the monitoring requirements. A simple single facility may only require one camera to meet the monitoring requirements, while for large or important traffic facilities, such as complex intersection signal light groups, multiple cameras are required to shoot from different angles (for example, one camera is arranged above and one camera is arranged on the side).

[0023] Next, configure the deployed intelligent cameras according to the pre-set parameters. For example, set the shooting resolution to high definition (1080P and above) to ensure that the details of traffic facilities can be clearly presented in the captured images; set the shooting frame rate, such as 25 frames per second, to ensure that the obtained image sequence has high coherence for subsequent status analysis.

[0024] Then, the intelligent cameras start the real-time shooting function and continuously shoot the first traffic facility at the positions and angles set by those skilled in the art. During the shooting process, using network transmission technology (such as 4G / 5G network, whose transmission rate can meet the real-time transmission requirements of high-definition images), transmit the captured images to the data storage and processing system of the terminal in real time.

[0025] In the data storage and processing system, number and store the received images in chronological order. Each image is marked with an accurate timestamp (accurate to the millisecond level), thereby generating the time sequence of the first facility images.

[0026] By generating the time sequence of the first facility images, the status of the first traffic facility at different times is completely recorded, providing rich and continuous data support for subsequent anomaly recognition and analysis.

[0027] The anomaly recognition module 30 is used to recognize the status anomaly of the first traffic facility based on the time sequence of the first facility images and generate the first anomaly recognition result.

[0028] In the embodiment of the present application, the first anomaly recognition result is the conclusion obtained by the anomaly recognition module 30 based on the time sequence of the first facility images after recognizing the status anomaly of the first traffic facility.

[0029] Specifically, first identify the type of the first traffic facility to obtain the first facility type. If it is a static traffic safety facility, perform image quality analysis on each frame of the time sequence of the first facility images, extract the frame with the highest quality to identify the status anomaly, and then generate the first anomaly recognition result. The specific steps are described in detail in the facility type and anomaly result generation unit.

[0030] The anomaly reminder module 40 is used to send the first anomaly recognition result to the associated maintenance personnel for reminder.

[0031] Specifically, first read multiple facilities showing anomalies within the first preset time, then perform anomaly impact analysis on these anomalous facilities to generate multiple anomaly impact indicators, and finally rank the multiple anomalous facilities according to these indicators and send them to the associated maintenance personnel for reminder. The specific steps are described in detail in the anomalous facility reading unit, anomaly indicator generation unit, and personnel reminder unit.

[0032] In a possible implementation manner, the facility distribution determination module 10 further includes:

[0033] A static network generation unit, configured to collect the distribution positions of static traffic safety facilities within a preset road area and generate a static traffic safety facilities distribution network; a dynamic network generation unit, configured to collect the distribution positions of dynamic traffic safety facilities within a preset road area and generate a dynamic traffic safety facilities distribution network; a traffic network generation unit, configured to construct a two-layer facility network based on the static traffic safety facilities distribution network and the dynamic traffic safety facilities distribution network and generate the traffic safety facilities distribution network.

[0034] Specifically, first, for the static traffic safety facilities within a preset road area, such as guardrails, road markings, traffic signs, etc., because their positions are relatively fixed, a high-precision Global Positioning System (GPS) device with a positioning accuracy of up to meter level or even higher is used to collect the position of each static facility. Taking traffic signs as an example, during measurement, the GPS receiver is placed at the center position of the bottom of the sign to obtain its accurate longitude and latitude coordinate data. The collected coordinate data of numerous static facilities are integrated and processed through Geographic Information System (GIS) technology to construct a static traffic safety facilities distribution network. In this network, the position of each static facility can be accurately marked on the electronic map, facilitating subsequent viewing and management.

[0035] Next, for dynamic traffic safety facilities, such as traffic lights whose states change, sensors with positioning functions are used and combined with Internet of Things technology to collect their distribution positions. For example, a positioning sensor is installed in the control box of the traffic light. The sensor can obtain the position information of the traffic light in real time and transmit the data to the terminal management platform (a system for receiving, sorting, analyzing, and storing the data of dynamic traffic safety facilities transmitted by the Internet of Things) at regular time intervals (such as every 5 minutes) through the Internet of Things. The management platform generates a dynamic traffic safety facilities distribution network based on the received time-series position data, thereby realizing real-time tracking of the position changes of dynamic facilities.

[0036] Finally, the static traffic safety facilities distribution network and the dynamic traffic safety facilities distribution network are fused. Through an algorithm based on a fusion algorithm of spatial index and association rules, based on the spatial information of the road, the distribution data of static and dynamic facilities are associated to construct a two-layer facility network.

[0037] Step a: For the data in the static traffic safety facility distribution network and the dynamic traffic safety facility distribution network, construct spatial indexes respectively. For static facilities, using the quadtree spatial index algorithm, based on the geographical locations (latitude and longitude coordinates) of static facilities, first regard the entire preset road area as a large rectangular range. Then, recursively divide this rectangular area into four equal sub-areas, namely four quadrants. Each quadrant becomes a node. If the number of static facilities contained in a certain quadrant exceeds the set threshold or the quadrant can still be further divided, perform the four-equal-division operation on it again, and so on. During the division process, each grid (i.e., node) corresponds to static facilities within a certain range, and finally a quadtree structure is constructed. Through this tree, the location information of static facilities can be quickly located and queried, improving the management and retrieval efficiency of static facility location data.

[0038] Step b: For dynamic facilities, according to their location change range and time series, adopt a spatio-temporal index structure, such as TB-tree (Time-Based tree), to associate the locations of dynamic facilities with time. First, use the location change range and time series of dynamic facilities as the basic data. Then, create a TB-tree structure and insert the location information of dynamic facilities at different time points into the tree. During the insertion process, determine the node location of the facility in the tree according to its time and space attributes. The nodes of the tree are divided according to time and space ranges. For example, the upper-level nodes represent larger time intervals and space ranges, while the lower-level nodes are subdivided into more precise time points and smaller space areas. In this way, when it is necessary to obtain the distribution of dynamic facilities at a specific time point or time period, by searching according to the time attribute in the TB-tree, the corresponding node can be quickly located, and then the location information of dynamic facilities within this time range can be obtained, realizing the effective association of location and time.

[0039] Step c: Make associations based on the spatial information of the road. Extract information such as the centerline, boundary, and section division of the road from the road database (a set storing road-related information, including the spatial information of the road, such as the centerline, boundary, and section division data of the road), and match the distribution data of static and dynamic facilities with the road information. For static facilities, determine which road section they are located on and associate their relevant attributes (such as facility type, specifications, etc.) with the road section information; for dynamic facilities, also determine the road sections they are in at different time points and establish the corresponding relationship between the location changes of dynamic facilities and the road sections.

[0040] Step d: Perform data fusion according to the association rules. First, set the rules that if static facilities and dynamic facilities are within the same road section range and there is an intersection in time (for example, the dynamic facility passes through the section containing the static facility within a certain time period), they are regarded as associated objects. By traversing the spatial index and spatio-temporal index, find all pairs of static and dynamic facilities that meet the association rules, and integrate their distribution data. For example, merge the relevant data of static traffic signs and dynamic variable message signs passing through the same road section to construct a record containing the fixed information of static facilities and the real-time or dynamically changing information of dynamic facilities.

[0041] Step e: Construct a two-layer facility network based on these integrated records. Take the road section as the basic node of the network, and connect the static facility and dynamic facility information within the section under each node to form a hierarchical network structure. In this two-layer facility network, the upper layer shows the overall layout of the road section and the associated static facilities, and the lower layer presents in detail the state changes of dynamic facilities at different times for each section, finally generating a comprehensive and accurate traffic safety facility distribution network.

[0042] Through the above steps, it provides key data support for functions such as image acquisition and anomaly recognition of subsequent intelligent inspection terminals, which helps to improve the management efficiency of road traffic safety facilities.

[0043] In a possible implementation manner, the static network generation unit further includes:

[0044] The static traffic safety facilities include traffic safety facilities with fixed facility status, and the dynamic traffic safety facilities include traffic safety facilities with dynamically changing facility status.

[0045] Specifically, the static traffic safety facilities include guardrails, road markings, traffic signs, etc.

[0046] For the position measurement of the guardrail, first use GPS to obtain the approximate position information of the guardrail, and then use a total station or laser rangefinder to measure the distance and angle between each key point of the guardrail and the control point starting from the known control point, so as to determine the position and orientation of the guardrail; for the position measurement of the road markings, use a road marking measurement vehicle with positioning function. During the driving process of the vehicle, the vehicle's driving trajectory and distance are recorded in real time by combining the in-vehicle GPS device and the odometer, so as to determine the position information of the road markings; for the position measurement of the traffic signs, it is the same as the implementation steps of the static network generation unit.

[0047] The dynamic traffic safety facilities include traffic lights, variable message signs, etc.

[0048] Position measurement of traffic lights and implementation steps of the dynamic network generation unit; position measurement of variable message signs, similar to traffic lights, install positioning sensors on variable message signs, and transmit position information to the management platform in real time through the Internet of Things to achieve dynamic monitoring of their positions.

[0049] In a possible implementation manner, as Figure 2 shown, the anomaly recognition module 30 further includes:

[0050] A facility type generation unit, configured to identify the facility type of the first traffic facility and generate a first facility type; an anomaly result generation unit, configured to judge the first facility type. If the first facility type is a static traffic safety facility, perform image quality analysis on each frame of the first facility image time series, extract the frame of image with the highest image quality for state anomaly recognition, and generate the first anomaly recognition result.

[0051] In the embodiments of the present application, the first facility type is the type to which the traffic facility belongs identified during the state anomaly recognition of the first traffic facility, and is divided into static traffic safety facilities and dynamic traffic safety facilities.

[0052] Specifically, first, identify the facility type of the first traffic facility. After the intelligent camera of the intelligent patrol terminal captures an image of the first traffic facility, the data storage and processing system analyzes features such as the shape, color, and pattern of the facility in the image. For example, by identifying the shape (round, triangular, rectangular, etc.), color (red, yellow, blue, etc.) of the traffic sign and the text or pattern information on it, and combining with a preset traffic facility type database (obtained from the specification standard documents of different regions and different types of traffic facilities), accurately determine whether the facility is a traffic sign, a guardrail, or other types of traffic safety facilities.

[0053] When the first facility type is determined to be a static traffic safety facility, the anomaly result generation unit performs image quality analysis on each frame of the first facility image time series. First, the evaluation sequence generation subunit performs multi-dimensional quality index evaluation on each frame of image to generate a first multi-dimensional evaluation result sequence; then the image sequence generation subunit performs weighted fusion of multi-dimensional quality index values on any multi-dimensional evaluation result in the sequence to generate a first image quality index sequence; finally, the state anomaly recognition subunit locates the frame of image with the highest image quality in the sequence for state anomaly recognition, and the specific steps are described in detail in the evaluation sequence, the image sequence generation subunit, and the state anomaly recognition subunit.

[0054] For example, for traffic signs, it is determined whether they are damaged by detecting whether the outlines of the signs in the image are complete; whether they have collapsed is judged by analyzing the position and angle of the signs in the image and combining the surrounding environmental information; object segmentation technology is used to identify the objects around the signs to determine whether they are blocked by sundries. If any of these abnormal situations are found, corresponding first abnormal recognition results are generated, providing an important basis for subsequent facility maintenance.

[0055] By identifying the facility type of the first traffic facility, the technical effect of identifying the abnormal state of the first traffic facility is achieved.

[0056] In a possible implementation manner, the abnormal result generation unit further includes:

[0057] An evaluation sequence generation subunit, configured to evaluate multi-dimensional quality metrics for each frame of the first facility image sequence in time series, and generate a first multi-dimensional evaluation result sequence; an image sequence generation subunit, configured to perform weighted fusion of multi-dimensional quality metric values for any one of the multi-dimensional evaluation results in the first multi-dimensional evaluation result sequence, and generate a first image quality metric sequence; a state abnormality recognition subunit, configured to perform state abnormality recognition by locating a frame of image with the highest image quality metric based on the first image quality metric sequence.

[0058] In the embodiment of the present application, the first multi-dimensional evaluation result sequence is a result sequence generated by evaluating multi-dimensional quality metrics for each frame of the first facility image sequence in time series. The first image quality metric sequence is a sequence generated by performing weighted fusion of multi-dimensional quality metric values for any one of the multi-dimensional evaluation results in the first multi-dimensional evaluation result sequence.

[0059] Specifically, first, after the intelligent camera captures the images of the first traffic facility, a first facility image sequence in time series is formed. For each frame of the sequence, multi-dimensional quality metrics are evaluated to generate a first multi-dimensional evaluation result sequence. The specific steps will be described in detail in the following content of the multi-dimensional quality metrics.

[0060] Then, according to the importance of different metrics in reflecting the image quality, weights are assigned to each metric (determined by those skilled in the art according to experience and actual scenarios). For example, in some scenarios, image sharpness is more critical for judging the abnormal state of traffic facilities, and a higher weight of 0.4 can be assigned to it; the weight of image information entropy is set to 0.2, the weight of peak signal-to-noise ratio is set to 0.2, the weight of structural similarity index is set to 0.1, and the weight of contrast is set to 0.1. The values of each metric are multiplied by the corresponding weights and then added together to generate the first image quality metric sequence.

[0061] Next, based on the first image quality metric sequence, find the maximum value in the sequence and locate the frame of the image with the highest image quality metric. This frame of the image can most accurately reflect the actual state of the traffic facilities.

[0062] Finally, use an image recognition algorithm to analyze this frame of the image to determine whether there is an abnormal state in the traffic facilities, such as whether the traffic signs are damaged or deformed, and whether the markings are blurred.

[0063] Step f: Traffic sign anomaly judgment. First, establish a database containing the characteristics of various normal traffic signs. These characteristics include color characteristics (e.g., red represents a prohibition sign, blue represents an indication sign, etc.), shape characteristics (circle, triangle, rectangle, etc.), and pattern characteristics (e.g., speed limit numbers, arrow patterns, etc.).

[0064] After obtaining the frame of the image with the highest image quality, perform preprocessing operations such as grayscale conversion and noise reduction on it to improve the accuracy of subsequent feature extraction. Then, use an edge detection algorithm to extract the edge contour of the traffic signs in the image, such as the Canny edge detection algorithm. The specific process is to calculate the gradient magnitude and direction of the preprocessed image to determine the intensity and direction of pixel changes in the image. Then, through non-maximum suppression, refine the edge lines and remove possible false edges. Finally, use double-threshold detection and edge connection. According to the set high and low thresholds, retain the strong edges, and also retain the weak edges when they are connected to the strong edges, so as to obtain an accurate edge contour of the traffic signs, and then preliminarily judge the type of traffic signs (circle, triangle, rectangle, etc.) according to the contour shape.

[0065] Then, for the determined sign type, retrieve the corresponding normal sign characteristics from the feature database. Taking the color characteristic as an example, by calculating the color histogram of the sign area in the image and comparing it with the normal sign color histogram, calculate the similarity between the two. If the similarity is lower than a certain threshold, there may be a color anomaly, such as fading. For the pattern characteristic, match the extracted sign pattern with the standard pattern in the database and calculate the matching degree. If the matching degree is low, it indicates that the pattern may be damaged or deformed. Based on the comprehensive comparison results of multiple characteristics such as color, shape, and pattern, judge whether there is an anomaly in the traffic signs.

[0066] Step g: Abnormal marking line judgment, using the U-Net semantic segmentation algorithm (U-Net Convolutional Networks for Biomedical Image Segmentation, a U-shaped convolutional neural network for biomedical image segmentation). Similarly, first collect images containing clear and blurred marking lines to build a dataset, and perform pixel-level annotation on the marking lines. The U-Net network has an encoder and a decoder structure. The encoder gradually extracts high-level features of the image through convolution and pooling operations, and the decoder restores the high-level features to the original image size through upsampling and convolution operations, and combines the features of the encoder to achieve the classification of each pixel in the image. When processing the marking line image, the U-Net network will segment the marking line from the background to obtain the pixel-level mask of the marking line. The blurriness of the marking line is judged by calculating the edge gradient value and line continuity of the marking line mask. If the edge gradient value is lower than the set threshold, it indicates that the edge of the marking line is not clear; if there are many discontinuities and breaks in the line, it can also be judged that there is an abnormality in the marking line. For example, in an image of a lane dividing line, if the edge of the segmented marking line is blurred and the line has multiple interruptions, it can be determined that the marking line is blurred.

[0067] Through the above steps, the accurate identification of the abnormal state of the first traffic facility is realized, and the detection accuracy and efficiency are improved compared with the traditional method.

[0068] In a possible implementation manner, the evaluation sequence generation subunit further includes:

[0069] The multi-dimensional quality indicators include image information entropy, peak signal-to-noise ratio, structural similarity index, image sharpness, and contrast.

[0070] Specifically, the evaluation of image information entropy depends on the distribution of gray values in the image. Image information entropy is used to measure the amount of information contained in the image. Regarding the first facility image as a set composed of numerous pixels, each pixel has its specific gray value. First, count the frequency of each gray value appearing in the image, and then calculate the probability of each gray value appearing. When the types of gray values in the image are rich and the distribution is relatively uniform, it means that the number of pixels with different gray values varies little. At this time, the image contains more information, and the calculated image information entropy is higher. For example, an image taken of a panoramic view of a traffic intersection, which contains vehicles, pedestrians, various traffic signs and marking lines, has diverse gray values of its pixels, and such an image will have a relatively high information entropy; on the contrary, if the image content is single, such as only a solid color area with very few gray value changes, the information entropy will be very low.

[0071] Peak Signal-to-Noise Ratio (PSNR) Evaluation: The peak signal-to-noise ratio mainly measures whether distortion occurs during image processing. In practical applications, the acquired image is compared with the original high-quality reference image (assuming there is an ideal distortion-free image as a reference). The distortion situation is evaluated by comparing the gray value differences of corresponding pixel points in the images. If the gray value differences between the acquired image and the reference image are small, it indicates that the distortion degree of the image during acquisition, transmission, or processing is low, the value of the peak signal-to-noise ratio will be high, which also means better image quality. For example, if the pixel gray difference between the acquired traffic sign image and the standard traffic sign image is small, its peak signal-to-noise ratio will be relatively high, and the image quality is more guaranteed.

[0072] Structural Similarity Index (SSIM) Analysis: The structural similarity index focuses on the structural features of the image. It compares the acquired image with the original reference image from three aspects: brightness, contrast, and structure. For brightness, the average brightness differences of corresponding regions in the two images are compared; for contrast, it examines whether the brightness change degrees in different regions of the image are similar; in terms of structure, the similarity degree of structural information such as the edges and contours of objects in the image is analyzed for comprehensive evaluation. When the structural similarity index is closer to 1, it indicates that the acquired image and the reference image are more similar in structure, and the image quality is higher. For example, for a traffic marking image, if it is highly similar to the standard marking image in terms of shape, line distribution, and other structural features, the structural similarity index will be close to 1.

[0073] Image Sharpness Calculation: An edge detection algorithm is used to calculate the image sharpness (the algorithm process is the same as in step f). The edge detection algorithm will find the places where the pixel gray values in the image change sharply, which usually correspond to the edges of objects. When the edges in the image are clear and sharp, it indicates obvious pixel changes and high image sharpness. For example, when detecting the contour of traffic facilities, if the edge lines of traffic facilities in the image are clear and there is no blur or blurring, the sharpness index obtained through the edge detection algorithm will be relatively high, otherwise it will be low.

[0074] Contrast Determination: Contrast reflects the difference degree between different gray levels in the image. When calculating the contrast, the gray differences between the brightest region and the darkest region in the image, as well as the distribution of different gray levels, are analyzed. Appropriate contrast can make the details of the image clearer. For example, the text and patterns on traffic signs are more easily recognizable in an image with high contrast. If the contrast of the image is low, the image will appear blurred and lack a sense of hierarchy, which is not conducive to judging the state of traffic facilities.

[0075] By sequentially evaluating the quality indicators of each frame of image in the above five dimensions, a first multi-dimensional evaluation result sequence is finally generated, providing basic data for subsequent comprehensive image quality evaluation and traffic facility state judgment.

[0076] In a possible implementation manner, the abnormal result generating unit further includes:

[0077] An analysis result generating subunit, configured to perform abnormal analysis on state changes of multiple consecutive images in the first facility image time series if the first facility type is a dynamic traffic safety facility, and generate a first state change abnormal analysis result; an identification result generating subunit, configured to extract any one frame of the multiple consecutive images for static state abnormal identification, and generate a first static abnormal identification result; an abnormal result generating subunit, configured to generate the first abnormal identification result based on the first state change abnormal analysis result and the first static abnormal identification result.

[0078] In the embodiment of the present application, the first state change abnormal analysis result is a result generated by performing abnormal analysis on state changes of multiple consecutive images in the first facility image time series when the first facility type is a dynamic traffic safety facility. The first static abnormal identification result is a result generated by extracting any one frame of the multiple consecutive images in the first facility image time series for static state abnormal identification.

[0079] Specifically, first, perform abnormal analysis on state changes of multiple consecutive images in the first facility image time series. Taking a traffic light as an example, an intelligent camera continuously collects images of the traffic light to form an image time series, and the terminal judges whether the change time and sequence are normal by analyzing the change situation of the traffic light color in adjacent images. Assume that the time interval for the green light to turn red under normal circumstances is x seconds (which is set by those skilled in the art according to the actual situation of different intersections). If it is detected in the image time series that the duration of the green light is much greater than or less than x seconds, or there is a situation that does not conform to the normal switching sequence of red, green, and yellow, such as the green light directly turning into the yellow light, it is recorded as an abnormal situation, and the first state change abnormal analysis result is generated by integrating this information.

[0080] Next, randomly extract any frame of image from multiple consecutive frames of images for static state anomaly recognition. First, preprocess the collected images containing traffic lights, that is, enhance the contrast of the images through gray-scale transformation to make the features of the traffic lights more obvious; use algorithms such as median filtering to remove the noise in the images (select a pixel point on the image, set a filtering window centered on this pixel point, such as a 3×3 or 5×5 matrix window, sort the gray-scale values of all pixels within the window, and take the median value to replace the gray-scale value of the central pixel point. Traverse each pixel point in the image in this way to complete the noise removal of the image). Next, adopt an edge detection algorithm, such as the Canny algorithm, to extract the edge contour information of the traffic lights (the same as in step f). According to the common shapes of traffic lights, such as circular, square, etc., use a shape matching algorithm to compare the extracted edge contour with the standard traffic light shape model, calculate the similarity between the two. If the similarity is lower than the preset threshold, it may mean that the lampshade of the traffic light is damaged, resulting in an irregular shape (the same as in step f).

[0081] In terms of color analysis, converting the image from the RGB color space to the HSV space is more convenient for analyzing color features. By statistically analyzing the color distribution within the traffic light area, calculate statistical quantities such as the mean and variance of the color. Compare with the HSV value range of the standard traffic light color. If the color mean exceeds the normal range or the color variance is too large, it indicates that the color is off. For the judgment of abnormal brightness, analyze in the V channel (representing brightness) in the HSV space, calculate the average brightness value of the traffic light area, and compare it with the standard brightness value. If the deviation exceeds a certain range, it is determined that the brightness is abnormal. Combining the analysis results of shape, color, and brightness, if there are situations such as irregular shape, color deviation, or abnormal brightness, it is determined that the static state is abnormal, and then the first static anomaly recognition result is generated.

[0082] Finally, synthesize the first state change anomaly analysis result and the first static anomaly recognition result. If any one of them is abnormal, the first anomaly recognition result can be generated, indicating that the dynamic traffic safety facility (such as a traffic light) is in an abnormal state and needs to be maintained or repaired.

[0083] Through the above steps, the abnormal conditions of dynamic traffic safety facilities can be detected in a timely manner to ensure road traffic safety.

[0084] In a possible implementation manner, the anomaly reminder module 40 further includes:

[0085] An abnormal facility reading unit for reading multiple abnormal facilities with display anomalies within a first preset time; an abnormal index generation unit for performing an abnormal impact analysis on the multiple abnormal facilities to generate multiple abnormal impact indexes; a personnel reminder unit for sending the multiple abnormal facilities to the associated maintenance personnel for reminder after prioritizing them based on the multiple abnormal impact indexes.

[0086] In the embodiment of the present application, the first preset time is a time range for setting the reading of multiple abnormal facilities with display anomalies. The abnormal impact index is an index generated after performing an abnormal impact analysis on multiple abnormal facilities.

[0087] Specifically, when the first abnormal recognition result is generated, the system first reads the information of multiple abnormal facilities with display anomalies within the first preset time. Those skilled in the art can flexibly set the preset time according to actual needs. For example, it can be set to the past 24 hours, which can ensure that recent anomalies can be processed in a timely manner.

[0088] Then, an abnormal impact analysis is performed on these abnormal facilities. Taking traffic signs and traffic lights as examples, for traffic signs, if the speed limit sign at an important intersection is abnormal, considering the traffic flow data at this intersection (assuming the traffic flow per hour is 500 vehicles) and the vehicle speed situation (average vehicle speed of 60 km / h), if the speed limit sign is damaged, it may cause drivers to be unable to obtain accurate speed limit information, thereby increasing the risk of vehicle speeding. According to the historical data statistics of the local traffic management department, the probability of accidents caused by speeding on this section of the road will increase by 30%. Thus, it is determined that its abnormal impact index is relatively high; for traffic lights, if a certain traffic light fails, the traffic flow data at the intersection where the traffic light is located is statistically analyzed (such as 80 vehicles passing through per minute during the morning rush hour). When the traffic light fails, the average traffic congestion time at the intersection is extended by 20 minutes. This not only causes a significant decrease in vehicle passing efficiency but also may lead to traffic accidents, thereby obtaining its abnormal impact index.

[0089] Finally, based on the calculated multiple abnormal impact indexes, the system will prioritize the multiple abnormal facilities. The higher the abnormal impact index, the greater the impact of the abnormality of the facility on traffic safety and traffic smoothness, and the higher its priority. For example, the abnormal traffic signs at the above-mentioned important intersections and the abnormal traffic lights at intersections with large traffic flows will be ranked at a higher priority due to their high impact indexes. After the ranking is completed, the system sends the situation of these abnormal facilities with priority information to the associated maintenance personnel for reminder, so that the maintenance personnel can, according to the priority order, give priority to handling the abnormal problems of facilities that have a greater impact on traffic safety and efficiently ensure road traffic safety.

[0090] By prioritizing abnormal facilities, the technical effect of specifically reminding relevant maintenance personnel of the situation of abnormal facilities is achieved, so that they can efficiently handle the abnormalities.

[0091] It should be noted that the above order of the embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of this specification have been described. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0092] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

[0093] This specification and the drawings are only exemplary descriptions of the present application and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these changes and modifications.

Claims

1. A new type of intelligent inspection terminal for road traffic safety facilities, characterized in that: include: A facility distribution determination module is used to determine the traffic safety facility distribution network within a preset road area; An image acquisition module, used to acquire real-time images of a first traffic facility in the traffic safety facility distribution network through a pre-configured smart camera, and generate a time sequence of images of the first facility; an abnormality identification module, used to identify the abnormal state of the first traffic facility based on the time sequence of the first facility image, and generate a first abnormality identification result; The abnormality reminder module is used to send the first abnormality identification result to the associated maintenance personnel for reminder.

2. A new type of intelligent inspection terminal for road traffic safety facilities as claimed in claim 1, characterized in that: The facility distribution determination module includes: A static network generation unit, used to collect the distribution positions of static traffic safety facilities in a preset road area and generate a static traffic safety facility distribution network; A dynamic network generation unit, used to collect the distribution positions of dynamic traffic safety facilities in a preset road area and generate a dynamic traffic safety facility distribution network; The traffic network generation unit is used to construct a double-layer facility network with the static traffic safety facility distribution network and the dynamic traffic safety facility distribution network to generate the traffic safety facility distribution network.

3. A new type of intelligent inspection terminal for road traffic safety facilities as claimed in claim 2, characterized in that: The static traffic safety facilities include traffic safety facilities with fixed facility status, and the dynamic traffic safety facilities include traffic safety facilities with dynamically changing facility status.

4. A new type of intelligent inspection terminal for road traffic safety facilities as claimed in claim 1, characterized in that: The abnormality identification module also includes: a facility type generating unit, configured to identify the facility type of the first transportation facility and generate a first facility type; An abnormal result generating unit is used to judge the first facility type. If the first facility type is a static traffic safety facility, an image quality analysis is performed on each frame image in the first facility image time sequence, and the frame image with the highest image quality is extracted for state abnormality identification to generate the first abnormality identification result.

5. A new type of intelligent inspection terminal for road traffic safety facilities as claimed in claim 4, characterized in that: The abnormal result generating unit further includes: An evaluation sequence generating subunit, configured to evaluate the multi-dimensional quality index of each frame of the first facility image sequence, and generate a first multi-dimensional evaluation result sequence; an image sequence generating subunit, configured to perform weighted fusion of multi-dimensional quality index values ​​on any multi-dimensional evaluation result in the first multi-dimensional evaluation result sequence to generate a first image quality index sequence; The abnormal state identification subunit is used to locate the frame of image with the highest image quality index based on the first image quality index sequence to perform abnormal state identification.

6. A new type of intelligent inspection terminal for road traffic safety facilities as claimed in claim 5, characterized in that: The multi-dimensional quality indicators include image information entropy, peak signal-to-noise ratio, structural similarity index, image clarity and contrast.

7. A new type of intelligent inspection terminal for road traffic safety facilities as claimed in claim 4, characterized in that: The abnormal result generating unit further includes: an analysis result generating subunit, configured to, if the first facility type is a dynamic traffic safety facility, perform state change anomaly analysis on a plurality of continuous frames of images in the first facility image time sequence to generate a first state change anomaly analysis result; A recognition result generating subunit, used for extracting any frame of the plurality of continuous images to perform static state abnormality recognition and generate a first static state abnormality recognition result; The abnormal result generating subunit is used to generate the first abnormality recognition result based on the first state change abnormality analysis result and the first static abnormality recognition result.

8. The novel intelligent inspection terminal for road traffic safety facilities according to claim 1, characterized in that: The abnormality reminder module also includes: An abnormal facility reading unit, used for reading a plurality of abnormal facilities showing abnormalities within a first preset time; An abnormality index generating unit, used for performing abnormality impact analysis on the plurality of abnormal facilities and generating a plurality of abnormality impact indicators; A personnel reminder unit is used to prioritize the multiple abnormal facilities based on the multiple abnormal impact indicators and then send them to the associated maintenance personnel for reminder.

Images