A city expressway sound barrier hidden danger analysis method based on video analysis

Abstract

The application belongs to the field of video recognition, and discloses a city expressway sound barrier hidden danger analysis method based on video analysis, which comprises the following steps: S1, shooting the sound barrier on the roadside of the city expressway to obtain a patrol video containing the sound barrier; S2, obtaining video frames for image recognition from the patrol video and saving the obtained video frames to a to-be-recognized set; S3, respectively inputting each video frame in the to-be-recognized set into a pre-trained neural network model for recognition to obtain the recognition result of the video frame; and S4, saving the recognition result to a result set. The application can greatly reduce the probability of missed detection, so that the safety hidden danger of the sound barrier can be found in time and comprehensively when the sound barrier has a safety hidden danger.

Description

Technical Field

[0001] This invention relates to the field of video recognition, and more particularly to a method for analyzing the potential hazards of sound barriers on urban expressways based on video analysis. Background Technology

[0002] A sound barrier is a specially designed acoustic barrier placed between a noise source and a noise receiver. Because some urban expressways are located close to buildings, sound barriers are typically installed on both sides of urban expressways to reduce the impact of noise on buildings near the expressway.

[0003] Sound barriers are affected by factors such as temperature differences, rain, strong winds, and road vibrations during use. Therefore, the structure of the sound barriers may deform, rust, or have parts missing, potentially leading to safety hazards. Regular inspections of sound barriers are necessary, and any barriers exhibiting safety risks should be repaired or replaced promptly.

[0004] The traditional inspection method involves visually inspecting the sound barrier to determine if there are any safety hazards. However, this method is prone to missed inspections due to eye fatigue. Summary of the Invention

[0005] The purpose of this invention is to disclose a method for analyzing potential hazards of urban expressway sound barriers based on video analysis, and to solve the problem of how to reduce the probability of missed inspections during sound barrier inspections.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] This invention provides a method for analyzing the potential hazards of noise barriers on urban expressways based on video analysis, including:

[0008] S1, to film the sound barrier located on the side of the urban expressway and obtain the inspection video containing the sound barrier;

[0009] S2, Obtain video frames for image recognition from the inspection video and save the obtained video frames to the set to be recognized;

[0010] S3, each video frame in the set to be identified is input into the pre-trained neural network model for identification, and the identification result of the video frame is obtained;

[0011] S4, save the recognition results to the result set.

[0012] Preferably, the inspection video, which includes images of the noise barriers located along the roadside of an urban expressway, includes:

[0013] By using vehicles equipped with cameras to film the noise barriers along urban expressways, inspection videos containing the noise barriers are obtained. During the filming process, the vehicle speed information is recorded at fixed acquisition intervals to obtain a collection of speed information.

[0014] Preferably, the speed information includes speed and recording time.

[0015] Preferably, before obtaining the video frames for image recognition from the inspection video, the method further includes:

[0016] The video frames in the inspection video are numbered sequentially according to the time they were captured; the later the capture time, the larger the number.

[0017] Preferably, acquiring video frames from the inspection video for image recognition includes:

[0018] The first time the video is acquired, the first frame of the inspection video is used as the video frame for image recognition.

[0019] The process for the nth acquisition is as follows, where n is greater than or equal to 2:

[0020] The video frame obtained for image recognition in the (n-1)th iteration is quickly identified to obtain the distance between the camera and the sound barrier. ;

[0021] Obtain the number of the video frame used for image recognition obtained in the (n-1)th iteration. and shooting time ;

[0022] Get the time from recording time to shooting time Recent speed ;

[0023] The following function is used to calculate the number of the video frame used for image recognition in the nth inspection video. :

[0024]

[0025] in, Indicates standard distance. Indicates the calculated coefficient. , Indicates standard speed. This represents a set constant. , and These represent the lower and upper limits of the change in the number, respectively.

[0026] Preferably, the video frame obtained for image recognition in the (n-1)th iteration is quickly identified to obtain the distance between the camera and the sound barrier. ,include:

[0027] use This represents the video frame obtained in the (n-1)th iteration for image recognition.

[0028] right The process of rapid identification is as follows:

[0029] To each The edge coefficient of each pixel is calculated;

[0030] exist Delete pixels with edge coefficients less than a set edge coefficient threshold to obtain multiple connected regions, and save the obtained connected regions to a set. ;

[0031] Obtain each The length and width of each connected region in the array;

[0032] Remove connected regions whose length is less than a set length threshold from the set. Delete to get a set ;

[0033] Remove connected regions whose width is less than a set width threshold from the set. Delete to get a set ;

[0034] Get the collections respectively The area of ​​each connected region in the equation;

[0035] Sets obtained based on area The connected area of ​​the pillars belonging to the sound barrier;

[0036] Calculate the image distance between adjacent connected regions belonging to the sound barrier pillars;

[0037] The distance between the camera and the sound barrier is calculated based on the image distance. .

[0038] Preferably, the time between the recording time and the shooting time is obtained. Recent speed ,include:

[0039] The recorded times in the velocity information set are sorted in ascending order to obtain the set. ;

[0040] Calculate separately Each recorded moment in the data is related to... The absolute value of the difference between them;

[0041] The speed is the speed corresponding to the recording time corresponding to the absolute value of the smallest difference. .

[0042] Preferably, each video frame in the set to be identified is input into a pre-trained neural network model for identification to obtain the identification result of the video frame, including:

[0043] Each video frame in the set to be identified is preprocessed to obtain the video frame to be identified.

[0044] The video frames to be identified are input into a pre-trained neural network model for recognition, and the recognition results of the video frames are obtained.

[0045] Preferably, the identification result is that the sound barrier in the video frame has a safety hazard or does not have a safety hazard.

[0046] Preferably, when the identification result indicates that the sound barrier in the video frame poses a safety hazard, the identification result also includes the type of safety hazard present in the sound barrier in the video frame.

[0047] This invention, when inspecting sound barriers installed on urban expressways, first acquires inspection video containing the sound barriers, then obtains a set of images to be identified based on the inspection video, and finally identifies the video frames in the set to obtain the identification results. Compared with traditional inspection methods relying on human eyes, this invention, because it uses machine recognition, is not subject to fatigue, significantly reducing the probability of missed detections. Therefore, it can promptly and comprehensively detect safety hazards in sound barriers when they exist. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of a method for analyzing the potential hazards of urban expressway sound barriers based on video analysis, according to the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this application or its application or use. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0051] like Figure 1 As shown in one embodiment, the present invention provides a method for analyzing the potential hazards of sound barriers on urban expressways based on video analysis, including:

[0052] S1, to film the sound barrier located on the side of the urban expressway and obtain the inspection video containing the sound barrier;

[0053] S2, Obtain video frames for image recognition from the inspection video and save the obtained video frames to the set to be recognized;

[0054] S3, each video frame in the set to be identified is input into the pre-trained neural network model for identification, and the identification result of the video frame is obtained;

[0055] S4, save the recognition results to the result set.

[0056] This invention, when inspecting sound barriers installed on urban expressways, first acquires inspection video containing the sound barriers, then obtains a set of images to be identified based on the inspection video, and finally identifies the video frames in the set to obtain the identification results. Compared with traditional inspection methods relying on human eyes, this invention, because it uses machine recognition, is not subject to fatigue, significantly reducing the probability of missed detections. Therefore, it can promptly and comprehensively detect safety hazards in sound barriers when they exist.

[0057] Preferably, the inspection video, which includes images of the noise barriers located along the roadside of an urban expressway, includes:

[0058] By using vehicles equipped with cameras to film the noise barriers along urban expressways, inspection videos containing the noise barriers are obtained. During the filming process, the vehicle speed information is recorded at fixed acquisition intervals to obtain a collection of speed information.

[0059] Specifically, vehicles equipped with cameras include both manned and driverless vehicles.

[0060] When a vehicle equipped with a camera arrives at the area where the sound barrier is located, the camera begins to film the sound barrier. When the vehicle leaves the area where the sound barrier is located, the camera stops filming the sound barrier.

[0061] When filming the sound barrier from the vehicle, the vehicle travels in the lane closest to the sound barrier, maintaining the same distance from the sound barrier as much as possible, and keeping the vehicle speed as constant as possible.

[0062] Specifically, the fixed acquisition period value can be set according to the actual situation; for example, it can be 1 second or 0.1 seconds.

[0063] The vehicle's speed can be obtained either by connecting to the vehicle's computer or by using GPS.

[0064] Preferably, the speed information includes speed and recording time.

[0065] Preferably, before obtaining the video frames for image recognition from the inspection video, the method further includes:

[0066] The video frames in the inspection video are numbered sequentially according to the time they were captured; the later the capture time, the larger the number.

[0067] Specifically, in this invention, the video frame number is an integer.

[0068] Preferably, acquiring video frames from the inspection video for image recognition includes:

[0069] The first time the video is acquired, the first frame of the inspection video is used as the video frame for image recognition.

[0070] The process for the nth acquisition is as follows, where n is greater than or equal to 2:

[0071] The video frame obtained for image recognition in the (n-1)th iteration is quickly identified to obtain the distance between the camera and the sound barrier. ;

[0072] Obtain the number of the video frame used for image recognition obtained in the (n-1)th iteration. and shooting time ;

[0073] Get the time from recording time to shooting time Recent speed ;

[0074] The following function is used to calculate the number of the video frame used for image recognition in the nth inspection video. :

[0075]

[0076] in, Indicates standard distance. Indicates the calculated coefficient. , Indicates standard speed. This represents a set constant. , and These represent the lower and upper limits of the change in the number, respectively.

[0077] In this invention, image recognition is not performed on all video frames. Directly performing image recognition on all video frames would be time-consuming due to the high complexity of image recognition algorithms, resulting in a lengthy process for obtaining the safety hazard analysis and detection results for the sound barrier. Therefore, this invention obtains a set of images to be recognized by non-continuously extracting video frames from the inspection video. The images in this set are then identified to obtain the recognition results. Since the number of images in the set to be recognized is significantly smaller than the number of images in the inspection video, this invention achieves the safety hazard analysis and detection results for the sound barrier much faster.

[0078] Furthermore, in this invention, the increment of the video frame number used for image recognition is not uniform, but rather related to the distance between the camera and the sound barrier, and the speed closest to the recording time and the shooting time, because it is difficult to keep the vehicle speed and distance completely constant. Therefore, in this invention, the greater the distance between the camera and the sound barrier, and the lower the vehicle speed, the more... and The larger the difference between them, the higher the number. Video frames and numbers The lower the overlap between video frames, the better it is to reduce the number of video frames in the set to be identified while maintaining the quality of the video frames. This is because, when the speed is constant, the greater the distance, the larger the area of ​​the sound barrier captured by the camera. Therefore, the numbering of the next video frame used for image recognition can vary more significantly while ensuring that there is an overlapping area among adjacent image frames, thus reducing the number of video frames in the set to be identified. Furthermore, when the distance is constant, the lower the speed, the higher the overlap of video frames captured within the same second. In this case, the variation in the video frame numbering can be increased, thus ensuring that there is an overlapping area among adjacent image frames.

[0079] This invention allows the increment of the video frame number used for image recognition to vary with the distance between the camera and the sound barrier and the speed of the vehicle, thereby effectively reducing the number of video frames in the set to be recognized while ensuring that there is an overlapping area in adjacent image frames.

[0080] The constraint between standard speed and standard distance is to ensure that the video frame includes only the two pillars of the sound barrier.

[0081] Preferably, the video frame obtained for image recognition in the (n-1)th iteration is quickly identified to obtain the distance between the camera and the sound barrier. ,include:

[0082] use This represents the video frame obtained in the (n-1)th iteration for image recognition.

[0083] right The process of rapid identification is as follows:

[0084] To each The edge coefficient of each pixel is calculated;

[0085] exist Delete pixels with edge coefficients less than a set edge coefficient threshold to obtain multiple connected regions, and save the obtained connected regions to a set. ;

[0086] Obtain each The length and width of each connected region in the array;

[0087] Remove connected regions whose length is less than a set length threshold from the set. Delete to get a set ;

[0088] Remove connected regions whose width is less than a set width threshold from the set. Delete to get a set ;

[0089] Get the collections respectively The area of ​​each connected region in the equation;

[0090] Sets obtained based on area The connected area of ​​the pillars belonging to the sound barrier;

[0091] Calculate the image distance between adjacent connected regions belonging to the sound barrier pillars;

[0092] The distance between the camera and the sound barrier is calculated based on the image distance. .

[0093] This invention does not employ traditional image recognition algorithms to... Perform identification to obtain Instead of directly identifying the areas belonging to the sound barrier pillars, the system first obtains edge coefficients and then uses these coefficients to determine the connected regions. The length, width, and area of ​​these connected regions are then assessed, enabling rapid acquisition of the areas belonging to the sound barrier pillars and effectively improving the accuracy of distance measurement between the camera and the sound barrier. The speed.

[0094] Specifically, the columns are vertically erected along the side of urban expressways and are used to connect two sound insulation panels and / or sound absorption panels.

[0095] Preferably, respectively for The edge coefficient of each pixel is calculated, including:

[0096] right Perform grayscale conversion to obtain the processed image;

[0097] For pixel k in the processed image, the formula for calculating its edge coefficient is:

[0098]

[0099] This represents the edge coefficient of pixel k. This represents the grayscale value of pixel k. If pixel k is not the last column of pixels in the image being processed, then... This represents the grayscale value of the pixel to the right of pixel k. If pixel k is the last column of pixels in the image being processed, then... This represents the grayscale value of the pixel to the left of pixel k.

[0100] Specifically, the edge coefficient represents the difference in grayscale value between a pixel and its neighboring pixels. When a pixel is located at the edge of the area of ​​the pillars of the sound barrier, its edge coefficient will be significantly greater than that of pixels located inside the area of ​​the pillars of the sound barrier. This calculation method enables the rapid acquisition of pixels in the edge area.

[0101] Preferably, obtain respectively The length and width of each connected region in the data include:

[0102] For a connected region A, the length of the connected region A is obtained by subtracting the minimum value of the y-coordinate from the maximum value of the y-coordinate of the pixels in the connected region A.

[0103] The function for calculating the width is:

[0104]

[0105] in, This represents the width of the connected region A. This represents the row number of pixels in connected region A. and Let x and y represent the maximum and minimum x-coordinates of the pixel in the i-th row of the connected region A, respectively.

[0106] Specifically, in a video frame, the length of a connected region belonging to a pillar is significantly greater than the length of other connected regions. Therefore, by obtaining the length, most of the connected regions that do not belong to a pillar can be eliminated.

[0107] Then, since columns generally have a certain width, filtering by width can further eliminate connected areas that do not belong to columns. The above implementation method can quickly eliminate the vast majority of connected areas that are not columns.

[0108] Preferably, the area of ​​the connected region can be represented by the number of pixels.

[0109] Preferably, the set is obtained based on the area. The connected area of ​​the pillars belonging to the sound barrier includes:

[0110] for The area of ​​the connected region D in the equation is then compared with the area of ​​the connected region D. Compare the areas of each connected region except for connected region D. If there exists at least one other connected region E such that the difference in area between connected region E and connected region D is less than a set area threshold, then connected region D is retained. If there is no other connected region E such that the difference in area between connected region E and connected region D is less than a set area threshold, then connected region D is removed from the list of connected regions. Delete;

[0111] Will The remaining connected regions are considered as the connected regions of the pillars belonging to the sound barrier.

[0112] To further improve accuracy, this invention also uses area for further filtering. Ideally, the area of ​​the pillars in a video frame should be the same. However, due to the identification of connected components and the influence of other factors during the shooting process, the areas of the pillars in the video frames are not entirely identical. Therefore, this invention uses an area threshold to further exclude components, obtaining a set containing only connected components belonging to the pillars, which effectively improves subsequent acquisition accuracy. The accuracy of.

[0113] Preferably, calculating the image distance between adjacent connected regions belonging to the sound barrier pillars includes:

[0114] from In this context, we can arbitrarily obtain two adjacent connected regions F and G.

[0115] Calculate the distance between the center of connected region F and the center of connected region G. ,Will Image distance as the connected region of the pillars belonging to the sound barrier.

[0116] Preferably, the distance between the camera and the sound barrier is calculated based on the image distance. ,include:

[0117] Calculate using the following function :

[0118]

[0119] in, The preset shooting distance. This represents the distance between the center of connected region F and the center of connected region G in an image frame obtained at a preset shooting distance. .

[0120] Specifically, because the distance between the camera and the sound barrier fluctuates within a relatively small range during the shooting process, the distance between the camera and the sound barrier can be obtained from the image distance using a given calculation coefficient. A larger image distance indicates a larger distance between the camera and the sound barrier. At the preset shooting distance, only two pillars are included in the video frame. Here, the shooting distance refers to the distance between the camera and the sound barrier.

[0121] Preferably, the time between the recording time and the shooting time is obtained. Recent speed ,include:

[0122] The recorded times in the velocity information set are sorted in ascending order to obtain the set. ;

[0123] Calculate separately Each recorded moment in the data is related to... The absolute value of the difference between them;

[0124] The speed is the speed corresponding to the recording time corresponding to the absolute value of the smallest difference. .

[0125] Preferably, each video frame in the set to be identified is input into a pre-trained neural network model for identification to obtain the identification result of the video frame, including:

[0126] Each video frame in the set to be identified is preprocessed to obtain the video frame to be identified.

[0127] The video frames to be identified are input into a pre-trained neural network model for recognition, and the recognition results of the video frames are obtained.

[0128] Specifically, preprocessing includes noise reduction, etc.

[0129] Preferably, the identification result is that the sound barrier in the video frame has a safety hazard or does not have a safety hazard.

[0130] Preferably, when the identification result indicates that the sound barrier in the video frame poses a safety hazard, the identification result also includes the type of safety hazard present in the sound barrier in the video frame.

[0131] Specifically, the types of safety hazards that sound barriers may pose include deformation, detachment, and rust.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for analyzing the potential hazards of sound barriers on urban expressways based on video analysis, characterized in that, include: S1, to film the sound barrier located on the side of the urban expressway and obtain the inspection video containing the sound barrier; S2, Obtain video frames for image recognition from the inspection video and save the obtained video frames to the recognition set; S3, each video frame in the set to be identified is input into the pre-trained neural network model for identification, and the identification result of the video frame is obtained; S4, save the recognition results to the result set; The inspection video, which included images of the noise barriers along the urban expressway, was filmed. By using vehicles equipped with cameras to film the noise barriers along the roadside of urban expressways, inspection videos containing the noise barriers are obtained. During the filming process, the vehicle speed information is recorded at a fixed acquisition cycle to obtain a collection of speed information. Speed ​​information includes speed and recording time; Before obtaining video frames from the inspection video for image recognition, the process also includes: The video frames in the inspection video are numbered consecutively according to the time of shooting, with the later the shooting time, the larger the number. Video frames used for image recognition are obtained from the inspection video, including: The first time the video is acquired, the first frame of the inspection video is used as the video frame for image recognition. The process for the nth acquisition is as follows, where n is greater than or equal to 2: The video frame obtained for image recognition in the (n-1)th iteration is quickly identified to obtain the distance between the camera and the sound barrier. ; Obtain the number of the video frame used for image recognition obtained in the (n-1)th iteration. and shooting time ; Get the time from recording time to shooting time Recent speed ; The following function is used to calculate the number of the video frame used for image recognition in the nth inspection video. : in, Indicates standard distance. Indicates the calculated coefficients. , Indicates standard speed. This represents a set constant. , and These represent the lower and upper limits of the change in the number, respectively.

2. The method for analyzing the potential hazards of urban expressway sound barriers based on video analysis according to claim 1, characterized in that, The video frame obtained for image recognition in the (n-1)th iteration is quickly identified to obtain the distance between the camera and the sound barrier. ,include: use This represents the video frame obtained in the (n-1)th iteration for image recognition. right The process of rapid identification is as follows: To each The edge coefficient of each pixel is calculated; exist Delete pixels with edge coefficients less than a set edge coefficient threshold to obtain multiple connected regions, and save the obtained connected regions to a set. ; Obtain each The length and width of each connected region in the array; Remove connected regions whose length is less than a set length threshold from the set. Delete to get a set ; Remove connected regions whose width is less than a set width threshold from the set. Delete to get a set ; Get the collections respectively The area of ​​each connected region in the equation; Sets obtained based on area The connected area of ​​the pillars belonging to the sound barrier; Calculate the image distance between adjacent connected regions belonging to the sound barrier pillars; The distance between the camera and the sound barrier is calculated based on the image distance. .

3. The method for analyzing the potential hazards of urban expressway sound barriers based on video analysis according to claim 1, characterized in that, Get the time from recording time to shooting time Recent speed ,include: The recorded times in the velocity information set are sorted in ascending order to obtain the set. ; Calculate separately Each recorded moment in the data is related to... The absolute value of the difference between them; The speed is the speed corresponding to the recording time corresponding to the absolute value of the smallest difference. .

4. The method for analyzing the potential hazards of urban expressway sound barriers based on video analysis according to claim 1, characterized in that, Each video frame in the set to be identified is input into a pre-trained neural network model for recognition, and the recognition results of the video frames are obtained, including: Each video frame in the set to be identified is preprocessed to obtain the video frame to be identified. The video frames to be identified are input into a pre-trained neural network model for recognition, and the recognition results of the video frames are obtained.

5. The method for analyzing the potential hazards of urban expressway sound barriers based on video analysis according to claim 1, characterized in that, The identification results indicate whether the sound barrier in the video frame poses a safety hazard or not.

6. The method for analyzing the potential hazards of urban expressway sound barriers based on video analysis according to claim 5, characterized in that, When the identification result indicates that the sound barrier in the video frame poses a safety hazard, the identification result also includes the type of safety hazard that exists in the sound barrier in the video frame.

Images