Method for measuring rising and falling time of electric barrier based on neural network model

Abstract

The invention relates to an electric barrier rising and falling time measuring method based on a neural network model. The problem that the rising and falling time of a high-speed electric barrier is not accurately measured is solved. The core of the method is that a high-speed camera is used for collecting video image data of rising and falling of the electric barrier, and an improved YOLOv5 neural network model is constructed. The model integrates an angle regression function at a detection head part so as to accurately predict the dynamic angle of the handrail; and meanwhile, a RepVGG module is introduced to optimize the reasoning speed, so that the real-time processing capability is ensured. And by adopting a CIoU angle loss function, the accuracy of angle prediction is further improved. According to the invention, a time compensation algorithm based on an optical flow method is also fused to correct a time error possibly caused by unstable video frame rate. Finally, according to a judgment mechanism that the continuous multi-frame angle change rate reaches a preset threshold value, the starting and stopping moments of the railing are accurately recognized, and therefore high-precision measurement of the rising and falling time of the electric railing machine is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of electric barrier rise and fall time testing technology, specifically a method for measuring the rise and fall time of electric barriers based on a neural network model. This method can be widely used in fields such as highway electromechanical testing, barrier product testing, and traffic management, enabling accurate measurement and status monitoring of electric barrier rise and fall times. Background Art

[0002] With the continuous development of highway electromechanical engineering, especially after the nationwide abolition of provincial highway toll booths in 2019, intelligent and efficient toll collection facilities have gradually become mainstream. In current highway vehicle traffic management systems, electric barriers are widely used for lane control and traffic management, and their rise and fall time has become a key parameter affecting overall traffic efficiency.

[0003] Traditional electric barrier machines used in highway applications, especially early equipment, mostly have slow rise and fall speeds, with rise and fall times usually between 2 and 4 seconds. These early devices could be simply measured with a stopwatch, and under the application environment at the time, the measurement method was able to meet actual needs. However, with the upgrade of intelligent highway equipment and technological development, especially the introduction of high-speed barrier machines, the rise and fall time of the equipment has been significantly shortened. The total rise and fall time of modern high-speed barrier machines is generally controlled within 1.4 seconds, and even each rise or fall time is only about 0.6 seconds. The stopwatch method cannot accurately capture such a short rise and fall process, and the measurement error is large. This is mainly because the stopwatch operation is difficult to synchronize with the actual rise and fall time of the barrier machine, resulting in a strong subjectivity in the judgment of the rise and fall time, which in turn affects the accuracy of the measurement results.

[0004] Currently, it's impossible to accurately capture such a brief rise and fall process. Frame-reading methods are currently used for measurement. This method uses a high-speed camera to capture the entire rise and fall of an electric barrier. The barrier's lifting and lowering movements are recorded as a series of continuous video frames by the high-speed camera. Manual analysis using video editing software requires an operator to observe and record the rise and fall of the barrier frame by frame, identifying the starting and ending points, and thus calculating the rise and fall times. This method has several drawbacks: It requires significant manual intervention, relying on manual frame-by-frame analysis of the video, which is highly subjective. Manual analysis can lead to measurement errors due to factors such as fatigue and lack of concentration. It is also time-consuming. Although the video captures the entire process, manual frame-by-frame analysis of the video images is time-consuming, especially at high speeds. Accurately determining the timing of each rise and fall movement can require significant analysis time. Some existing video detection methods utilize traditional object detection algorithms to detect the movement of electric barriers. However, in practice, video streams are often affected by frame rate fluctuations and frame loss, resulting in insufficient detection accuracy and time synchronization, making them inadequate for high-precision real-time measurement.

[0005] The patent number is 202411050766.0, and the patent name is "A method and device for measuring the rising and falling time of an electric railing". It discloses a method and device for measuring the rising and falling time of an electric railing, including an electric railing machine and a railing arm. The railing arm is rotatably connected to the electric railing machine. A magnetic proximity switch 1 and a magnetic proximity switch 2 are arranged in a cross pattern on one side of the electric railing machine and below the railing arm. When the railing arm drops, it contacts the top of the magnetic proximity switch 1. When the railing arm is fully raised, it contacts the right end of the magnetic proximity switch 2. The problems are that the sensor installation and maintenance are complicated, the measurement accuracy is limited by the magnetic switch itself, it is impossible to detect the rising and falling angles of the railing throughout the entire process, and it is difficult to implement on a large scale.

[0006] Despite numerous studies, there remains a lack of an efficient, accurate, and objective solution for measuring the rise and fall times of high-speed electric barriers. Therefore, a key challenge remains: developing a solution that ensures high-precision measurement during the rapid rise and fall of high-speed barriers while overcoming the shortcomings of traditional methods. Summary of the Invention

[0007] In view of this, the present invention provides a method for measuring the rise and fall time of an electric barrier machine based on a neural network model to solve the problems of inaccurate measurement of the rise and fall time of a high-speed electric barrier machine and low degree of automation.

[0008] To achieve the above object, the technical solution adopted by the present invention is: a method for measuring the rise and fall time of an electric barrier machine based on a neural network model, the steps are as follows:

[0009] 1) Data collection and preprocessing

[0010] Use high-speed cameras to collect video image data of the electric barrier machine rising and falling to ensure data integrity, video image quality and timing stability;

[0011] Preprocess the captured video images, resize them to a preset size, and annotate the key angle values ​​of the barrier machine and its motion state to form a sample dataset with angle and state labels.

[0012] Data augmentation techniques are used on sample datasets to expand the dataset and improve the model's generalization ability to different environmental changes;

[0013] 2) Build an improved YOLOv5 model

[0014] The improved model consists of a sequentially connected backbone feature extraction network, an attention feature extraction network, a neck network, and a detection head. An angle regression branch is added to the detection head to predict the barrier's take-off and landing angle θ, where θ∈[0°,90°]. The reparameterization mechanism of the RepVGG module is introduced into the partial convolutional modules of the backbone feature extraction network and the neck network, enabling fast image processing and detection results in real-time video streams.

[0015] 3) Training and defining CIoU angle loss function

[0016] Based on each sample image, the sample image is used as input, and the detection position, angle, and confidence corresponding to each rise and fall state of the electric barrier in the image are used as output. The Adam optimizer is combined with the CIoU angle loss function to train the improved YOLOv5 model network to obtain the rise and fall time detection model of the electric barrier. The CIoU angle loss function is defined as:

[0017] L angle =1-cos(θ pred -θ true )

[0018] Where: L angle : represents the angle loss value;

[0019] θ pred : represents the angle of the barrier predicted by the model;

[0020] θ true : Indicates the actual marked angle of the barrier machine;

[0021] 4) Optical flow compensation algorithm

[0022] The optical flow method is used to calculate the motion vector between adjacent frame images to compensate for the time error caused by video frame rate fluctuation or frame loss, and the compensation time Δt is obtained. comp , and apply it to the time calculation module, so that the final measured rise and fall time T total satisfy:

[0023] T total =(t end -t start )+Δt comp

[0024] Among them, T total : Indicates the final measured complete raising and lowering time of the barrier machine;

[0025] t start : Indicates the moment when the barrier machine starts moving;

[0026] t end : Indicates the moment when the barrier machine completes the movement;

[0027] Δt comp : represents the time error compensation value calculated by the optical flow compensation algorithm;

[0028] 5) Motion state determination method based on angle change rate:

[0029] By determining that the rate of change of the barrier machine angle in 5 consecutive frames is greater than 2° / frame, the transition moment from the static state to the moving state of the barrier machine is determined to ensure that the recorded t start and t end With angle change continuity;

[0030] 6) Deployment and implementation:

[0031] The improved YOLOv5 network is deployed using the TensorRT acceleration engine to automatically detect and measure the rising and falling status of electric barrier machines in real-time video, and the measurement results and status monitoring information are output in real time through the user interface.

[0032] Further, the specific method of step 2) is:

[0033] 2.1) Backbone feature extraction network construction:

[0034] 2.1.1) Construct a backbone feature extraction network. Its input receives the preprocessed image and first slices it through a Focus layer to generate a preliminary feature map.

[0035] 2.1.2) The preliminary feature map output by the Focus layer is processed in sequence by multiple downsampling convolutions and feature stacking units to extract the front features of different Csplayer levels, and the final output early high-level semantic features are sent to the attention feature extraction network;

[0036] The P3 layer features, P4 layer features, and P5 layer features output by the backbone feature extraction network correspond to feature maps at different levels, respectively. The P3 layer features and P4 layer features are directly sent to the neck network, and the P5 layer features are sent to the attention feature extraction network for processing. The processed output is the highest-level semantic feature.

[0037] 2.1.3) The Csplayer layer adopts the idea of ​​cross-stage local connection and contains multiple bottleneck layers. Each bottleneck layer is composed of several convolutional layers and residual connections.

[0038] 2.2) Attention feature extraction network construction:

[0039] 2.2.1) Construct an attention feature extraction network, which receives the highest-level semantic features output by the backbone feature extraction network, including the SPPF_t layer and the CBAM layer in sequence;

[0040] 2.2.2) The SPPF_t layer includes 1x1 convolutional layers at the beginning and end, and multiple parallel maximum pooling layers and concat layers with different kernel sizes in the middle. The number of output channels is the same as the number of input channels.

[0041] 2.2.3) The CBAM layer connects the channel attention module and the spatial attention module in series, and enhances and filters the features output by the SPPF_t layer by learning and applying attention weights. The number of output channels is consistent with the number of input channels.

[0042] 2.3) Neck network construction:

[0043] 2.3.1) Using the PANet architecture, the input is the P3 layer pre-features and P4 layer pre-features from the backbone feature extraction network, as well as the highest-level semantic features processed by the attention feature extraction network;

[0044] 2.4) Detection head construction:

[0045] 2.4.1) Receive feature maps at three different scales from the neck network output by the detection head. Each detection head at each scale contains one or more convolutional layers for outputting object detection results. Based on the bounding box coordinates, object confidence, and category information, the detection head also includes an angle regression branch, also composed of one or more convolutional layers, for outputting the barrier's lift and landing angle θ, where θ ranges from [0° to 90°].

[0046] 2.5) Overall network integration and introduction of RepVGG module

[0047] 2.5.1) Connect the backbone feature extraction network, attention feature extraction network, neck network, and detection head in the aforementioned input and output order to form an improved YOLOv5 network model;

[0048] 2.5.2) The convolutional layers within the Csplayer layer in the backbone feature extraction network of the improved YOLOv5 network model, as well as the convolutional layers within the Csplayer layer used for feature fusion in the PANet structure in the neck network, introduce the reparameterization mechanism of the RepVGG module. That is, a multi-branch structure is adopted in the training phase, and the parameters of these multi-branch structures are equivalently merged into a single 3x3 convolution operation parameter in the inference phase.

[0049] Furthermore, in step 1), the video image data is collected with a fixed frame rate of 100 frames per second, and the marked key angle values ​​are at intervals of 5°.

[0050] Furthermore, the Csplayer layer in step 2.1.2) and the PANet structure in step 2.3.1) are composed of an input convolutional layer, an output convolutional layer, and multiple bottleneck layers (Bottleneck) in the middle. The bottleneck layer usually contains two 1x1 convolutional layers and one 3x3 convolutional layer with residual connections. The feature map is divided into two paths inside Csplayer, one is processed by the bottleneck layer, and the other is directly connected. Finally, the two features are fused through the Concat operation.

[0051] Furthermore, in step 2.3.1), the PANet structure generates three different scale output feature maps (P3_out, P4_out, and P5_out corresponding to small, medium, and large objects) for the detection head through a top-down feature pyramid path (up-sampling and backbone feature fusion followed by a Csplayer layer) and a bottom-up feature enhancement path (down-sampling and previous path feature fusion followed by a Csplayer layer).

[0052] Furthermore, the detection head in step 2.4.2) contains three detection units for different target scales, corresponding to the P3_out, P4_out and P5_out feature maps output by the neck network, to achieve multi-scale target detection.

[0053] Compared with the prior art, the advantages of the present invention are as follows:

[0054] 1) By introducing the YOLOv5 model and adding an angle regression branch on its basis, the present invention enables the model to not only detect the position of the barrier machine, but also accurately predict the angle change of the barrier, ensuring the accurate detection of small-scale dynamic targets (electric barrier machines). The advantages of YOLOv5 in small target detection are fully utilized in this application.

[0055] 2) The present invention introduces a state determination algorithm based on spatiotemporal continuity. By determining the angle change rate (e.g., the angle change per frame is >2° / frame), the motion state of the barrier machine is determined. The present invention can accurately determine the start and end times of the barrier machine's movement. In addition, the optical flow method is used for time compensation, which effectively improves the calculation accuracy and avoids the calculation errors of traditional methods.

[0056] 3) By introducing the RepVGG module into the YOLOv5 model, this paper significantly improves the model's inference speed, enabling efficient processing of large amounts of data. This enables automated measurement, particularly in complex test scenarios, significantly improving test efficiency. The RepVGG module's reparameterization technique significantly improves inference efficiency by combining multiple convolution operations into a single one during inference, ensuring efficient real-time video stream processing and meeting the needs of real-time measurement.

[0057] 4) The present invention uses the YOLOv5 training model to identify the motion state of the electric railing in each frame of video image and calculate the rise and fall time of the railing. It can capture subtle dynamic changes and provide real-time feedback of measurement results within milliseconds. Through the adaptive frame rate compensation algorithm, it reduces the error introduced by frame rate fluctuation factors, ensures the continuity of measurement, and further improves the measurement accuracy.

[0058] 5) The present invention relies entirely on video images and the YOLOv5 model for target detection, avoiding the installation and maintenance costs of traditional sensor systems, while also avoiding problems caused by sensor damage or errors.

[0059] 6) This invention employs an adaptive frame rate compensation algorithm, using optical flow interpolation to compensate for timing errors caused by frame loss or frame rate fluctuations, thereby ensuring accurate measurement of rise and fall times for low-frame rate video streams. This technological innovation ensures that barrier rise and fall times can be accurately calculated even under unstable frame rate conditions.

[0060] 7) During the model training process, the present invention adopts a learning rate warm-up strategy to gradually increase the learning rate to a preset value to improve the convergence speed and stability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 The process of the present invention Figure 1 .

[0062] Figure 2 The process of the present invention Figure 2 . DETAILED DESCRIPTION

[0063] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention. The technical solutions provided by each embodiment of the present invention are described in detail below in conjunction with the drawings.

[0064] This embodiment provides a method for measuring the rise and fall time of an electric barrier machine based on a neural network model. Figure 1 and 2 As shown, the steps are:

[0065] Step 1: Data collection and preprocessing

[0066] Use a high-speed camera to collect video image data of the electric barrier machine rising and falling, ensure data integrity, ensure that the video image resolution is not less than 1080P, and timing stability, the frame rate is fixed at 100 frames / second, to ensure a balance between measurement accuracy and calculation efficiency, and to ensure video image quality and timing stability;

[0067] Each captured video image is preprocessed, including image denoising, color balancing, and geometric correction. The images are uniformly resized to a preset size (640x640 pixels). The key angle values ​​of the barrier in the image (at intervals of 5 degrees) and its motion state (such as stationary, raised, or lowered) are annotated to form a sample dataset with angle and state labels.

[0068] For the above sample dataset, data enhancement techniques (including but not limited to rotation, scaling, cropping, color jittering, etc.) are further used to expand the dataset and improve the model's generalization ability to different environmental changes;

[0069] Step 2: Build an improved YOLOv5 model

[0070] This step is to build an improved YOLOv5 model that can accurately predict the rise and fall angles of the barrier machine and has high inference efficiency.

[0071] The improved model includes a backbone feature extraction network, an attention feature extraction network, a neck network and a detection head connected in sequence. An angle regression branch is added to the detection head to accurately predict the take-off and landing angles of the barrier machine (θ∈[0°,90°]). The reparameterization mechanism of the RepVGG module is introduced in the partial convolution modules of the backbone feature extraction network and the neck network to ensure that images can be processed quickly and detection results can be obtained in real-time video streams.

[0072] The specific method is:

[0073] 2.1) Backbone feature extraction network construction:

[0074] 2.1.1) Construct a backbone feature extraction network, which receives a preprocessed image (640x640x3) at its input. The network extracts image features of different scales layer by layer through multiple layers of convolution and downsampling operations, including a preliminary feature map and subsequent P3, P4, and P5 layer features. The image first passes through a Focus layer, where the input image is split into multiple small blocks and concatenated, halving the image width and height and increasing the number of channels to four times the original (input 640x640x3->320x320x12 after the Focus operation). Subsequently, the feature map passes through a convolutional layer (Conv2D, convolution kernel size 3x3, stride 1, padding 1, output channel number 64) -> Batch Normalization (BN) layer -> SiLU activation function, resulting in a preliminary feature map of size 320x320x64.

[0075] 2.1.2) The preliminary feature map output by the Focus layer is processed in sequence through multiple downsampling convolutions and feature stacking units to extract features at different levels:

[0076] a. The first downsampling convolution and feature stacking unit:

[0077] The network first passes through a downsampling convolutional layer (Conv2D, kernel size 3x3, stride 2, padding 1, output channels 128) -> Batch Normalization (BN) -> SiLU activation function, reducing the feature map size to 160x160x128. It is then connected to the first Csplayer layer (C3 module, consisting of three bottleneck layers, 128 input channels, and 128 output channels).

[0078] b. The second downsampling convolution and feature stacking unit:

[0079] The feature map output by the first Csplayer layer (160x160x128) passes through a downsampling convolutional layer (Conv2D, kernel size 3x3, stride 2, padding 1, 256 output channels) -> Batch Normalization (BN) -> SiLU activation function, reducing the feature map size to 80x80x256. It is then connected to the second Csplayer layer (C3 module, consisting of six bottleneck layers, 256 input channels, and 256 output channels). The output of this layer serves as the P3 layer feature map (80x80x256) and is sent to the neck network.

[0080] c. The third downsampling convolution and feature stacking unit:

[0081] The feature map (80x80x256) output by the second Csplayer layer passes through a downsampling convolutional layer (Conv2D, kernel size 3x3, stride 2, padding 1, 512 output channels) -> Batch Normalization (BN) -> SiLU activation function, reducing the feature map size to 40x40x512. It is then connected to the third Csplayer layer (C3 module, consisting of 9 bottleneck layers, 512 input channels, and 512 output channels). The output of this layer is used as the P4 layer feature map (40x40x512) and is sent to the neck network.

[0082] d. The fourth downsampling convolution and feature stacking unit:

[0083] The feature map (40x40x512) output by the third Csplayer layer passes through a downsampling convolutional layer (Conv2D, kernel size 3x3, stride 2, padding 1, 512 output channels) -> Batch Normalization (BN) -> SiLU activation function, reducing the feature map size to 20x20x512. It is then connected to a fourth Csplayer layer (C3 module, consisting of three bottleneck layers, with 512 input and output channels). The output of this layer serves as the P5 layer feature map (20x20x512) and is fed into the subsequent SPPF_t and CBAM modules.

[0084] The P3 layer features, P4 layer features, and P5 layer features output by the backbone feature extraction network correspond to feature maps at different levels, respectively. The P3 layer features and P4 layer features are directly sent to the neck network, and the P5 layer features are sent to the attention feature extraction network for processing. The processed output is the highest-level semantic feature.

[0085] 2.1.3) The Csplayer layer adopts the idea of ​​cross-stage local connection and contains multiple bottleneck layers. Each bottleneck layer is composed of several convolutional layers and residual connections.

[0086] 2.2) Attention feature extraction network construction:

[0087] 2.2.1) Construct an attention feature extraction network, which receives the highest-level semantic features (20x20x512 or higher) output from the P5 layer of the backbone feature extraction network. This feature is processed by the SPPF_t layer and the CBAM layer in turn.

[0088] 2.2.2) The SPPF_t layer (Spatial Pyramid Pooling Fast) consists of 1x1 convolutional layers at the beginning and end, and multiple parallel max pooling layers and concat layers with different kernel sizes (5x5, 9x9, 13x13) in the middle. It is used to perform multi-scale receptive field fusion on the input features. The number of output channels is the same as the number of input channels (512).

[0089] 2.2.3) The CBAM layer (Convolutional Block Attention Module) connects the channel attention module and the spatial attention module in series. By learning and applying attention weights, it enhances and filters the features output by the SPPF_t layer. The number of output channels is consistent with the number of input channels (for example, 512).

[0090] 2.3) Neck network construction:

[0091] 2.3.1) uses the PANet (Path Aggregation Network) structure, whose input is the P3 layer pre-features (80x80x256) from the backbone feature extraction network, the P4 layer pre-features (40x40x512), and the top-level semantic features (20x20x512 or higher) (derived from the original P5 layer) processed by the attention feature extraction network.

[0092] The above-mentioned PANet structure generates three different scale output feature maps (P3_out, P4_out, and P5_out corresponding to small, medium, and large objects) for the detection head through a top-down feature pyramid path (up-sampling and backbone feature fusion followed by a Csplayer layer) and a bottom-up feature enhancement path (down-sampling and previous path feature fusion followed by a Csplayer layer).

[0093] Top-down path: P5 features (20x20x512) undergo 1x1 convolution (adjusting channels to 256) and upsampling (to 40x40), concatenated with P4 features (40x40x512, 1x1 convolution adjusting channels to 256), and then pass through the C3 module (3 bottleneck layers, output channels of 256 or 512) to obtain P4_fused. P4_fused undergoes 1x1 convolution (adjusting channels to 128) and upsampling (to 80x80), concatenated with P3 features (80x80x256, 1x1 convolution adjusting channels to, for example, 128), and then pass through the C3 module (3 bottleneck layers, output channels of 256) to obtain P3_out (80x80x256).

[0094] Bottom-up path: P3_out (80x80x256) undergoes downsampling convolution (3x3, s2, output channels 256 or 512) to 40x40, concatenated with P4_fused (after adjusting channels), and then passes through the C3 module (3 bottleneck layers, output channels 512) to obtain P4_out (40x40x512). P4_out undergoes downsampling convolution (3x3, s2, output channels 512 or higher) to 20x20, concatenated with P5 features (after adjusting channels), and then passes through the C3 module (3 bottleneck layers, output channels 512 or higher) to obtain P5_out (20x20x512 or higher).

[0095] 2.4) Detection head construction:

[0096] 2.4.1) The head unit receives feature maps of three scales, P3_out (80x80x256), P4_out (40x40x512), and P5_out (20x20x512 or higher), from the neck network. Each detection head at each scale contains one or more convolutional layers to output object detection results. In addition to outputting bounding box coordinates, object confidence, and category information, the detection head also includes an angle regression branch, also composed of one or more convolutional layers, specifically for outputting the barrier's take-off and landing angle θ, where θ ranges from [0° to 90°].

[0097] 2.4.2) The detection head comprises multiple detection units P3, P4 and P5 for different target scales, corresponding to small, medium and large-sized barrier machine target detection respectively, to ensure the accuracy of multi-scale target detection.

[0098] 2.5) Overall network integration and introduction of RepVGG module:

[0099] 2.5.1) Connect the backbone feature extraction network, attention feature extraction network, neck network (Neck), and detection head (Head) according to the above input and output sequence to form an improved YOLOv5 network model;

[0100] 2.5.2) In the convolutional layers within the Csplayer layer (C3 module) in the backbone feature extraction network of the improved YOLOv5 network model, as well as the convolutional layers within the Csplayer layer (C3 module) used for feature fusion in the PANet structure in the neck network, a reparameterization mechanism of the RepVGG module is introduced. That is, a multi-branch structure (3x3 convolution, 1x1 convolution, and identity mapping branch) is adopted in the training phase. In the inference phase, the parameters of these multi-branch structures are equivalently merged into a single 3x3 convolution operation parameter to improve the inference speed of the model.

[0101] The Csplayer layer in steps 2.1.2) and 2.3.2) consists of an input convolution layer, an output convolution layer, and multiple bottleneck layers (Bottleneck) in the middle. The bottleneck layer usually contains two 1x1 convolution layers and one 3x3 convolution layer with residual connections. The feature map is divided into two paths inside Csplayer, one is processed by the bottleneck layer, and the other is directly connected. Finally, the two features are fused through the Concat operation.

[0102] Step 3: Train and define the CIoU (Complete Intersection over Union) angle loss function

[0103] Based on each sample image, the sample image is used as input, and the detection position, angle and confidence corresponding to each rise and fall state of the electric barrier in the image are used as output. The Adam optimizer is combined with the CIoU angle loss function to train the improved YOLOv5 model network. In the early stage of training, the learning rate warm-up strategy is adopted, that is, the learning rate is gradually increased linearly from a small initial value to the preset learning rate within the warm-up steps to improve the convergence speed and stability of the model training, and the electric barrier rise and fall time detection model is obtained. The CIoU angle loss function is defined as:

[0104] L angle =1-cos(θ pred -θ true )

[0105] Where: L angle : Indicates the angle loss value.

[0106] θ pred : represents the angle of the barrier predicted by the model.

[0107] θ true : Indicates the actual (or marked) angle of the barrier machine.

[0108] cos(): represents the cosine function.

[0109] The CIoU angle loss function effectively measures the error in angle prediction by calculating the cosine of the difference between the predicted and actual angles. The cosine function effectively handles angle periodicity and provides a smooth loss function, facilitating model optimization. This loss function is designed to ensure that the predicted angle is as close as possible to the actual angle, thereby improving the accuracy of the barrier's lift and landing angle predictions.

[0110] Step 4: Optical flow compensation algorithm:

[0111] The optical flow method (Farneback dense optical flow algorithm) is used to calculate the motion vector between adjacent frame images to compensate for the time error caused by video frame rate fluctuation or frame loss, and the compensation time Δt is obtained. comp , and apply it to the time calculation module, so that the final measured rise and fall time T total satisfy:

[0112] T total =(t end -t start )+Δt comp

[0113] Among them, T total : Indicates the final measured complete raising and lowering time of the barrier machine;

[0114] t start : Indicates the moment when the barrier machine starts moving;

[0115] t end : Indicates the moment when the barrier machine completes the movement;

[0116] Δt comp : represents the time error compensation value calculated by the optical flow compensation algorithm;

[0117] The core purpose of this formula is to improve the accuracy of the barrier machine's up and down time measurement by estimating the time error through the optical flow method when the video frame rate is unstable.

[0118] Step 5: Motion state determination method based on angle change rate:

[0119] By determining that the rate of change of the barrier machine angle in 5 consecutive frames is greater than 2° / frame, the transition moment from the static state to the moving state of the barrier machine is determined to ensure that the recorded t start and t end It has angle change continuity, thereby improving the measurement accuracy of take-off and landing time.

[0120] Step 6: Deployment and implementation:

[0121] The improved YOLOv5 network is deployed using the TensorRT acceleration engine to automatically detect and measure the rising and falling status of electric barrier machines in real-time video, and the measurement results and status monitoring information are output in real time through the user interface.

[0122] The numerous convolutional modules in the improved YOLOv5 network of the present invention (including the post-Focus convolution in the backbone network, the downsampling convolutions and convolutions within the Csplayer / C3 module, the convolutions within the attention module SPPF_t, the upsampling and downsampling convolutions and convolutions within the Csplayer / C3 module in the PANet network, and the prediction convolution in the detection head) all primarily utilize a combination of convolution, batch normalization, and SiLU activation functions. The network implements residual connections via a bottleneck layer within the Csplayer / C3 module. The Focus layer (initial stage), the SPPF_t layer, and the feature concatenation (Concat) operations within the PANet network structure achieve efficient fusion of multi-scale and multi-level features, ensuring that the network maintains high detection accuracy while maintaining high computational speed.

[0123] The present invention uses the above method to reduce false detections and missed detections while ensuring real-time performance, thereby improving the accuracy of measuring the rise and fall times of electric barrier machines. This method avoids the limitations of traditional manual stopwatch methods and sensor solutions, providing an efficient measurement method based entirely on image data without the need for physical sensors.

[0124] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of protection of the technical solution of the present invention.

Claims

1. A method for measuring the rise and fall time of an electric barrier machine based on a neural network model, characterized in that: The steps are: 1) Data collection and preprocessing Use high-speed cameras to collect video image data of the electric barrier machine rising and falling to ensure data integrity, video image quality and timing stability; Preprocess the captured video images, resize them to a preset size, and annotate the key angle values ​​of the barrier machine and its motion state to form a sample dataset with angle and state labels. Data augmentation techniques are used on sample datasets to expand the dataset and improve the model's generalization ability to different environmental changes; 2) Build an improved YOLOv5 model The improved model consists of a sequentially connected backbone feature extraction network, an attention feature extraction network, a neck network, and a detection head. An angle regression branch is added to the detection head to predict the barrier's take-off and landing angle θ, where θ∈[0°,90°]. The reparameterization mechanism of the RepVGG module is introduced into the partial convolutional modules of the backbone feature extraction network and the neck network, enabling fast image processing and detection results in real-time video streams. 3) Training and defining CIoU angle loss function Based on each sample image, the sample image is used as input, and the detection position, angle, and confidence corresponding to each rise and fall state of the electric barrier in the image are used as output. The Adam optimizer is combined with the CIoU angle loss function to train the improved YOLOv5 model network to obtain the rise and fall time detection model of the electric barrier. The CIoU angle loss function is defined as: L angle =1-cos(θ pred -θ true ) Where: L angle : represents the angle loss value; θ pred : represents the angle of the barrier predicted by the model; θ true : Indicates the actual marked angle of the barrier machine; 4) Optical flow compensation algorithm The optical flow method is used to calculate the motion vector between adjacent frame images to compensate for the time error caused by video frame rate fluctuation or frame loss, and the compensation time Δt is obtained. comp , and apply it to the time calculation module, so that the final measured rise and fall time T total satisfy: T total =(t end -t start )+Δt comp Among them, T total : Indicates the final measured complete raising and lowering time of the barrier machine; t start : Indicates the moment when the barrier machine starts moving; t end : Indicates the moment when the barrier machine completes the movement; Δt comp : represents the time error compensation value calculated by the optical flow compensation algorithm; 5) Motion state determination method based on angle change rate: By determining that the rate of change of the barrier machine angle in 5 consecutive frames is greater than 2° / frame, the transition moment from the static state to the moving state of the barrier machine is determined to ensure that the recorded t start and t end With angle change continuity; 6) Deployment and implementation: The improved YOLOv5 network is deployed using the TensorRT acceleration engine to automatically detect and measure the rising and falling status of electric barrier machines in real-time video, and the measurement results and status monitoring information are output in real time through the user interface.

2. The method for measuring the rise and fall time of an electric barrier machine based on a neural network model according to claim 1, wherein the specific method of step 2) is: 2.1) Backbone feature extraction network construction: 2.1.1) Construct a backbone feature extraction network. Its input receives the preprocessed image and first slices it through a Focus layer to generate a preliminary feature map. 2.1.2) The preliminary feature map output by the Focus layer is processed in sequence by multiple downsampling convolutions and feature stacking units to extract the front features of different Csplayer levels, and the final output early high-level semantic features are sent to the attention feature extraction network; The P3 layer features, P4 layer features, and P5 layer features output by the backbone feature extraction network correspond to feature maps at different levels, respectively. The P3 layer features and P4 layer features are directly sent to the neck network, and the P5 layer features are sent to the attention feature extraction network for processing. The processed output is the highest-level semantic feature. 2.1.3) The Csplayer layer adopts the idea of ​​cross-stage local connection and contains multiple bottleneck layers. Each bottleneck layer is composed of several convolutional layers and residual connections. 2.2) Attention feature extraction network construction: 2.2.1) Construct an attention feature extraction network, which receives the highest-level semantic features output by the backbone feature extraction network, including the SPPF_t layer and the CBAM layer in sequence; 2.2.2) The SPPF_t layer includes 1x1 convolutional layers at the beginning and end, and multiple parallel maximum pooling layers and concat layers with different kernel sizes in the middle. The number of output channels is the same as the number of input channels. 2.2.3) The CBAM layer connects the channel attention module and the spatial attention module in series, and enhances and filters the features output by the SPPF_t layer by learning and applying attention weights. The number of output channels is consistent with the number of input channels. 2.3) Neck network construction: 2.3.1) Using the PANet architecture, the input is the P3 layer pre-features and P4 layer pre-features from the backbone feature extraction network, as well as the highest-level semantic features processed by the attention feature extraction network; 2.4) Detection head construction: 2.4.1) Receive feature maps at three different scales from the neck network output by the detection head. Each detection head at each scale contains one or more convolutional layers for outputting object detection results. Based on the bounding box coordinates, object confidence, and category information, the detection head also includes an angle regression branch, also composed of one or more convolutional layers, for outputting the barrier's lift and landing angle θ, where θ ranges from [0° to 90°]. 2.5) Overall network integration and introduction of RepVGG module 2.5.1) Connect the backbone feature extraction network, attention feature extraction network, neck network, and detection head in the aforementioned input and output order to form an improved YOLOv5 network model; 2.5.2) The convolutional layers within the Csplayer layer in the backbone feature extraction network of the improved YOLOv5 network model, as well as the convolutional layers within the Csplayer layer used for feature fusion in the PANet structure in the neck network, introduce the reparameterization mechanism of the RepVGG module. That is, a multi-branch structure is adopted in the training phase, and the parameters of these multi-branch structures are equivalently merged into a single 3x3 convolution operation parameter in the inference phase.

3. The method for measuring the rise and fall time of an electric barrier machine based on a neural network model according to claim 2, characterized in that: The video image data acquisition in step 1) has a fixed video frame rate of 100 frames per second, and the marked key angle values ​​are at intervals of 5°.

4. The method for measuring the rise and fall time of an electric barrier machine based on a neural network model according to claim 3, characterized in that: The Csplayer layer in step 2.1.2 and the PANet structure in step 2.3.1 are composed of an input convolution layer, an output convolution layer, and multiple bottleneck layers (Bottleneck) in the middle. The bottleneck layer usually contains two 1x1 convolution layers and one 3x3 convolution layer with residual connections. The feature map is divided into two paths inside Csplayer, one is processed by the bottleneck layer, and the other is directly connected. Finally, the two features are fused through the Concat operation.

5. The method for measuring the rise and fall time of an electric barrier machine based on a neural network model according to claim 4, characterized in that: In step 2.3.1, the PANet structure generates three different scale output feature maps (P3_out, P4_out, and P5_out corresponding to small, medium, and large targets) for the detection head through a top-down feature pyramid path (up-sampling and backbone feature fusion followed by a Csplayer layer) and a bottom-up feature enhancement path (down-sampling and previous path feature fusion followed by a Csplayer layer).

6. The method for measuring the rise and fall time of an electric barrier machine based on a neural network model according to claim 5, characterized in that: The detection head in step 2.4.2) includes three detection units for different target scales, corresponding to the P3_out, P4_out and P5_out feature maps output by the neck network, to achieve multi-scale target detection.

Images