Pantograph net arcing identification method and device, electronic equipment and computer program product
By using pre-trained small models in the high-speed rail system for bow network position detection and arc recognition, the problem of low arc recognition accuracy in the prior art is solved, and higher recognition accuracy and system reliability are achieved.
Patent Information
- Application Number
- CN202411528809.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-02-21
AI Technical Summary
The existing artificial intelligence recognition model has low accuracy in arc recognition of high-speed rail contact networks, making it difficult to meet the high requirements of high-speed rail safe operation.
By acquiring the image of the bow net to be detected, the pre-trained target bow net position detection small model is used to locate the bow net position, and the image is cropped based on the position, and then the cropped bow net area image is input to the pre-trained target arc recognition small model , to achieve accurate detection of arcing of bow nets.
It improves the accuracy of arc recognition in the bow net, can monitor and early warning arc phenomena in real time, enhances vehicle safety and healthy operation and maintenance of power traction systems, and strengthens the reliability and efficiency of rail transit.
Smart Images
Figure CN119495052A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image recognition technology, and in particular, relates to a bow-catenary arcing recognition method, device, electronic equipment and computer program product. Background Art
[0002] With the rapid development of high-speed railway technology, high-speed railway has become an important link between cities. Its safe and stable operation plays a vital role in protecting the lives and property of passengers and maintaining the lifeline of national transportation. Under high-speed operation conditions, arcing is prone to occur between the pantograph and the contact network, which not only affects the efficiency of power transmission, but also may cause damage to the contact network and the pantograph. In severe cases, it may even cause power outages and safety accidents. Therefore, detecting whether there is arcing between the pantograph and the contact network is of great significance to the safe operation of trains.
[0003] Traditional arc detection methods mainly rely on manual inspections and periodic physical inspections, which require high manpower and material resources, and have poor real-time performance, making it difficult to meet the high requirements for safe operation of high-speed railways. With the continuous development of computer vision technology, intelligent and real-time artificial intelligence recognition models have become a new direction for identifying arc problems in high-speed railway contact networks. However, the arc recognition accuracy of existing artificial intelligence recognition models needs to be improved. Summary of the invention
[0004] The present application provides a bow-catenary arcing recognition method, device, electronic equipment and computer program product, which can solve the problem of low arcing recognition accuracy of artificial intelligence recognition models in existing solutions.
[0005] A first aspect of an embodiment of the present application provides a method for identifying arcing of a bow and a catenary, the method comprising:
[0006] Acquire the bow-net image to be detected;
[0007] Input the to-be-detected bow-net image into a pre-trained target bow-net position detection small model to obtain the bow-net position in the to-be-detected bow-net image;
[0008] The bow-net image to be detected is cropped based on the bow-net position to obtain a bow-net area image;
[0009] The bow-catenary area image is input into a pre-trained target arcing recognition small model to obtain a bow-catenary arcing detection result corresponding to the bow-catenary image to be detected.
[0010] A second aspect of the embodiments of the present application further provides a bow-catenary arcing identification device, the bow-catenary arcing identification device comprising:
[0011] An acquisition unit, used for acquiring the bow-net image to be detected;
[0012] A detection unit, used for inputting the bow-net image to be detected into a pre-trained small model for detecting the target bow-net position, to obtain the bow-net position in the bow-net image to be detected;
[0013] A clipping unit, used for clipping the bow-net image to be detected based on the bow-net position to obtain a bow-net area image;
[0014] The recognition unit is used to input the bow-catenary area image into a pre-trained target arcing recognition small model to obtain a bow-catenary arcing detection result corresponding to the bow-catenary image to be detected.
[0015] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the bow-net arcing identification method described in the first aspect are implemented.
[0016] A fourth aspect of an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the bow-catenary arcing identification method described in the first aspect are implemented.
[0017] In an embodiment of the present application, the position of the bow and net is first located by using a target bow and net position detection small model, and then the bow and net image to be detected is cropped based on the bow and net position to obtain a bow and net area image, and the bow and net area image is input into the target arcing recognition small model to obtain a bow and net arcing detection result corresponding to the bow and net image to be detected, thereby realizing bow and net arcing detection at the level of position detection task + classification task of whether there is arcing or not, improving the accuracy of bow and net arcing recognition, and solving the problem of low arcing recognition accuracy of the artificial intelligence recognition model in the existing solution, contributing intelligent power to vehicle safety and healthy operation and maintenance of electric traction systems, and enhancing the reliability and efficiency of rail transit. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 A schematic diagram of the implementation flow of the bow-catenary arcing identification method provided in an embodiment of the present application.
[0019] Figure 2 A schematic diagram of the training process of the small model for detecting the target bow-net position provided in an embodiment of the present application.
[0020] Figure 3 A schematic diagram of the structure of a small model for detecting the target bow-net position provided in an embodiment of the present application.
[0021] Figure 4 A schematic diagram of the structure of a multi-scale feature fusion module for a small model of target bow-net position detection provided in an embodiment of the present application.
[0022] Figure 5 A schematic diagram of the structure of a small model bow-catcher position output module for target bow-catcher position detection provided in an embodiment of the present application.
[0023] Figure 6 A schematic diagram of the implementation flow of the target bow-net position detection small model training step 204 provided in an embodiment of the present application.
[0024] Figure 7 A schematic diagram of the training process of the target arcing recognition small model provided in an embodiment of the present application.
[0025] Figure 8a A schematic diagram of the structure of a small target arcing identification model provided in an embodiment of the present application.
[0026] Figure 8b A schematic diagram of the structure of a small model arc classification head module for target arc identification provided in an embodiment of the present application.
[0027] Fig. 9 A schematic diagram of the optimization training process of the small model provided in an embodiment of the present application.
[0028] Fig.10 A schematic diagram of the structure of the bow-catenary arcing identification device provided in an embodiment of the present application.
[0029] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0030] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0031] It should be understood that in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0032] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0033] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.
[0034] With the rapid development of high-speed railway technology, high-speed railway has become an important link between cities. Its safe and stable operation plays a vital role in protecting the lives and property of passengers and maintaining the lifeline of national transportation. Under high-speed operation conditions, arcing is prone to occur between the pantograph and the contact network, which not only affects the efficiency of power transmission, but also may cause damage to the contact network and the pantograph. In severe cases, it may even cause power outages and safety accidents. Therefore, detecting whether there is arcing between the pantograph and the contact network is of great significance to the safe operation of trains.
[0035] Traditional arc detection methods mainly rely on manual inspections and periodic physical inspections, which require high manpower and material resources, and have poor real-time performance, making it difficult to meet the high requirements for safe operation of high-speed railways. With the continuous development of computer vision technology, intelligent and real-time artificial intelligence recognition models have become a new direction for identifying arc problems in high-speed railway contact networks. However, the arc recognition accuracy of existing artificial intelligence recognition models needs to be improved.
[0036] Based on this, the embodiments of the present application provide a bow-catenary arcing recognition method, device, electronic device and computer program product, which can solve the problem of low arcing recognition accuracy of artificial intelligence recognition models in existing solutions.
[0037] In an embodiment of the present application, the above-mentioned bow-catenary arcing identification method can be executed by an on-board computer on the train, or by a cloud server at the train operation end. The train can be a high-speed train, a motor train, a tram, or other rail transit vehicle powered by a bow-catenary system. The present application does not limit the specific type of train.
[0038] like Figure 1 As shown, it is a schematic diagram of the implementation process of a bow-catenary arcing identification method provided in an embodiment of the present application. The bow-catenary arcing identification method can be implemented by the following steps 101 to 104.
[0039] Step 101, obtaining the bow-net image to be detected.
[0040] In the embodiment of the present application, the above-mentioned bow-catcher image to be detected may be a bow-catcher image collected in real time during the running of the train.
[0041] Step 102, input the bow-net image to be detected into a pre-trained small model for detecting the target bow-net position, and obtain the bow-net position in the bow-net image to be detected.
[0042] In the embodiment of the present application, the small model for detecting the target bow-net position is an artificial intelligence detection model with a small number of parameters, suitable for deployment on embedded devices, and can run in real time.
[0043] In one embodiment of the present application, a small model for detecting the target bow-net position can be deployed on a chip with an AI computing power of 1.2 Tops, and the frame rate of the model can reach 15 frames per second.
[0044] Step 103, cropping the bow-net image to be detected based on the bow-net position to obtain a bow-net area image.
[0045] Step 104, input the bow-catenary area image into the pre-trained target arcing recognition small model to obtain the bow-catenary arcing detection result corresponding to the bow-catenary image to be detected.
[0046] In the embodiment of the present application, the target arcing recognition small model is similar to the above-mentioned target bow-net position detection small model. It is an artificial intelligence recognition model with a small number of parameters, suitable for deployment on embedded devices, and can run in real time.
[0047] In one embodiment of the present application, the target arcing recognition small model can be deployed on a chip with an AI computing power of 1.2Tops, and the frame rate of the model can reach 15 frames per second.
[0048] This application deploys a small model for target bow-net position detection and a small model for target arc identification on the embedded board to perform accurate visual analysis on the real-time acquired bow-net images to be detected, thereby achieving real-time monitoring and early warning of arcing phenomena, contributing intelligent power to vehicle safety and the healthy operation and maintenance of the electric traction system, and enhancing the reliability and efficiency of rail transit.
[0049] Specifically, in the embodiment of the present application, the position of the bow and net is first located by using the target bow and net position detection small model, and then the bow and net image to be detected is cropped based on the bow and net position to obtain the bow and net area image, and the bow and net area image is input into the target arcing recognition small model to obtain the bow and net arcing detection result corresponding to the bow and net image to be detected, thereby realizing the bow and net arcing detection at the level of position detection task + classification task of whether there is arcing or not, improving the accuracy of bow and net arcing recognition, and can solve the problem of low arcing recognition accuracy of the artificial intelligence recognition model in the existing solution, and can contribute intelligent power to vehicle safety and the healthy operation and maintenance of the electric traction system, and enhance the reliability and efficiency of rail transit.
[0050] In order to make the target bow-catenary position detection model an artificial intelligence model with high detection accuracy and high robustness, so as to improve the accuracy of bow-catenary position detection, and further improve the accuracy of bow-catenary arcing recognition. In one embodiment of the present application, Figure 2 As shown, the above-mentioned small model for detecting the target bow-net position can be trained by adopting the following steps 201 to 204.
[0051] Step 201, obtaining a first bow-net sample image data set and a first bow-net position label corresponding to each first bow-net sample image in the first bow-net sample image data set.
[0052] In practical applications, various types of bow net video materials collected in different weather conditions, different regions, and different time periods can be collected, and each frame of the image can be used as the first bow net sample image. By manually labeling, whether there is a bow net in each first bow net sample image and the position of the bow net in the image can be annotated to obtain the first bow net position label corresponding to each first bow net sample image.
[0053] Step 202, based on the first bow-net sample image data set and the first bow-net position label, the pre-constructed initial bow-net position detection small model and the initial bow-net position detection large model are trained to obtain the bow-net position detection small model to be optimized and the target bow-net position detection large model.
[0054] In the embodiment of the present application, the target bow-net position detection large model is a position detection model with a large number of parameters and high performance. It is suitable for automatic labeling of sample images (the second bow-net sample image data set) and is not suitable for deployment on embedded devices.
[0055] In one embodiment of the present application, the initial pantograph-catenary position detection large model may be a detection model constructed based on an open source detection model Grounding DINO.
[0056] In one embodiment of the present application, Figure 3 As shown, the initial bow-net position detection model may include a multi-scale feature extraction module 31, a multi-scale feature fusion module 32 and a bow-net position output module 33.
[0057] In one embodiment of the present application, Figure 3 As shown, the multi-scale feature extraction module may include five sub-modules, namely stem, stage1, stage2, stage3 and stage4, for extracting multi-scale features of the first bow-net sample image.
[0058] Among them, in the stem submodule, the input image with a resolution of 288×512 is downsampled by a convolution layer with a stride of 2 and a convolution kernel size of 3×3, and the output feature map is 16×144×256.
[0059] In the stage1 submodule, the feature map of size 16×144×256 output by the stem submodule is first downsampled by a convolution layer with a stride of 2 and a convolution kernel size of 3×3, and then extracted by two consecutive convolution layers with a convolution kernel size of 3×3, and the output feature map is 32×72×128.
[0060] In the stage2 submodule, the feature map of size 32×72×128 output by the stage1 submodule is first downsampled by a convolution layer with a stride of 2 and a convolution kernel size of 3×3, and then feature extracted by three consecutive convolution layers with a convolution kernel size of 3×3, and the output feature map is 64×36×64.
[0061] In the stage3 submodule: the feature map of size 64×36×64 output by the stage2 submodule is first downsampled by a convolution layer with a stride of 2 and a convolution kernel size of 3×3, and then feature extracted by 3 consecutive convolution layers with a convolution kernel size of 3×3, and the output feature map is 128×18×32.
[0062] In the stage4 submodule: the feature map of size 128×18×32 output by the stage3 submodule is first downsampled by a convolution layer with a stride of 2 and a convolution kernel size of 3×3, and then feature extracted by two consecutive convolution layers with a convolution kernel size of 3×3, and the output feature map is 256×9×16.
[0063] In one embodiment of the present application, Figure 4 As shown, the multi-scale feature fusion module may include four convolution units (a first convolution unit, a second convolution unit, a third convolution unit, and a fourth convolution unit), two deconvolution units (a first deconvolution unit and a second deconvolution unit), and two combination units (a first combination unit and a second combination unit).
[0064] The first convolution unit convolves the feature map of size 64×36×64 output by the stage2 submodule through a convolution layer with a convolution kernel size of 3×3 to output a feature map of size 64×36×64.
[0065] The second convolution unit convolves the feature map of size 128×18×32 output by the stage3 submodule through a convolution layer with a convolution kernel size of 3×3 and outputs a feature map of size 64×18×32.
[0066] The third convolution unit convolves the feature map of size 256×9×16 output by the stage4 submodule through a convolution layer with a convolution kernel size of 3×3 to output a feature map of size 64×9×16.
[0067] The first deconvolution unit operates the feature map of size 64×9×16 output by the third convolution unit through a deconvolution layer with a step size of 2 and a deconvolution kernel size of 3×3, and outputs a feature map of size 64×18×32.
[0068] The first combination unit combines the feature map of size 64×18×32 output by the first deconvolution unit with the feature map of size 64×18×32 output by the second convolution unit, and outputs a feature map of size 128×18×32.
[0069] The second deconvolution unit operates the feature map of size 128×18×32 output by the first combination unit through a deconvolution layer with a step size of 2 and a deconvolution kernel size of 3×3, and outputs a feature map of size 64×36×64.
[0070] The second combination unit combines the feature map of size 64×36×64 output by the second deconvolution unit with the feature map of size 64×36×64 output by the first convolution unit, and outputs a feature map of size 128×36×64.
[0071] The fourth convolution unit convolves the feature map of size 128×36×64 output by the second combination unit through a convolution layer with a convolution kernel size of 3×3 to output a feature map of size 64×36×64. The feature map of size 64×36×64 is the final output result of the multi-scale feature fusion module.
[0072] In one embodiment of the present application, Figure 5 As shown, the bow-net position output module may include a feature extraction unit, a classification unit and a position output unit; wherein the feature extraction unit extracts high-level semantic features from the feature map of size 64×36×64 output by the multi-scale feature fusion module through two convolution layers with a convolution kernel size of 3×3, and then outputs a feature map of size 64×36×64. The classification unit performs a classification task on the feature map of size 64×36×64 output by the feature extraction unit through two convolution layers with a convolution kernel size of 3×3, and outputs a classification result of whether the bow-net is included. The position output unit performs a regression task on the feature map of size 64×36×64 output by the feature extraction unit through two convolution layers with a convolution kernel size of 3×3, and outputs the four position coordinates of the bow-net, which are used to locate the position of the bow-net in the image.
[0073] In an embodiment of the present application, in the above step 202, the first bow-net sample image can be input into a pre-constructed initial bow-net position detection small model to obtain a prediction result output by the initial bow-net position detection small model; then the prediction result is compared with the first bow-net position label obtained by manual annotation, and after adjusting the parameters of the initial bow-net position detection small model based on the comparison result, the initial bow-net position detection small model is iteratively trained to finally obtain the bow-net position detection small model to be optimized.
[0074] Similarly, the first bow-net sample image can be input into the pre-built initial bow-net position detection large model to obtain the prediction result output by the initial bow-net position detection large model; then, after comparing the prediction result with the first bow-net position label obtained by manual annotation, the parameters of the initial bow-net position detection large model are adjusted based on the comparison result, and the initial bow-net position detection large model is iteratively trained to finally obtain the target bow-net position detection large model.
[0075] In the implementation of this application, the process of obtaining the small bow-net position detection model to be optimized and the large target bow-net position detection model through the above-mentioned training can refer to the training process of the model in the relevant technology, and this application does not limit its specific training method.
[0076] Step 203, obtaining a second bow-net sample image data set.
[0077] In an embodiment of the present application, the second bow-net sample image in the second bow-net sample image data set is different from the first bow-net sample image in the first bow-net sample image data set; that is, they belong to different data sets, and the sample size in the second bow-net sample image data set can be much larger than the sample size in the first bow-net sample image data set.
[0078] For example, the first bow-net sample image data set may include 50,000 first bow-net sample images, and the second bow-net sample image data set may include 1 million second bow-net sample images.
[0079] Step 204, optimizing and training the small model for detecting the position of the bow and the net to be optimized based on the second bow and the net sample image data set and the large model for detecting the position of the target bow and the net, to obtain the small model for detecting the position of the target bow and the net.
[0080] In an embodiment of the present application, in order to improve the robustness and stability of the small bow-net position detection model to be optimized, in an embodiment of the present application, the small bow-net position detection model to be optimized is optimized and trained based on the second bow-net sample image data set and the target bow-net position detection large model, so that the final target bow-net position detection small model can perform bow-net position detection on bow-net images in complex scenes to meet the high requirements for safe operation of high-speed railways.
[0081] Specifically, since the small model for detecting the position of the bow and net to be optimized has a smaller number of parameters than the large model for detecting the position of the bow and net to be targeted, when the same number of sample images is used for training, the detection accuracy of the small model for detecting the position of the bow and net to be optimized is low, and it is unable to accurately detect the position of the bow and net images in complex scenes, and there may be omissions or false detections. The large model for detecting the position of the bow and net to be targeted has a large number of parameters, high robustness and stability, and can accurately detect the position of the bow and net images in complex scenes. Therefore, the large model for detecting the position of the bow and net to be targeted can be used to annotate the second bow and net sample images in the second bow and net sample image data set, and obtain the second bow and net position label corresponding to each second bow and net sample image in the second bow and net sample image data set, so as to replace manual annotation, thereby saving manpower and material resources for sample annotation, and then, the small model for detecting the position of the bow and net to be optimized can be optimized and trained using the second bow and net sample image data set and the second bow and net position label, so as to improve the robustness and stability of the small model for detecting the position of the bow and net to be optimized, and finally obtain the small model for detecting the position of the bow and net to be targeted.
[0082] In order to make the optimization training more targeted, in one embodiment of the present application, Figure 6 As shown, the above step 204, in the process of optimizing and training the small bow-net position detection model to be optimized based on the second bow-net sample image data set and the target bow-net position detection large model to obtain the target bow-net position detection small model, can be implemented by the following steps 601 to 603.
[0083] Step 601, input each second bow-net sample image in the second bow-net sample image data set into the small model for detecting the bow-net position to be optimized and the large model for detecting the target bow-net position, respectively, to obtain the first position prediction result output by the small model for detecting the bow-net position to be optimized and the second position prediction result output by the large model for detecting the target bow-net position.
[0084] Step 602, when the first position prediction result is inconsistent with the second position prediction result, the corresponding second bow-net sample image is used as the difficult sample image, and the second position prediction result corresponding to the difficult sample image output by the target bow-net position detection model is used as the second bow-net position label corresponding to the difficult sample image.
[0085] Since the target bow-net position detection large model has high robustness and stability, therefore, when the first position prediction result is inconsistent with the second position prediction result, the second position prediction result output by the target bow-net position detection large model can be assumed to be the accurate prediction result, and the first position prediction result output by the small bow-net position detection model to be optimized is the prediction result that needs to be discarded, and the second bow-net sample image corresponding to the inconsistency between the first position prediction result and the second position prediction result is used as the difficult sample image corresponding to the small bow-net position detection model to be optimized, and the second position prediction result corresponding to the difficult sample image output by the target bow-net position detection large model is used as the second bow-net position label corresponding to the difficult sample image.
[0086] Step 603, optimizing and training the small model for detecting the position of the bow and the reticle to be optimized based on the difficult sample image and the second bow and the reticle position label to obtain the small model for detecting the position of the target bow and the reticle.
[0087] In the embodiment of the present application, by using the target bow-net position detection large model to screen out difficult sample images that have errors in recognition and cannot be correctly recognized by the small bow-net position detection model to be optimized, and then using the difficult sample images and the second bow-net position label to perform targeted training on the small bow-net position detection model to be optimized, the obtained target bow-net position detection small model is a model with higher robustness and stability, which improves the accuracy of bow-net position detection in complex scenarios, helps to improve the accuracy of bow-net arcing recognition, and at the same time, also improves the efficiency of optimizing the training of the small bow-net position detection model to be optimized.
[0088] In one embodiment of the present application, in order to make the target arcing recognition model an artificial intelligence model with high detection accuracy and high robustness, in one embodiment of the present application, Figure 7 As shown, the above-mentioned target arcing recognition small model can be trained by adopting the following steps 701 to 704.
[0089] Step 701, obtaining a first bow-catenary region sample image data set and a first arcing label corresponding to each first bow-catenary region sample image in the first bow-catenary region sample image data set.
[0090] In an embodiment of the present application, the above-mentioned first bow-net area sample image data set can be obtained based on the above-mentioned first bow-net sample image data set and the first bow-net position label corresponding to each first bow-net sample image in the first bow-net sample image data set.
[0091] Specifically, based on the first bow-net position label obtained by manual annotation, each first bow-net sample image can be cropped to obtain a first bow-net area sample image data set.
[0092] In an embodiment of the present application, after obtaining the first bow-net area sample image data set, each first bow-net area sample image in the first bow-net area sample image data set is manually labeled to determine whether arcing occurs, so that the first arcing label corresponding to each first bow-net area sample image can be obtained.
[0093] Step 702, based on the first bow-catenary area sample image data set and the first arcing label, the pre-constructed initial arcing recognition small model and the initial arcing recognition large model are trained to obtain the arcing recognition small model to be optimized and the target arcing recognition large model.
[0094] In the embodiment of the present application, the target arc recognition model is an arc recognition model with a large number of parameters and high performance. It is suitable for automatic labeling of sample images (for example, the second bow-net area sample image data set), but is not suitable for deployment on embedded devices.
[0095] In one embodiment of the present application, the initial arcing identification model may be an arcing identification model constructed based on the open source model DINOV2.
[0096] In the present application embodiment, Figure 8a As shown, the initial arcing identification model may include a multi-scale feature extraction module 81 and an arcing classification head module 82. Figure 8a As shown, the structure of the multi-scale feature extraction module 81 in the initial arcing identification model can be the same as the structure of the multi-scale feature extraction module 31 in the initial bow-net position detection model, the only difference is the resolution of the input image.
[0097] In the embodiment of the present application, the size of the feature map output by the multi-scale feature extraction module 81 in the initial arcing recognition small model can be 256×4×8.
[0098] In one embodiment of the present application, Figure 8b As shown, the arc classification head module may include two convolution units (the fifth convolution unit and the sixth convolution unit), a global average pooling layer, two fully connected layers (the first fully connected layer and the second fully connected layer) and a classification decision layer.
[0099] The feature map size output by the multi-scale feature extraction module in the initial arcing recognition model is 256×4×8. The 256×4×8 feature map is downsampled by the convolution layer with a step size of 2 and a convolution kernel size of 3×3 in the fifth convolution unit to output a feature map of size 256×2×2. Then, the 256×2×2 feature map is feature extracted by the convolution layer with a convolution kernel size of 3×3 in the sixth convolution unit, and a feature map of size 128×2×2 is output. Then, the 128×2×2 feature map is pooled by the global average pooling (GAP) layer to output 128-dimensional features. The 128-dimensional features are first reduced to 64 dimensions by the first fully connected layer, and then reduced to 2 dimensions by the second fully connected layer. Finally, after passing through the classification decision layer (for example, the classification decision layer is composed of a normalized exponential function (softmax activation function)), the arcing classification score is obtained. In the embodiment of the present application, a classification score greater than a score threshold may be determined as arcing, and a classification score less than or equal to the score threshold may be determined as arcing not existing. The score threshold may be preset based on actual application scenarios.
[0100] In an embodiment of the present application, in the above-mentioned step 702, based on the first bow-net area sample image data set and the first arcing label, the pre-constructed initial arcing recognition small model and the initial arcing recognition large model are trained to obtain the arcing recognition small model to be optimized and the target arcing recognition large model. In the process, the first bow-net area sample image can be input into the pre-constructed initial arcing recognition small model to obtain the prediction result of the output of the initial arcing recognition small model; then the prediction result is compared with the first arcing label obtained by manual annotation, and after adjusting the parameters of the initial arcing recognition small model based on the comparison result, the initial arcing recognition small model is iteratively trained to finally obtain the arcing recognition small model to be optimized.
[0101] Similarly, the sample image of the first bow-net area can be input into the pre-built initial arcing recognition large model to obtain the prediction result of the output of the initial arcing recognition large model; then the prediction result is compared with the first arcing label obtained by manual annotation, and the parameters of the initial arcing recognition large model are adjusted based on the comparison result. Then, the initial arcing recognition large model is iteratively trained to finally obtain the target arcing recognition large model.
[0102] Step 703, obtaining a second pantograph area sample image dataset.
[0103] In an embodiment of the present application, in an embodiment of the present application, the above-mentioned second bow-net area sample image dataset can be obtained based on the second bow-net sample image dataset in the aforementioned step 203.
[0104] Specifically, by inputting the second bow-net sample images in the second bow-net sample image data set into the target bow-net position detection large model, the bow-net position detection result corresponding to each second bow-net sample image output by the target bow-net position detection large model is obtained, and the second bow-net sample image containing the bow-net is cropped based on the bow-net position detection result to obtain the second bow-net area sample image data set.
[0105] Step 704, optimizing and training the arcing recognition small model to be optimized based on the second bow-catenary area sample image data set and the target arcing recognition large model to obtain the target arcing recognition small model.
[0106] In order to improve the robustness and stability of the small arc recognition model to be optimized, in an embodiment of the present application, the small arc recognition model to be optimized is optimized and trained based on the second bow-net area sample image data set and the target arc recognition large model, so that the final target arc recognition small model can perform arc recognition on the bow-net images in complex scenes, so as to meet the high requirements for safe operation of high-speed railways.
[0107] Specifically, since the small model for arc recognition to be optimized has a smaller number of parameters than the large model for target arc recognition, when the same number of sample images is used for training, the robustness and stability of the small model for arc recognition to be optimized are low, and it is unable to perform arc recognition on bow-net images in complex scenes. However, the large model for target arc recognition has a large number of parameters, high robustness and stability, and can perform more accurate arc recognition on bow-net images in complex scenes. Therefore, the large model for target arc recognition can be used to annotate the second bow-net area sample images in the second bow-net area sample image data set, and obtain the second bow-net arc label corresponding to each second bow-net area sample image in the second bow-net sample image data set, so as to replace manual annotation, thereby saving manpower and material resources for sample annotation, and then, the second bow-net sample image data set and the second bow-net arc label are used to optimize the training of the small model for arc recognition to improve the robustness and stability of the small model for arc recognition to be optimized, and finally obtain the small model for target arc recognition.
[0108] In order to make the optimization training more targeted, in one embodiment of the present application, the above-mentioned step 704, in the process of optimizing the arc recognition small model to be optimized based on the second bow-net area sample image data set and the target arc recognition large model to obtain the target arc recognition small model, can be implemented by the following steps 801 to 803.
[0109] Step 801, input each sample image of the second bow-catenary area in the second bow-catenary area sample image data set into the small arc recognition model to be optimized and the target arc recognition large model respectively, to obtain the first arc prediction result output by the small arc recognition model to be optimized and the second arc prediction result output by the target arc recognition large model.
[0110] Step 802, when the first arcing prediction result is inconsistent with the second arcing prediction result, the corresponding second bow-net area sample image is used as the difficult sample image, and the second arcing prediction result corresponding to the difficult sample image output by the target arcing recognition large model is used as the second bow-net arcing label corresponding to the difficult sample image.
[0111] Since the target arcing recognition large model has high robustness and stability, when the first arcing prediction result is inconsistent with the second arcing prediction result, the second arcing prediction result output by the target arcing recognition large model can be assumed to be the accurate prediction result, and the first arcing prediction result output by the small arcing recognition model to be optimized is the prediction result that needs to be discarded. The second bow-net area sample image corresponding to the inconsistency between the first arcing prediction result and the second arcing prediction result is used as the difficult sample image corresponding to the small arcing recognition model to be optimized, and the second arcing prediction result corresponding to the difficult sample image output by the target arcing recognition large model is used as the second bow-net arcing label corresponding to the difficult sample image.
[0112] Step 803, optimizing and training the arcing recognition small model to be optimized based on the difficult sample image and the second bow-net arcing label to obtain the target arcing recognition small model.
[0113] In an embodiment of the present application, by using the target arc recognition large model to screen out difficult sample images in which the small arc recognition model to be optimized has recognition errors and cannot correctly recognize, and then using the difficult sample images and the second arc label to perform targeted training on the small arc recognition model to be optimized, the obtained target arc recognition small model is a model with higher robustness and stability, which improves the accuracy of arc recognition in complex scenarios, helps to improve the accuracy of bow-net arc recognition, and at the same time, also improves the efficiency of optimization training of the small arc recognition model to be optimized.
[0114] In one embodiment of the present application, after the target bow-net position detection small model and the target arc recognition small model are deployed in an embedded device, difficult sample images of the target bow-net position detection small model and the target arc recognition small model can be continuously collected, so that the target bow-net position detection small model and the target arc recognition small model can further improve the robustness and stability of the model after learning the difficult sample images to meet the accuracy requirements of different application scenarios.
[0115] In order to facilitate understanding of the optimization training process of the small model for detecting the pantograph position to be optimized and the small model for identifying the arcing to be optimized in this application, this application combines Fig. 9 Steps 901 to 909 shown illustrate the specific implementation process of the optimization training of the small model for detecting the pantograph position to be optimized and the small model for identifying the arcing to be optimized in actual applications.
[0116] Step 901, collect a large number of high-speed rail catenary videos.
[0117] Step 902: extract the current frame image.
[0118] Step 903, using the small model for detecting the position of the bow and the catenary to be optimized and the large model for detecting the position of the target bow and the catenary to locate the position of the bow and the catenary in the current frame image respectively.
[0119] Step 904, determine whether the coordinates of the bow-net position output by the small model for detecting the bow-net position to be optimized and the large model for detecting the target bow-net position are consistent. If they are consistent, execute step 906; if they are inconsistent, execute step 905.
[0120] Step 905, update the current frame image and the bow-net position coordinates output by the target bow-net position detection large model to the difficult example sample data set of the bow-net position detection small model, and then execute step 906.
[0121] Step 906, cropping the current frame image based on the bow-catcher position coordinates output by the target bow-catcher position detection large model to obtain the current frame bow-catcher area image.
[0122] Step 907, using the small arc identification model to be optimized and the large target arc identification model to respectively identify whether there is spark (arcing) in the bow-net area image of the current frame, and output the identification result.
[0123] Step 908, determine whether the recognition results output by the small arc recognition model to be optimized and the target arc recognition large model are consistent. If they are consistent, return to step 902, if not, execute step 909.
[0124] Step 909, update the current frame bow-net area image and the recognition result output by the target arcing recognition large model to the arcing recognition small model difficult example sample data set, and return to execute step 902.
[0125] Step 910, after extracting all single-frame images in the high-speed railway bow-net video, the bow-net position detection small model to be optimized is optimized and trained based on the difficult samples in the difficult sample data set of the bow-net position detection small model, and the arc recognition small model to be optimized is optimized and trained based on the difficult samples in the difficult sample data set of the arc recognition small model.
[0126] In a practical application of the present application, Fig. 9 The differentiated data mining shown in the figure collected 450,000 samples. The final training samples are 50,000 manually labeled samples + 450,000 samples automatically labeled by the large model. The accuracy of the small model trained with 500,000 samples is much better than that of the small model trained with 50,000 samples, which improves the arc recognition accuracy and solves the problem of low arc recognition accuracy of the artificial intelligence recognition model in the existing solution.
[0127] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, and in some embodiments of the present application, certain steps may be performed in other orders.
[0128] The present application also provides a device for identifying arcing of a bow-cat, such as Fig.10 As shown, the bow-catenary arcing identification device 10 may include:
[0129] An acquisition unit 11 is used to acquire a bow-net image to be detected;
[0130] The detection unit 12 is used to input the image of the bow and net to be detected into the pre-trained small model for detecting the position of the target bow and net, and obtain the position of the bow and net in the image of the bow and net to be detected;
[0131] A clipping unit 13, used for clipping the bow-net image to be detected based on the bow-net position to obtain a bow-net area image;
[0132] The recognition unit 14 is used to input the bow-catenary area image into a pre-trained target arcing recognition small model to obtain a bow-catenary arcing detection result corresponding to the bow-catenary image to be detected.
[0133] It should be noted that, for the convenience and simplicity of description, the specific working process of the above-described pantograph-catenary arcing identification device 10 can refer to the above-mentioned Figures 1 to 5 The corresponding process of the method described in will not be repeated here.
[0134] like Fig.11 As shown, the embodiment of the present application also provides an electronic device. Fig.11 As shown, the electronic device may include: a processor 110, a memory 111, and a computer program 112 stored in the memory 111 and executable on the processor 110. When the processor 110 executes the computer program 112, the steps in the above-mentioned various embodiments of the bow-catenary arcing identification method are implemented, for example, Figure 1 Steps 101 to 104 are shown.
[0135] The processor 110 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0136] The memory 111 may be an internal storage unit of the electronic device, such as a hard disk or a memory. The memory 111 may also be an external storage device for the electronic device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 111 may also include both an internal storage unit of the electronic device and an external storage device. The memory 111 is used to store the above-mentioned computer program and other programs and data required by the electronic device.
[0137] The computer program may be divided into one or more units, which are stored in the memory 111 and executed by the processor 110 to complete the present application. The one or more units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the process of the computer program executing the bow-catenary arcing identification method in an electronic device.
[0138] For example, the above computer program can be divided into: an acquisition unit, a detection unit, a cropping unit and a recognition unit, and the specific functions are as follows:
[0139] An acquisition unit, used for acquiring the bow-net image to be detected;
[0140] A detection unit, used for inputting the bow-net image to be detected into a pre-trained small model for detecting the target bow-net position, and obtaining the bow-net position in the bow-net image to be detected;
[0141] A clipping unit, used for clipping the bow-net image to be detected based on the bow-net position to obtain a bow-net area image;
[0142] The recognition unit is used to input the bow-catenary area image into a pre-trained target arcing recognition small model to obtain the bow-catenary arcing detection result corresponding to the bow-catenary image to be detected.
[0143] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0144] The embodiments of the present application also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, it is used to implement the steps of the bow-catenary arcing identification method in the aforementioned embodiments.
[0145] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0146] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0147] In the embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the electronic device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, systems or units, which can be electrical, mechanical or other forms.
[0148] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0149] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0150] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or recording medium that can carry computer program code, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.
[0151] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A method for identifying arcing between a bow and a catenary, characterized in that: The method for identifying arcing of a bow-catenary includes: Acquire the bow-net image to be detected; Input the to-be-detected bow-net image into a pre-trained target bow-net position detection small model to obtain the bow-net position in the to-be-detected bow-net image; The bow-net image to be detected is cropped based on the bow-net position to obtain a bow-net area image; The bow-catenary area image is input into a pre-trained target arcing recognition small model to obtain a bow-catenary arcing detection result corresponding to the bow-catenary image to be detected.
2. The method for identifying arcing between a bow and a catenary according to claim 1, characterized in that: The training steps of the target bow-net position detection small model include: Acquire a first bow-net sample image data set and a first bow-net position label corresponding to each first bow-net sample image in the first bow-net sample image data set; Based on the first bow-net sample image data set and the first bow-net position label, a pre-constructed initial bow-net position detection small model and an initial bow-net position detection large model are trained to obtain a bow-net position detection small model to be optimized and a target bow-net position detection large model; Obtain a second bownet sample image data set; Based on the second bow-net sample image data set and the target bow-net position detection large model, the small bow-net position detection model to be optimized is optimized and trained to obtain the target bow-net position detection small model.
3. The method for identifying arcing between a bow and a catenary according to claim 2, characterized in that: The optimizing and training the small model for detecting the position of the bow and the catenary to be optimized based on the second bow and the catenary sample image data set and the large model for detecting the position of the target bow and the catenary to obtain the small model for detecting the position of the target bow and the catenary, comprises: Input each second bow-net sample image in the second bow-net sample image data set into a small bow-net position detection model to be optimized and a large bow-net position detection model to be targeted, respectively, to obtain a first position prediction result output by the small bow-net position detection model to be optimized and a second position prediction result output by the large bow-net position detection model to be targeted; The second bow-net sample image corresponding to the inconsistency between the first position prediction result and the second position prediction result is used as a difficult sample image, and the second position prediction result corresponding to the difficult sample image output by the target bow-net position detection large model is used as the second bow-net position label corresponding to the difficult sample image; The small bow-net position detection model to be optimized is optimized and trained based on the difficult sample image and the second bow-net position label to obtain the small bow-net position detection model of the target.
4. The method for identifying arcing between a bow and a catenary according to claim 2 or 3, characterized in that: The initial bow-net position detection small model includes a multi-scale feature extraction module, a multi-scale feature fusion module and a bow-net position output module.
5. The method for identifying arcing between a bow and a catenary according to any one of claims 1 to 3, characterized in that: The training steps of the target arcing recognition small model include: Acquire a first bow-catenary region sample image data set and a first arcing label corresponding to each first bow-catenary region sample image in the first bow-catenary region sample image data set; Based on the first bow-catenary area sample image data set and the first arcing label, a pre-constructed initial arcing recognition small model and an initial arcing recognition large model are trained to obtain an arcing recognition small model to be optimized and a target arcing recognition large model; Obtain a sample image data set of the second bow-net area; The small arcing recognition model to be optimized is optimized and trained based on the second bow-catenary area sample image data set and the target arcing recognition large model to obtain the target arcing recognition small model.
6. The method for identifying arcing between a bow and a catenary according to claim 5, characterized in that: The optimizing and training the arcing recognition small model to be optimized based on the second pantograph-catenary region sample image data set and the target arcing recognition large model to obtain the target arcing recognition small model includes: Input each second bow-catenary area sample image in the second bow-catenary area sample image data set into the small arcing recognition model to be optimized and the target arcing recognition large model respectively, to obtain the first arcing prediction result output by the small arcing recognition model to be optimized and the second arcing prediction result output by the target arcing recognition large model; The second bow-net area sample image corresponding to the inconsistency between the first arcing prediction result and the second arcing prediction result is used as a difficult sample image, and the second arcing prediction result corresponding to the difficult sample image output by the target arcing recognition large model is used as the second bow-net arcing label corresponding to the difficult sample image; The arcing recognition small model to be optimized is optimized and trained based on the difficult sample image and the second bow-net arcing label to obtain the target arcing recognition small model.
7. The method for identifying arcing between a bow and a catenary according to any one of claims 1 to 3, characterized in that: The bow-net image to be detected is a real-time bow-net image.
8. A bow-cat arcing identification device, characterized in that: The pantograph-catenary arcing identification device comprises: An acquisition unit, used for acquiring the bow-net image to be detected; A detection unit, used for inputting the bow-net image to be detected into a pre-trained small model for detecting the target bow-net position, to obtain the bow-net position in the bow-net image to be detected; A clipping unit, used for clipping the bow-net image to be detected based on the bow-net position to obtain a bow-net area image; The recognition unit is used to input the bow-catenary area image into a pre-trained target arcing recognition small model to obtain a bow-catenary arcing detection result corresponding to the bow-catenary image to be detected.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the steps of the bow-catenary arcing identification method as described in any one of claims 1 to 7 are implemented.
10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the bow-catenary arcing identification method as claimed in any one of claims 1 to 7 are implemented.