A method, apparatus, medium, and equipment for detecting pantograph anomalies
By acquiring and aligning the depth map of the pantograph and detecting the optical flow estimates using an abnormality detection model, the problem of low accuracy and efficiency of pantograph abnormality detection in the prior art is solved, and high accuracy and low error detection are achieved The abnormal detection effect of rate improves the safety of train driving.
Patent Information
- Application Number
- CN202311738218.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-18
- Publication Date
- 2025-02-14
- Estimated Expiration
- 2043-12-18
AI Technical Summary
The prior art has problems of low accuracy and efficiency in pantograph abnormal detection, especially when the probability of foreign object invasion and the number of abnormal samples is limited, it is difficult to complete model training, resulting in poor implementation of the detection method.
The real-time depth map and standard depth map of the pantograph to be measured are obtained, and the depth map is aligned using the alignment network, and the real-time depth alignment map is detected in combination with the pre-obtained anomaly detection model. By comparing the optical flow estimate with the preset threshold, Determine whether there is any abnormality in the pantograph.
It improves the accuracy and efficiency of pantograph abnormal detection, and can accurately identify abnormalities of different types and sizes in complex environments. It has high accuracy and low error detection rate, effectively improving train driving safety.
Smart Images
Figure CN117710331B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a pantograph abnormality detection method, device, medium and equipment, belonging to the technical field of image recognition. Background Art
[0002] With the rapid development of railway industry, railway is one of the main modes of transportation for people to travel, and its operation safety has received extensive attention. Pantograph is one of the main components that provide energy to electric locomotives. If there is foreign matter on the pantograph, it will cause serious traffic accidents. When the pantograph is invaded by foreign matter (such as plastic bags, sky lanterns, branches, etc.), it will cause the pantograph to get stuck, the net to be pulled, the line to catch fire and other faults. For this reason, once the pantograph is in an abnormal state of foreign matter intrusion, it is necessary to stop the train immediately for inspection and rectification to ensure the safe operation of the train. Therefore, accurate and timely detection of foreign matter on the pantograph is of great significance to the maintenance of the pantograph and driving safety.
[0003] The traditional method of detecting foreign objects in pantographs is to manually climb to the top of the vehicle and conduct inspections with rooftop video recordings. Manual inspections of key areas of pantographs consume a lot of manpower and pose safety risks. Rooftop video recordings rely on manual observation, which is prone to visual fatigue and leads to missed inspections.
[0004] In the existing technology, the deep learning foreign body detection method based on supervised yolov5 or yolov8 needs to obtain a certain amount of data of foreign body intrusion images to ensure the accuracy of the training model. However, in the actual train operation scenario, the probability of foreign body intrusion in the pantograph is relatively low, and the number of abnormal samples is limited, making it difficult to complete the model training, resulting in poor feasibility of the above methods and inability to promote their application. Summary of the invention
[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a pantograph abnormality detection method, device, medium and equipment, which can improve the accuracy and efficiency of pantograph abnormality detection. To achieve the above purpose, the present invention is implemented by adopting the following technical solutions:
[0006] In a first aspect, the present invention provides a pantograph abnormality detection method, comprising:
[0007] Obtain the real-time depth map and standard depth map of the pantograph to be tested;
[0008] Using the alignment network, the acquired real-time depth map is aligned with the standard depth map to obtain a real-time depth alignment map;
[0009] Using a pre-obtained anomaly detection model to detect the real-time depth alignment image, to obtain an optical flow estimation value;
[0010] The optical flow estimation value is compared with a preset first threshold value. When the optical flow estimation value is greater than the first preset threshold value, it is output that there is an abnormality in the pantograph to be tested. When the optical flow estimation value is less than or equal to the first preset threshold value, it is output that there is no abnormality in the pantograph to be tested.
[0011] In combination with the first aspect, optionally, the pre-obtained anomaly detection model is obtained by the following steps:
[0012] Obtain the inspection results, historical depth map and corresponding standard depth map of the pantograph that has been inspected;
[0013] Using the alignment network, the acquired historical depth map is aligned with the corresponding standard depth map to obtain a historical depth alignment map;
[0014] Based on the historical depth alignment graph of pantographs with normal detection results, the optimizer is used to optimize the parameters of the pre-built anomaly detection model; the loss function is used to evaluate the optimized trained anomaly detection model to obtain the trained anomaly detection model;
[0015] Based on the historical depth alignment graph of pantographs with abnormal detection results, the trained anomaly detection model is input to detect whether the training of the anomaly detection model is completed, and in response to the completion of the training, the pre-obtained anomaly detection model is obtained.
[0016] In combination with the first aspect, optionally, using an alignment network to align the acquired depth map with the standard depth map includes:
[0017] Perform geometric transformation on the acquired depth map to obtain images with different posture relationships;
[0018] Input the pictures of different poses and the standard depth map into the alignment network to extract feature points and descriptors respectively;
[0019] Based on the extracted feature points and descriptors, the images of different positions are matched with the standard depth map to obtain the matching relationship between the feature points, form feature point pairs, and calculate the matching degree of the feature point pairs;
[0020] A preset second threshold is used to select feature point pairs with a high matching degree, and an overlapping area is extracted through affine transformation, and the overlapping area is used as a depth alignment map.
[0021] In combination with the first aspect, optionally, the pre-built anomaly detection model includes:
[0022] Feature extraction module: used to extract features from the input image using a set of convolutional layers and output a first feature map;
[0023] CBAM attention mechanism module: used to further extract the features of the key area from the first feature map using the channel attention mechanism and the spatial attention mechanism, and output the second feature map;
[0024] The first channel processing module is used to perform channel rearrangement and channel cutting on the second feature map, and output a third feature map;
[0025] Cross-scale processing: used to use multiple sets of convolutional layers to process features of different scales on the third feature map, capture motions of different scales, and output initial optical flow estimates of different scales;
[0026] The second channel processing module is used to perform channel splicing on the initial optical flow estimation of different scales and output the optical flow estimation value.
[0027] In combination with the first aspect, optionally, the loss function is to add the numerical losses of the initial optical flow estimation at different scales and divide the result by the number of initial optical flow estimation elements, and use gradient clipping to keep the gradient stable, wherein the maximum gradient norm is 1,
[0028] The loss function is calculated by formula (1):
[0029]
[0030] In formula (1), L(y) is the loss function of the output optical flow estimation value y, p is the true value of the optical flow estimation value of the pantograph with a normal detection result, Y(y) is the predicted value of the optical flow estimation value of the pantograph with a normal detection result, z is the vector obtained by combining the initial optical flow estimations of different scales, For bias.
[0031] In combination with the first aspect, optionally, the cross-scale processing is expressed by formula (2):
[0032]
[0033]
[0034] In formula (2), y out,1 is the initial optical flow estimate of the first scale, y in,1 is the first scale feature map output by the CBAM attention mechanism module, y out,2 is the initial optical flow estimate of the second scale, y in,2 It is the feature map of the second scale output by the CBAM attention mechanism module, r1 is the first learning coefficient, r2 is the second learning coefficient, s1 is the scaling coefficient of the first scale, s2 is the scaling coefficient of the second scale, t1 is the first scale, and t2 is the second scale.
[0035] In combination with the first aspect, optionally, obtaining a real-time depth map of the pantograph to be tested includes:
[0036] Acquire a 3D image of the pantograph to be tested through a 3D camera;
[0037] The depth information of the pantograph to be tested is obtained from the 3D image, and the depth information is filtered using a preset depth threshold to obtain a real-time depth map of the pantograph to be tested.
[0038] In a second aspect, the present invention provides a pantograph abnormality detection device, comprising:
[0039] Acquisition module: used to obtain the real-time depth map and standard depth map of the pantograph to be tested;
[0040] Depth alignment module: used to align the acquired real-time depth map with the standard depth map using the alignment network to obtain a real-time depth alignment map;
[0041] Detection module: used to use the pre-obtained anomaly detection model output module: used to detect the real-time depth alignment map to obtain an optical flow estimation value;
[0042] Output module: used to compare the optical flow estimation value with a preset first threshold value. When the optical flow estimation value is greater than the first preset threshold value, it is output that there is an abnormality in the pantograph to be tested. When the optical flow estimation value is less than or equal to the first preset threshold value, it is output that there is no abnormality in the pantograph to be tested.
[0043] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the pantograph abnormality detection method as described in the first aspect.
[0044] In a fourth aspect, the present invention provides a device, comprising:
[0045] A memory for storing instructions;
[0046] The processor is used to execute the instructions so that the device performs operations to implement the pantograph abnormality detection method as described in the first aspect.
[0047] Compared with the prior art, the beneficial effects achieved by the pantograph abnormality detection method, device, medium and equipment provided by the embodiments of the present invention include:
[0048] The present invention obtains a real-time depth map and a standard depth map of a pantograph to be tested; uses an alignment network to align the obtained real-time depth map with the standard depth map to obtain a real-time depth alignment map; the present invention has strong adaptability to changes in environmental conditions and can obtain image information of the pantograph to be tested in various complex environments;
[0049] The present invention uses a pre-obtained anomaly detection model to detect the real-time depth alignment map to obtain an optical flow estimation value; the optical flow estimation value is compared with a preset first threshold value, and when the optical flow estimation value is greater than the first preset threshold value, it is output that the pantograph to be tested has an abnormality; when the optical flow estimation value is less than or equal to the first preset threshold value, it is output that the pantograph to be tested does not have an abnormality; the anomaly detection model provided by the present invention can accurately identify anomalies in the pantograph image, and the model integrates the full convolution cross-scale normalization flow, can detect anomalies of different types and sizes, has the characteristics of high accuracy and low false detection rate, and effectively improves the safety of train driving;
[0050] The anomaly detection model obtained in advance by the present invention can be obtained by training with normal samples and verifying with abnormal samples. It does not rely on a large number of abnormal samples for training, has good feasibility, and can be promoted and applied. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 is a flow chart of a pantograph abnormality detection method provided in Embodiment 1 of the present invention;
[0052] Figure 2 is a structural diagram of an abnormality detection model in a pantograph abnormality detection method provided in Embodiment 1 of the present invention;
[0053] Figure 3 This is the abnormality detection effect of a pantograph abnormality detection method provided in Example 1 of the present invention. DETAILED DESCRIPTION
[0054] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0055] Embodiment 1:
[0056] like Figure 1 As shown, this embodiment provides a pantograph abnormality detection method, including:
[0057] Obtain the real-time depth map and standard depth map of the pantograph to be tested;
[0058] Using the alignment network, the acquired real-time depth map is aligned with the standard depth map to obtain a real-time depth alignment map;
[0059] Using a pre-obtained anomaly detection model to detect the real-time depth alignment image, to obtain an optical flow estimation value;
[0060] The optical flow estimation value is compared with a preset first threshold value. When the optical flow estimation value is greater than the first preset threshold value, it is output that there is an abnormality in the pantograph to be tested. When the optical flow estimation value is less than or equal to the first preset threshold value, it is output that there is no abnormality in the pantograph to be tested.
[0061] The specific steps include:
[0062] Step 1: Obtain the real-time depth map and standard depth map of the pantograph to be tested.
[0063] Step 1.1: Obtain a 3D image of the pantograph to be tested through a 3D camera.
[0064] Step 1.2: Obtain the depth information of the pantograph to be tested from the 3D image, and filter the depth information using a preset depth threshold to obtain a real-time depth map of the pantograph to be tested.
[0065] This embodiment uses a 3D camera to obtain a 3D image of the pantograph, including width, height, and depth, while a 2D camera can only obtain 2D information of an object, that is, a plane image.
[0066] Since 3D cameras generally have higher accuracy and resolution, this embodiment is able to capture more details and information.
[0067] Since 3D cameras are usually not very sensitive to changes in ambient light, they can achieve good imaging results under different lighting conditions. However, the imaging effect of 2D cameras is greatly affected by ambient light, and changes in light may cause the image quality to deteriorate.
[0068] 3D cameras have depth perception capabilities and can measure the distance between objects and the camera.
[0069] This embodiment uses a 3D camera, which has strong adaptability to changes in environmental conditions and can accurately obtain depth information of the pantograph in various complex environments.
[0070] Step 2: Use the alignment network to align the acquired real-time depth map with the standard depth map to obtain a real-time depth alignment map.
[0071] Step 2.1: Perform geometric transformation on the acquired real-time depth map to obtain images with different posture relationships.
[0072] Step 2.2: Input the images of different poses and the standard depth map into the alignment network to extract feature points and descriptors respectively.
[0073] Step 2.3: Based on the extracted feature points and descriptors, perform feature matching on the images of different poses and the standard depth map to obtain the matching relationship between the feature points, form feature point pairs, and calculate the matching degree of the feature point pairs.
[0074] Step 2.4: Use the preset second threshold to select feature point pairs with high matching degree, extract the overlapping area through affine transformation, use the overlapping area as the depth alignment map, and obtain the real-time depth alignment map.
[0075] Step 3: Use the pre-obtained anomaly detection model to detect the real-time depth alignment map to obtain an optical flow estimation value.
[0076] The pre-obtained anomaly detection model is obtained through the following steps:
[0077] Step 3.1: Obtain the detection results, historical depth map and corresponding standard depth map of the pantograph that has been detected.
[0078] The pantograph data with abnormal detection results are annotated using the labelme annotation tool to generate a json file, and then the annotated json file is batch converted into a mask image.
[0079] Step 3.2: Use the alignment network to align the acquired historical depth map with the corresponding standard depth map to obtain a historical depth alignment map.
[0080] In this embodiment, the same steps as step 2 are adopted to obtain the historical depth alignment map.
[0081] Step 3.3: Based on the historical depth alignment graph of pantographs with normal detection results, use the optimizer to optimize the parameters of the pre-built anomaly detection model; use the loss function to evaluate the optimized trained anomaly detection model to obtain the trained anomaly detection model.
[0082] Among them, the pre-built anomaly detection models are as follows Figure 2 As shown, including:
[0083] Feature extraction module: used to extract features from the input image using a set of convolutional layers and output a first feature map;
[0084] CBAM attention mechanism module: used to further extract the features of the key area from the first feature map using the channel attention mechanism and the spatial attention mechanism, and output the second feature map;
[0085] The first channel processing module is used to perform channel rearrangement and channel cutting on the second feature map, and output a third feature map;
[0086] Cross-scale processing: used to use multiple sets of convolutional layers to process features of different scales on the third feature map, capture motions of different scales, and output initial optical flow estimates of different scales;
[0087] The second channel processing module is used to perform channel splicing on the initial optical flow estimation of different scales and output the optical flow estimation value.
[0088] Specifically, the loss function is to add the numerical losses of the initial optical flow estimation at different scales and divide it by the number of initial optical flow estimation elements. Gradient clipping is used to keep the gradient stable, where the maximum gradient norm is 1.
[0089] The loss function is calculated by formula (1):
[0090]
[0091] In formula (1), L(y) is the loss function of the output optical flow estimation value y, p is the true value of the optical flow estimation value of the pantograph with a normal detection result, Y(y) is the predicted value of the optical flow estimation value of the pantograph with a normal detection result, z is the vector obtained by combining the initial optical flow estimations of different scales, For bias.
[0092] In this embodiment, the Adam optimizer is used, the learning rate is 0.002, the weight decay is 0.00001, and betas = (0.5, 0.9).
[0093] Specifically, cross-scale processing is expressed by formula (2):
[0094]
[0095]
[0096] In formula (2), y out,1 is the initial optical flow estimate of the first scale, y in,1 is the first scale feature map output by the CBAM attention mechanism module, y out,2 is the initial optical flow estimate of the second scale, y in,2 It is the feature map of the second scale output by the CBAM attention mechanism module, r1 is the first learning coefficient, r2 is the second learning coefficient, s1 is the scaling coefficient of the first scale, s2 is the scaling coefficient of the second scale, t1 is the first scale, and t2 is the second scale.
[0097] In this embodiment, the first learning coefficient r1 and the second learning coefficient r2 are initialized to 0, so that the transformation is an identity transformation, and
[0098] In this embodiment, the first scale t1 represents a smaller time scale or space scale, and the second scale t2 represents a larger time scale or space scale.
[0099] Step 3.4: Based on the historical depth alignment map of the pantograph with abnormal detection results, the trained anomaly detection model is input to detect whether the anomaly detection model has been trained, and in response to the training being completed, the pre-obtained anomaly detection model is obtained.
[0100] The anomaly detection model obtained in advance in this embodiment integrates the full convolution cross-scale normalization flow, can detect anomalies of different types and sizes, has the characteristics of high accuracy and low false detection rate, and effectively improves the safety of train driving. Using deep learning technology, the three-dimensional shape features of the pantograph can be automatically extracted to accurately classify abnormal situations.
[0101] The anomaly detection model obtained in advance in this embodiment can be obtained by training with normal samples and verifying with abnormal samples. It does not rely on a large number of abnormal samples for training, has good feasibility, and can be promoted and applied.
[0102] Step 4: Compare the optical flow estimation value with a preset first threshold value. When the optical flow estimation value is greater than the first preset threshold value, output that the pantograph to be tested has an abnormality. When the optical flow estimation value is less than or equal to the first preset threshold value, output that the pantograph to be tested has no abnormality.
[0103] If the pre-obtained anomaly detection model detects Figure 3 If there is an abnormality shown, it is output that there is an abnormality in the pantograph to be tested, which can timely discover the pantograph fault, improve the efficiency and accuracy of pantograph abnormality detection, and reduce the cost and error of manual detection.
[0104] At the same time, the detection results are used to further train or test the pre-obtained anomaly detection model to improve the accuracy of the anomaly detection model and reduce the false detection rate.
[0105] Embodiment 2:
[0106] An embodiment of the present invention provides a pantograph abnormality detection device, comprising:
[0107] Acquisition module: used to obtain the real-time depth map and standard depth map of the pantograph to be tested;
[0108] Depth alignment module: used to align the acquired real-time depth map with the standard depth map using the alignment network to obtain a real-time depth alignment map;
[0109] Detection module: used to use the pre-obtained anomaly detection model output module: used to detect the real-time depth alignment map to obtain an optical flow estimation value;
[0110] Output module: used to compare the optical flow estimation value with a preset first threshold value. When the optical flow estimation value is greater than the first preset threshold value, it is output that there is an abnormality in the pantograph to be tested. When the optical flow estimation value is less than or equal to the first preset threshold value, it is output that there is no abnormality in the pantograph to be tested.
[0111] Embodiment three:
[0112] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the pantograph abnormality detection method as described in the first embodiment is implemented.
[0113] Embodiment 4:
[0114] An embodiment of the present invention further provides a device, including:
[0115] A memory for storing instructions;
[0116] The processor is used to execute the instruction so that the device performs the operation of implementing the pantograph abnormality detection method as described in Example 1.
[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0118] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0119] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0121] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A pantograph abnormality detection method, characterized in that: include: Obtain the real-time depth map and standard depth map of the pantograph to be tested; Using the alignment network, the acquired real-time depth map is aligned with the standard depth map to obtain a real-time depth alignment map; The real-time depth alignment map is detected using a pre-obtained anomaly detection model to obtain an optical flow estimation value; wherein the pre-constructed anomaly detection model includes: Feature extraction module: used to extract features from the input image using a set of convolutional layers and output a first feature map; CBAM attention mechanism module: used to further extract the features of the key area from the first feature map using the channel attention mechanism and the spatial attention mechanism, and output the second feature map; The first channel processing module is used to perform channel rearrangement and channel cutting on the second feature map, and output a third feature map; Cross-scale processing: used to use multiple sets of convolutional layers to process features of different scales on the third feature map, capture motions of different scales, and output initial optical flow estimates of different scales; The second channel processing module is used to perform channel splicing on the initial optical flow estimation of different scales and output the optical flow estimation value; The optical flow estimation value is compared with a preset first threshold value. When the optical flow estimation value is greater than the first preset threshold value, it is output that there is an abnormality in the pantograph to be tested. When the optical flow estimation value is less than or equal to the first preset threshold value, it is output that there is no abnormality in the pantograph to be tested.
2. The pantograph abnormality detection method according to claim 1, characterized in that: The pre-obtained anomaly detection model is obtained by the following steps: Obtain the inspection results, historical depth map and corresponding standard depth map of the pantograph that has been inspected; Using the alignment network, the acquired historical depth map is aligned with the corresponding standard depth map to obtain a historical depth alignment map; Based on the historical deep alignment graph of pantographs with normal detection results, the optimizer is used to optimize the parameters of the pre-built anomaly detection model; The trained anomaly detection model is evaluated and optimized using the loss function to obtain the trained anomaly detection model; Based on the historical depth alignment graph of pantographs with abnormal detection results, the trained anomaly detection model is input to detect whether the training of the anomaly detection model is completed, and in response to the completion of the training, the pre-obtained anomaly detection model is obtained.
3. The pantograph abnormality detection method according to any one of claims 1 or 2, characterized in that: Using the alignment network, the acquired depth map is aligned with the standard depth map, including: Perform geometric transformation on the acquired depth map to obtain images with different posture relationships; Input the pictures of different poses and the standard depth map into the alignment network to extract feature points and descriptors respectively; Based on the extracted feature points and descriptors, the images of different positions are matched with the standard depth map to obtain the matching relationship between the feature points, form feature point pairs, and calculate the matching degree of the feature point pairs; A preset second threshold is used to select feature point pairs with a high matching degree, and an overlapping area is extracted through affine transformation, and the overlapping area is used as a depth alignment map.
4. The pantograph abnormality detection method according to claim 2, characterized in that: The loss function is to add the numerical losses of the initial optical flow estimation at different scales and divide it by the number of initial optical flow estimation elements, and use gradient clipping to keep the gradient stable, where the maximum gradient norm is 1. The loss function is calculated by formula (1): In formula (1), L(y) is the loss function of the output optical flow estimation value y, p is the true value of the optical flow estimation value of the pantograph with a normal detection result, Y(y) is the predicted value of the optical flow estimation value of the pantograph with a normal detection result, z is the vector obtained by combining the initial optical flow estimations of different scales, For bias.
5. The pantograph abnormality detection method according to claim 1, characterized in that: The cross-scale processing is expressed by formula (2): In formula (2), y out,1 is the initial optical flow estimate of the first scale, y in,1 is the first scale feature map output by the CBAM attention mechanism module, y out,2 is the initial optical flow estimate of the second scale, y in,2 It is the feature map of the second scale output by the CBAM attention mechanism module, r1 is the first learning coefficient, r2 is the second learning coefficient, s1 is the scaling coefficient of the first scale, s2 is the scaling coefficient of the second scale, t1 is the first scale, and t2 is the second scale.
6. The pantograph abnormality detection method according to claim 1, characterized in that: The step of obtaining a real-time depth map of the pantograph to be tested comprises: Acquire a 3D image of the pantograph to be tested through a 3D camera; The depth information of the pantograph to be tested is obtained from the 3D image, and the depth information is filtered using a preset depth threshold to obtain a real-time depth map of the pantograph to be tested.
7. A pantograph abnormality detection device, characterized in that: include: Acquisition module: used to obtain the real-time depth map and standard depth map of the pantograph to be tested; Depth alignment module: used to align the acquired real-time depth map with the standard depth map using the alignment network to obtain a real-time depth alignment map; The detection module is used to use the pre-obtained anomaly detection model output module: used to detect the real-time depth alignment map to obtain the optical flow estimation value; wherein the pre-constructed anomaly detection model includes: Feature extraction module: used to extract features from the input image using a set of convolutional layers and output a first feature map; CBAM attention mechanism module: used to further extract the features of the key area from the first feature map using the channel attention mechanism and the spatial attention mechanism, and output the second feature map; The first channel processing module is used to perform channel rearrangement and channel cutting on the second feature map, and output a third feature map; Cross-scale processing: used to use multiple sets of convolutional layers to process features of different scales on the third feature map, capture motions of different scales, and output initial optical flow estimates of different scales; The second channel processing module is used to perform channel splicing on the initial optical flow estimation of different scales and output the optical flow estimation value; Output module: used to compare the optical flow estimation value with a preset first threshold value. When the optical flow estimation value is greater than the first preset threshold value, it is output that there is an abnormality in the pantograph to be tested. When the optical flow estimation value is less than or equal to the first preset threshold value, it is output that there is no abnormality in the pantograph to be tested.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the pantograph abnormality detection method as described in any one of claims 1 to 6 is implemented.
9. A pantograph abnormality detection device, characterized in that: include: A memory for storing instructions; A processor is used to execute the instruction so that the device performs the operation of implementing the pantograph abnormality detection method as described in any one of claims 1-6.
Citation Information
Patent Citations
Video analysis based abnormal behavior detection method and system
CN104123544A