Image processing method and device, train and equipment
Through the combination of target image segmentation network and pixel area determination rules, the problem of poor image quality in pantograph abnormal detection is solved, the detection accuracy and train operation safety are improved, and efficient identification of pantograph abnormalities is achieved.
Patent Information
- Application Number
- CN202411173620.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-15
AI Technical Summary
The image quality used in the prior art for pantograph abnormality detection is poor, resulting in low accuracy of detection results and the inability to effectively ensure the safety of train operation.
An image processing method is adopted to segment the pantograph running image through the target image segmentation network, and based on pixel area determination rules and connected area analysis, determine whether the initial segmented image is suitable for abnormal detection, and improve the image quality of the target segmented image .
It improves the accuracy of pantograph abnormal detection, enhances the safety of train operation, and enriches the data set by simulating various working conditions and fault states, and improves the accuracy and reliability of detection.
Smart Images

Figure CN118967650A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and more specifically, to an image processing method, an image processing device, a train and an electronic device for pantograph abnormality detection. Background Art
[0002] In recent years, high-speed trains have developed rapidly. As an important means of transportation, they occupy an important position in daily travel. To ensure driving safety, it is necessary to accurately monitor the operating status of rail vehicles. Among them, the pantograph is a key component of the power of high-speed trains. Once a failure occurs, it will have a serious impact on the safe operation of the railway system. The EMU is equipped with a pantograph video monitoring system to monitor the operating status of the pantograph in real time.
[0003] In the process of implementing the concept of the present disclosure, it was found that the image quality used for pantograph anomaly detection in the related art was poor, resulting in low accuracy of subsequent detection results. Summary of the invention
[0004] In view of this, the present disclosure provides an image processing method, an image processing device, a train, an electronic device, a computer-readable storage medium and a computer program product for pantograph anomaly detection.
[0005] One aspect of the present disclosure provides an image processing method for pantograph abnormality detection, comprising: In response to the image processing instruction, acquiring a pantograph operation image of the pantograph in an abnormal working state, wherein the abnormal working state includes an abnormal environmental state and / or an abnormal pantograph state; The pantograph operation image is segmented by using a target image segmentation network to obtain an initial segmented image; Perform pixel determination on the initial segmented image based on a pixel area determination rule to obtain a determination result; When the determination result indicates that the initial segmented image satisfies the pixel area determination rule, a connected region analysis process is performed on the initial segmented image to obtain an analysis result; When the analysis result indicates that the connected regions of the initial segmented image meet a preset region quantity threshold, the initial segmented image is determined as a target segmented image for anomaly detection.
[0006] According to an embodiment of the present disclosure, obtaining a pantograph operation image when the pantograph is in an abnormal working state includes: In the case where the abnormal working state is the abnormal environmental state, applying an abnormal working condition environment at the pantograph by using an environmental simulation device; A first image of the pantograph operating under the abnormal working condition is collected, and the first image is used as the pantograph operation image.
[0007] According to an embodiment of the present disclosure, an abnormal working condition is applied to the pantograph by using an environmental simulation device, including: Using a spraying device to spray the pantograph to simulate a rainy day environment; and / or Using a smoke device to spray the pantograph to simulate a haze environment; and / or Using a flow field simulation device to blow air to the pantograph to simulate a storm environment; and / or A background simulation device is used to simulate the working background environment of the pantograph.
[0008] According to an embodiment of the present disclosure, obtaining a pantograph operation image when the pantograph is in an abnormal working state includes: In the case where the abnormal working state is the abnormal state of the pantograph, adjusting the target component of the pantograph to obtain an abnormal pantograph; A second image when the abnormal pantograph is working is collected, and the second image is used as the pantograph operation image.
[0009] According to an embodiment of the present disclosure, the target component of the pantograph is adjusted to obtain an abnormal pantograph, including: Replace or disassemble the first target component to obtain the abnormal pantograph, wherein the first target component includes at least one of a bow angle and a bow head; and / or The working state of the second target component is adjusted to obtain the above-mentioned abnormal pantograph, wherein the above-mentioned second target component includes a pantograph lifting system.
[0010] According to an embodiment of the present disclosure, the target image segmentation network is trained in the following manner: Acquire an image training set, wherein the image training set includes a plurality of pantograph operation sample images and a label image corresponding to each of the pantograph operation sample images, and the label image includes a pixel identifier of the pantograph sample; Input the pantograph operation sample image into an initial image segmentation network and output a fused image, wherein the initial image segmentation network is generated based on a visual self-supervised model and a feature pyramid network; Input the fused image and the label image corresponding to the fused image into the target loss function, and output the loss result; The network parameters of the initial image segmentation network are iteratively adjusted according to the loss results to obtain the trained target image segmentation network.
[0011] According to an embodiment of the present disclosure, the pantograph operation sample image is input into an initial image segmentation network, and a fused image is output, including: Processing the pantograph operation sample image using the visual self-supervision model to obtain a first image feature; Processing the first image feature using a first deconvolution layer to obtain an enlarged second image feature; Processing the first image feature using a second deconvolution layer to obtain an enlarged third image feature; Processing the first image feature using an identity mapping network to obtain a fourth image feature; Processing the first image feature using a pooling layer to obtain a reduced fifth image feature, wherein the feature pyramid network includes the first deconvolution layer, the second deconvolution layer, the identity mapping network and the pooling layer; The second image feature, the third image feature, the fourth image feature and the fifth image feature are subjected to feature fusion processing to obtain the fused image, wherein each pixel of the pantograph sample is marked in the fused image.
[0012] According to an embodiment of the present disclosure, pixel determination is performed on the initial segmented image based on a pixel area determination rule to obtain a determination result, including: Calculating the number of pixels of the pantograph in the initial segmented image; When the number of pixels satisfies the first number interval, a first determination result is generated, wherein the first determination result indicates that the initial segmented image satisfies the pixel area determination rule; When the number of pixels does not satisfy the first number interval, a second determination result is generated, wherein the determination result includes the first determination result or the second determination result.
[0013] According to an embodiment of the present disclosure, the initial segmented image includes a bow head area, a support rod area and a base area, and the preset area quantity threshold includes a second quantity interval, a third quantity interval and a fourth quantity interval.
[0014] According to an embodiment of the present disclosure, the connected region analysis is performed on the initial segmented image to obtain an analysis result, including: When the number of components in the bow head region satisfies the second number interval, generating a first analysis result; When the number of components in the support rod region satisfies the third number interval, generating a second analysis result; When the number of components in the base area satisfies the fourth number interval, generating a third analysis result; The analysis result is generated based on the first analysis result, the second analysis result and the third analysis result.
[0015] Another aspect of the present disclosure provides an image processing device for pantograph abnormality detection, comprising: An acquisition module, configured to acquire, in response to an image processing instruction, a pantograph operation image of the pantograph in an abnormal working state, wherein the abnormal working state includes an abnormal environmental state and / or an abnormal pantograph state; A segmentation module, used for performing segmentation processing on the pantograph operation image by using a target image segmentation network to obtain an initial segmented image; A determination module, used for performing pixel determination on the initial segmented image based on a pixel area determination rule to obtain a determination result; An analysis module, configured to perform a connected region analysis process on the initial segmented image to obtain an analysis result when the determination result indicates that the initial segmented image satisfies the pixel area determination rule; The determination module is used to determine the initial segmented image as a target segmented image for anomaly detection when the analysis result indicates that the connected areas of the initial segmented image meet a preset area quantity threshold.
[0016] Another aspect of the present disclosure provides a train for pantograph abnormality detection, comprising the image processing device as described above.
[0017] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.
[0018] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.
[0019] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and when the instructions are executed, are used to implement the method as described above.
[0020] According to an embodiment of the present disclosure, when the pantograph is in an abnormal working state, the pantograph operation image in this state is input into the target image segmentation network for segmentation to obtain an initial segmented image, and the pixel area of the initial segmented image is determined based on the pixel area determination rule. After the determination is passed, the number of connected areas in the initial segmented image is analyzed again, thereby determining the initial segmented image as a target segmented image for abnormality detection after the connected area determination. Since the present disclosure determines whether the initial segmented image is suitable for detecting pantograph abnormalities by combining pixel area and connected areas, the image quality of the target segmented image for abnormality detection is improved, thereby improving the recognition accuracy of pantograph abnormalities, and further improving the operation safety of the train. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which: Figure 1 An exemplary system architecture to which an image processing method for pantograph abnormality detection according to an embodiment of the present disclosure can be applied is schematically shown; Figure 2 A flowchart of an image processing method for pantograph abnormality detection according to an embodiment of the present disclosure is schematically shown; Figure 3 The training flow chart of the target image segmentation network according to the embodiment of the present disclosure is schematically shown; Figure 4 A block diagram schematically shows an image processing device for pantograph abnormality detection according to an embodiment of the present disclosure; Figure 5 A block diagram schematically shows an image processing device for pantograph abnormality detection according to an embodiment of the present disclosure; Figure 6 A block diagram schematically illustrates a train according to an embodiment of the present disclosure; and Figure 7 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0022] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0023] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0024] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0025] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0026] The application of computer vision technology covers the process from "image acquisition--data processing--application analysis". Different types of acquired images (static and dynamic) lead to different difficulties in application analysis.
[0027] At present, computer vision technology is widely used in industrial manufacturing, medical imaging and other fields, such as component defect detection, obstacle target detection, etc. The image types collected by the above applications are static collection with little environmental interference, so the application effect is better.
[0028] High-speed EMUs are in high-speed motion, and the types of images collected are dynamic. The operating conditions are complex (including different weather conditions, different lines, etc.), which leads to complex and dynamic backgrounds in the surveillance video, and interference from direct sunlight, dirt, raindrops adhering to the lens, etc. In addition, there is little real fault data, and it is urgent to design a train pantograph fault and complex scene simulation test system, carry out pantograph fault simulation tests, and enrich the data set.
[0029] In view of this, the embodiments of the present disclosure provide an image processing method, an apparatus, a train, a device, a storage medium and a program product, which can be applied to the field of image processing technology. The method comprises: in response to an image processing instruction, obtaining a pantograph operation image of a pantograph in an abnormal working state, wherein the abnormal working state includes an abnormal environmental state and / or an abnormal pantograph state; performing segmentation processing on the pantograph operation image using a target image segmentation network to obtain an initial segmented image; performing pixel determination on the initial segmented image based on a pixel area determination rule to obtain a determination result; when the determination result indicates that the initial segmented image satisfies the pixel area determination rule, performing connected area analysis processing on the initial segmented image to obtain an analysis result; when the analysis result indicates that the connected area of the initial segmented image satisfies a preset area number threshold, determining the initial segmented image as a target segmented image for abnormality detection.
[0030] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security, and national security.
[0031] Figure 1 The exemplary system architecture 100 to which the image processing method for pantograph abnormality detection according to an embodiment of the present disclosure can be applied is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.
[0032] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, a server 105, a camera 106, and a train 107. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.
[0033] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).
[0034] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
[0035] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.
[0036] The train 107 may be any train equipped with a pantograph, such as a high-speed train, etc. The camera 106 is installed on the train in front of the pantograph to capture images of the pantograph.
[0037] It should be noted that the image processing method for pantograph abnormality detection provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the image processing device for pantograph abnormality detection provided in the embodiment of the present disclosure can generally be set in the server 105. The image processing method for pantograph abnormality detection provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the image processing device for pantograph abnormality detection provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the image processing method for pantograph abnormality detection provided in the embodiment of the present disclosure can also be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or can also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Correspondingly, the image processing device for pantograph abnormality detection provided in the embodiment of the present disclosure can also be set in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.
[0038] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0039] Figure 2 A flowchart of an image processing method for pantograph abnormality detection according to an embodiment of the present disclosure is schematically shown.
[0040] like Figure 2 As shown, the image processing method for pantograph abnormality detection includes operations S201~S205.
[0041] In operation S201, in response to an image processing instruction, a pantograph operation image of the pantograph in an abnormal working state is acquired, wherein the abnormal working state includes an abnormal environment state and / or an abnormal state of the pantograph; In operation S202, the pantograph operation image is segmented using a target image segmentation network to obtain an initial segmented image; In operation S203, pixel determination is performed on the initial segmented image based on a pixel area determination rule to obtain a determination result; In operation S204, when the determination result indicates that the initial segmented image satisfies the pixel area determination rule, a connected region analysis process is performed on the initial segmented image to obtain an analysis result; In operation S205 , when the analysis result indicates that the connected regions of the initial segmented image meet a preset region quantity threshold, the initial segmented image is determined as a target segmented image for abnormality detection.
[0042] According to an embodiment of the present disclosure, the image processing instruction may be generated by a staff member inputting a corresponding operation on an electronic device such as a computer, and the electronic device responding to the operation.
[0043] According to an embodiment of the present disclosure, an abnormal environment state may refer to an abnormal working environment such as strong wind, heavy rain, heavy fog, or the presence of an obstruction. An abnormal pantograph state may refer to a state in which the pantograph is damaged or broken. The preset area quantity threshold may be specifically set according to actual needs, based on the number of different components of the pantograph.
[0044] According to an embodiment of the present disclosure, the pixel area determination rule may refer to comparing the pixel area of the pantograph image in the image with the pixel number interval. If the pixel area of the pantograph image is within the pixel number interval, it is determined that the initial segmented image satisfies the pixel area determination rule.
[0045] According to an embodiment of the present disclosure, after the staff issues an image processing instruction, the pantograph operation image corresponding to the instruction is obtained from the database, and the image is input into the trained target image segmentation network for segmentation processing to obtain an initial segmented image, which includes the pantograph image.
[0046] According to an embodiment of the present disclosure, the pixel area of the pantograph image in the initial segmented image is counted, and the pixel area is judged based on the pixel area judgment rule to obtain a judgment result. If the judgment result shows that the pixel area of the pantograph satisfies the pixel area judgment rule, the connected area analysis of the pantograph image is continued to obtain an analysis result. Among them, the connected area refers to the pantograph image being divided into several independent image areas that are not connected to each other. In the case where the above analysis result shows that the connected area of the initial segmented image meets the preset area number threshold, the initial segmented image is determined as the target segmented image, so that the target segmented image can be used to detect whether there is an abnormality in the pantograph in the image.
[0047] According to an embodiment of the present disclosure, when the pantograph is in an abnormal working state, the pantograph operation image in this state is input into the target image segmentation network for segmentation to obtain an initial segmented image, and the pixel area of the initial segmented image is determined based on the pixel area determination rule. After the determination is passed, the number of connected areas in the initial segmented image is analyzed again, thereby determining the initial segmented image as a target segmented image for abnormality detection after the connected area determination. Since the present disclosure determines whether the initial segmented image is suitable for detecting pantograph abnormalities by combining pixel area and connected areas, the image quality of the target segmented image for abnormality detection is improved, thereby improving the recognition accuracy of pantograph abnormalities, and further improving the operation safety of the train.
[0048] According to an embodiment of the present disclosure, obtaining a pantograph operation image when the pantograph is in an abnormal working state includes: In the case where the abnormal working state is an abnormal environmental state, an abnormal working condition environment is applied at the pantograph using an environmental simulation device; A first image of the pantograph operating under an abnormal working condition is collected, and the first image is used as the pantograph operation image.
[0049] According to the embodiments of the present disclosure, in order to simulate the pantograph being in various complex working conditions when the train is running, such as abnormal working environments such as strong winds, heavy rain, heavy fog, and the presence of obstructions, the present disclosure can use an environmental simulation device to simulate the above-mentioned abnormal working conditions, thereby using the pantograph camera to capture the first image of the pantograph under the current working conditions for subsequent image processing and abnormality detection, which can improve the accuracy of the pantograph simulation and thereby improve the safety of the train operation.
[0050] According to an embodiment of the present disclosure, an abnormal working condition is applied to a pantograph by using an environmental simulation device, including: Use a spraying device to spray the pantograph to simulate a rainy day environment; and / or Use a smoke device to spray the pantograph to simulate a haze environment; and / or Using a flow simulation device to blow air over the pantograph to simulate a storm environment; and / or The background simulation device is used to simulate the working background environment of the pantograph.
[0051] According to the embodiments of the present disclosure, in order to simulate the flow field state during the operation of the train, a flow field simulation device can be arranged in front of the pantograph to blow air to the pantograph, thereby simulating the wind field environment under the high-speed operation condition of the train; in order to simulate weather scenes such as rain and fog, a spray and smoke device can be used to spray and mist in the pantograph area to realize the simulation of rain and fog conditions.
[0052] According to an embodiment of the present disclosure, the pantograph background simulation during train operation is realized by a background simulation device. The background sources include videos of different scenes (tunnels, station entry, etc.), different weather conditions (rain, snow, sunny, cloudy, foggy, etc.), and different time periods (day, night), which need to be manually recorded in advance. During the test, the background simulation device needs to be placed behind the pantograph so that the pantograph camera in front of the pantograph can collect images, wherein the background simulation device can be a display screen, and different background videos are played on the display screen to simulate the working background environment, wherein the playback speed of the background video is related to the speed of the train during the simulation.
[0053] According to an embodiment of the present disclosure, obtaining a pantograph operation image when the pantograph is in an abnormal working state includes: When the abnormal working state is an abnormal state of the pantograph, adjusting and processing the target component of the pantograph to obtain an abnormal pantograph; A second image is collected when the abnormal pantograph is working, and the second image is used as the pantograph operation image.
[0054] According to an embodiment of the present disclosure, the abnormal state of the pantograph may refer to a failure or damage of the pantograph. For example, the components of the pantograph are replaced or disassembled to obtain an abnormal pantograph. At this time, a second image of the abnormal pantograph is collected when the train is running, and image processing is performed on the second image.
[0055] According to the embodiments of the present disclosure, by adjusting the target components of the pantograph, the abnormality of the pantograph can be simulated, thereby comprehensively and comprehensively simulating the working state of the pantograph under different working conditions, which is beneficial to improving the safety of the train during operation.
[0056] According to an embodiment of the present disclosure, the target component of the pantograph is adjusted to obtain an abnormal pantograph, including: Replacing or disassembling the first target component to obtain an abnormal pantograph, wherein the first target component includes at least one of a bow angle and a bow head; and / or The working state of the second target component is adjusted to obtain an abnormal pantograph, wherein the second target component includes a pantograph lifting system.
[0057] According to an embodiment of the present disclosure, the bow horn is disassembled to simulate a bow horn missing state, and the original bow head is replaced with a broken bow head to simulate bow head damage, thereby obtaining an abnormal pantograph.
[0058] According to the embodiments of the present disclosure, in order to restore the raising and lowering state of the pantograph during the operation of the train, the working state of the pantograph raising and lowering system can be designed. The raising and lowering of the pantograph is controlled by a balancing system composed of air bags. The pressurized air of the air bags is provided by the pneumatic control unit (air compressor). Under the action of the pressurized air, the air bags extend to generate torque, which acts on the hinge of the lower arm through the cam and the elastic connecting shaft, thereby raising the pantograph. The height adjustment of the pantograph is achieved by controlling the switch by the compressor. If the supply of pressurized air is interrupted, the pantograph will automatically lower. The lowering of the pantograph is automatically achieved by gravity after the pressurized air in the air bag is emptied.
[0059] According to an embodiment of the present disclosure, the height of the pantograph is controlled by controlling the working state of the pantograph lifting system to simulate a situation where the pantograph cannot contact the power grid.
[0060] Figure 3 The training flowchart of the target image segmentation network according to an embodiment of the present disclosure is schematically shown.
[0061] like Figure 3 As shown, the training method of the target image segmentation network includes operations S301 to S304.
[0062] In operation S301, an image training set is obtained, wherein the image training set includes a plurality of pantograph operation sample images and a label image corresponding to each pantograph operation sample image, and the label image includes a pixel identifier of the pantograph sample; In operation S302, a pantograph operation sample image is input into an initial image segmentation network, and a fused image is output, wherein the initial image segmentation network is generated according to a visual self-supervisory model and a feature pyramid network; In operation S303, the fused image and the label image corresponding to the fused image are input into the target loss function, and the loss result is output; In operation S304, the network parameters of the initial image segmentation network are iteratively adjusted according to the loss result to obtain a trained target image segmentation network.
[0063] According to an embodiment of the present disclosure, the visual self-supervision model may be a Dinov2 model, the feature pyramid network may be a FPN (Feature Pyramid Networks) network, and the target loss function may be any type of binary classification loss function.
[0064] According to an embodiment of the present disclosure, a plurality of pantograph operation sample images and label images are obtained, and the pixels of the pantograph are marked on the label image. After the pantograph operation sample images are input into the initial image segmentation network, a fused image can be obtained, and the pantograph on the fused image is also marked. The fused image and the corresponding label image are input into the target loss function to obtain a loss result, and the network parameters of the initial image segmentation network are iteratively adjusted based on the loss result to obtain a trained target image segmentation network.
[0065] According to the embodiments of the present disclosure, a threshold of the number of iterations (e.g., 50 times) can be set, and after the number of iterations reaches the threshold of the number of iterations, the last initial image segmentation network is determined as the target image segmentation network. Or if the loss results of multiple consecutive times (e.g., 5 times) fluctuate within a preset fluctuation range (e.g., 0.01), the last initial image segmentation network can be determined as the target image segmentation network.
[0066] According to an embodiment of the present disclosure, a pantograph operation sample image is input into an initial image segmentation network, and a fused image is output, including: Processing the pantograph operation sample image using the visual self-supervision model to obtain the first image feature; Processing the first image feature using the first deconvolution layer to obtain an enlarged second image feature; Processing the first image feature using the second deconvolution layer to obtain an enlarged third image feature; Processing the first image feature using an identity mapping network to obtain a fourth image feature; Processing the first image feature using a pooling layer to obtain a reduced fifth image feature, wherein the feature pyramid network includes a first deconvolution layer, a second deconvolution layer, an identity mapping network, and a pooling layer; The second image feature, the third image feature, the fourth image feature and the fifth image feature are subjected to feature fusion processing to obtain a fused image, wherein each pixel of the pantograph sample is marked in the fused image.
[0067] According to an embodiment of the present disclosure, both the first deconvolution layer and the second deconvolution layer may be constructed according to ConvTranspose2d. The first deconvolution layer may include two layers of ConvTranspose2d, and the second deconvolution layer may include one layer of ConvTranspose2d.
[0068] According to an embodiment of the present disclosure, a visual self-supervised model is used to process a pantograph operation sample image to extract a strong image feature, namely, a first image feature. Thereafter, the first image feature is processed using a first deconvolution layer to enlarge the image size, namely, upsampling processing, thereby obtaining a second image feature, wherein the size of the second image feature is 4 times that of the first image feature.
[0069] According to an embodiment of the present disclosure, the first image feature is processed by the second deconvolution layer to obtain an enlarged third image feature, and the size of the third image feature is twice that of the first image feature. The first image feature is processed by the identity mapping network to obtain a fourth image feature, and the size of the fourth image feature is the same as that of the first image feature. The first image feature is processed by the pooling layer to obtain a reduced fifth image feature, and the size of the fifth image feature is 1 / 2 of the first image feature.
[0070] According to an embodiment of the present disclosure, the second image feature, the third image feature, the fourth image feature and the fifth image feature are subjected to feature fusion processing to obtain a fused image, wherein the feature fusion method may be to simultaneously perform upsampling, interpolation, splicing and fusion convolution processing on all image features. The pixel value of each pixel in the fused image is 0 or 1, 1 represents that the pixel is a pixel of the pantograph, and 0 represents that the pixel is not a pantograph, which is a background pixel.
[0071] According to an embodiment of the present disclosure, pixel determination is performed on the initial segmented image based on a pixel area determination rule to obtain a determination result, including: Count the number of pixels of the pantograph in the initial segmented image; When the number of pixels satisfies the first number interval, a first determination result is generated, wherein the first determination result indicates that the initial segmented image satisfies the pixel area determination rule; When the number of pixels does not satisfy the first number interval, a second determination result is generated, wherein the determination result includes the first determination result or the second determination result.
[0072] According to an embodiment of the present disclosure, the first quantity interval is greater than the minimum pixel threshold Z and less than X+δ, wherein the number of pixels of the pantograph in the picture is X, and due to the influence of dynamic changes, the pixel dynamic change range is set to δ.
[0073] According to an embodiment of the present disclosure, when the number of pixels of the pantograph in the initial segmented image satisfies a first number interval, it can be judged that the initial segmented image satisfies the pixel area determination rule, thereby generating a first determination result; if the number of pixels does not satisfy the first number interval, a second determination result is generated.
[0074] According to an embodiment of the present disclosure, by determining the number of pixels of the pantograph in the initial segmented image, the image quality of the initial segmented image can be initially screened, thereby improving the image quality used for abnormality detection and avoiding inaccurate abnormality detection due to poor image quality, thereby improving the operation safety of the train.
[0075] According to an embodiment of the present disclosure, the initial segmented image includes a bow head area, a support rod area and a base area, and the preset area quantity threshold includes a second quantity interval, a third quantity interval and a fourth quantity interval.
[0076] According to an embodiment of the present disclosure, connected region analysis is performed on the initial segmented image to obtain analysis results, including: When the number of components in the bow head region satisfies a second number interval, generating a first analysis result; When the number of components in the support rod region satisfies a third number interval, generating a second analysis result; When the number of components in the base area satisfies a fourth number interval, generating a third analysis result; An analysis result is generated according to the first analysis result, the second analysis result, and the third analysis result.
[0077] According to an embodiment of the present disclosure, the second number interval may be, wherein is the number of components in a normal bow head region, the bow head region including two bow angles and a bow head component, and is usually 4.
[0078] According to an embodiment of the present disclosure, the third quantity interval may be, wherein is the number of components in a normal support rod area, the support rod area including an upper support rod and a lower support rod, and is typically 2.
[0079] According to an embodiment of the present disclosure, the fourth quantity interval may be, wherein is the number of components in a normal base area, the base area including supporting insulators, lightning arresters, semi-rigid terminals, etc., and is typically 5.
[0080] According to an embodiment of the present disclosure, it is determined whether the number of components in the bow head region, the support rod region, and the base region meets the corresponding number intervals, thereby obtaining corresponding analysis results.
[0081] According to the embodiments of the present disclosure, by analyzing and judging the connected areas of the number of components in different areas of the pantograph, the image quality of the pantograph used for abnormality detection can be further improved in combination with the connected area judgment based on the initial screening of pixel area, thereby further improving the safety of train operation.
[0082] Figure 4 A block diagram of an image processing device for pantograph abnormality detection according to an embodiment of the present disclosure is schematically shown.
[0083] like Figure 4 As shown, the image processing device 400 for pantograph abnormality detection includes an acquisition module 410 , a segmentation module 420 , a determination module 430 , an analysis module 440 , and a determination module 450 .
[0084] An acquisition module 410 is used to acquire a pantograph operation image of the pantograph in an abnormal working state in response to an image processing instruction, wherein the abnormal working state includes an abnormal environmental state and / or an abnormal state of the pantograph; The segmentation module 420 is used to segment the pantograph operation image using the target image segmentation network to obtain an initial segmented image; A determination module 430 is used to perform pixel determination on the initial segmented image based on a pixel area determination rule to obtain a determination result; An analysis module 440 is used to perform a connected region analysis process on the initial segmented image to obtain an analysis result when the determination result indicates that the initial segmented image satisfies the pixel area determination rule; The determination module 450 is used to determine the initial segmented image as a target segmented image for anomaly detection when the analysis result indicates that the connected regions of the initial segmented image meet a preset region quantity threshold.
[0085] According to an embodiment of the present disclosure, when the pantograph is in an abnormal working state, the pantograph operation image in this state is input into the target image segmentation network for segmentation to obtain an initial segmented image, and the pixel area of the initial segmented image is determined based on the pixel area determination rule. After the determination is passed, the number of connected areas in the initial segmented image is analyzed again, thereby determining the initial segmented image as a target segmented image for abnormality detection after the connected area determination. Since the present disclosure determines whether the initial segmented image is suitable for detecting pantograph abnormalities by combining pixel area and connected areas, the image quality of the target segmented image for abnormality detection is improved, thereby improving the recognition accuracy of pantograph abnormalities, and further improving the operation safety of the train.
[0086] According to an embodiment of the present disclosure, the acquisition module 410 includes an application unit and a first acquisition unit.
[0087] An applying unit, used for applying an abnormal working condition environment at the pantograph by using an environmental simulation device when the abnormal working state is an abnormal environmental state; The first acquisition unit is used to acquire a first image of the pantograph operating under abnormal working conditions, and use the first image as the pantograph operation image.
[0088] According to an embodiment of the present disclosure, the applying unit includes at least one of a spraying subunit, a misting subunit, a blowing subunit, and a simulation subunit.
[0089] The spray sub-unit is used to spray the pantograph using a spray device to simulate a rainy day environment.
[0090] The spray sub-unit is used to spray the pantograph using a smoke device to simulate a haze environment.
[0091] The blowing sub-unit is used to blow air to the pantograph using a flow field simulation device to simulate a storm environment.
[0092] The simulation subunit is used to simulate the working background environment of the pantograph using a background simulation device.
[0093] According to an embodiment of the present disclosure, the acquisition module 410 includes an adjustment unit and a second subunit.
[0094] An adjustment unit, used for adjusting a target component of the pantograph to obtain an abnormal pantograph when the abnormal working state is an abnormal state of the pantograph; The second subunit is used to collect a second image when the abnormal pantograph is working, and use the second image as the pantograph operation image.
[0095] According to an embodiment of the present disclosure, the adjustment unit includes a component adjustment subunit and a state adjustment subunit.
[0096] A component adjustment subunit is used to replace or disassemble a first target component to obtain an abnormal pantograph, wherein the first target component includes at least one of a bow angle and a bow head; and / or The state adjustment subunit is used to adjust the working state of the second target component to obtain an abnormal pantograph, wherein the second target component includes a pantograph lifting system.
[0097] Figure 5 A block diagram of an image processing device for pantograph abnormality detection according to an embodiment of the present disclosure is schematically shown.
[0098] like Figure 5 As shown, the training device 500 of the target image segmentation network includes a second acquisition module 510, a processing module 520, a calculation module 530, and an adjustment module 540.
[0099] A second acquisition module 510 is used to acquire an image training set, wherein the image training set includes a plurality of pantograph operation sample images and a label image corresponding to each pantograph operation sample image, and the label image includes a pixel identifier of the pantograph sample; A processing module 520 is used to input the pantograph operation sample image into an initial image segmentation network and output a fused image, wherein the initial image segmentation network is generated according to a visual self-supervisory model and a feature pyramid network; A calculation module 530, used to input the fused image and the label image corresponding to the fused image into the target loss function, and output the loss result; The adjustment module 540 is used to iteratively adjust the network parameters of the initial image segmentation network according to the loss result to obtain a trained target image segmentation network.
[0100] According to an embodiment of the present disclosure, the processing module 520 includes a processing unit, a first convolution unit, a second convolution unit, a mapping unit, a pooling unit, and a fusion unit.
[0101] A processing unit, configured to process a pantograph operation sample image using a visual self-supervision model to obtain a first image feature; A first convolution unit, used for processing the first image feature by using a first deconvolution layer to obtain an amplified second image feature; A second convolution unit, used for processing the first image feature by using a second deconvolution layer to obtain an amplified third image feature; A mapping unit, configured to process the first image feature using an identity mapping network to obtain a fourth image feature; A pooling unit, used for processing the first image feature by using a pooling layer to obtain a reduced fifth image feature, wherein the feature pyramid network includes a first deconvolution layer, a second deconvolution layer, an identity mapping network and a pooling layer; The fusion unit is used to perform feature fusion processing on the second image feature, the third image feature, the fourth image feature and the fifth image feature to obtain a fused image, wherein each pixel of the pantograph sample is marked in the fused image.
[0102] According to an embodiment of the present disclosure, the determination module 430 includes a calculation unit, a first generation unit, and a second generation unit.
[0103] a calculation unit, for calculating the number of pixels of the pantograph in the initial segmented image; A first generating unit is used to generate a first determination result when the number of pixels meets a first number interval, wherein the first determination result indicates that the initial segmented image meets a pixel area determination rule; The second generating unit is used to generate a second determination result when the number of pixels does not satisfy the first number interval, wherein the determination result includes the first determination result or the second determination result.
[0104] According to an embodiment of the present disclosure, the initial segmented image includes a bow head area, a support rod area and a base area, and the preset area quantity threshold includes a second quantity interval, a third quantity interval and a fourth quantity interval.
[0105] According to an embodiment of the present disclosure, the analysis module 440 includes a third generation unit, a fourth generation unit, a fifth generation unit, and a sixth generation unit.
[0106] A third generating unit, configured to generate a first analysis result when the number of components in the bow head region satisfies a second number interval; A fourth generating unit, configured to generate a second analysis result when the number of components in the support rod region satisfies a third number interval; A fifth generating unit, configured to generate a third analysis result when the number of components in the base area satisfies a fourth number interval; The sixth generating unit is used to generate an analysis result according to the first analysis result, the second analysis result and the third analysis result.
[0107] According to the embodiments of the present invention, any one or more of the modules, units, and sub-units, or at least part of the functions of any one of them, can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, units, and sub-units can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, units, and sub-units can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, units, and sub-units can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.
[0108] For example, any multiple of the acquisition module 410, the segmentation module 420, the determination module 430, the analysis module 440, and the determination module 450 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the acquisition module 410, the segmentation module 420, the determination module 430, the analysis module 440, and the determination module 450 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the acquisition module 410 , the segmentation module 420 , the determination module 430 , the analysis module 440 , and the determination module 450 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.
[0109] It should be noted that the image processing device part in the embodiment of the present disclosure corresponds to the image processing method part in the embodiment of the present disclosure. The description of the image processing device part specifically refers to the image processing method part, which will not be repeated here.
[0110] Figure 6 A block diagram of a train according to an embodiment of the present disclosure is schematically shown.
[0111] like Figure 6 As shown, a train 600 for pantograph abnormality detection includes: The pantograph camera 601 is placed in front of the pantograph and is used to collect the pantograph operation image in real time during the test; The video monitoring server 602 is used to store the pantograph operation images; The intelligent analysis host 603 is equipped with the above-mentioned image processing device, which is used to generate a target segmentation image according to the pantograph operation image, and perform pantograph abnormality detection according to the target segmentation image to obtain a detection result; The monitoring screen 604 is used to display the target segmentation image and the detection results.
[0112] According to an embodiment of the present disclosure, when the pantograph is in an abnormal working state, the pantograph operation image in this state is input into the target image segmentation network for segmentation to obtain an initial segmented image, and the pixel area of the initial segmented image is determined based on the pixel area determination rule. After the determination is passed, the number of connected areas in the initial segmented image is analyzed again, thereby determining the initial segmented image as a target segmented image for abnormality detection after the connected area determination. Since the present disclosure determines whether the initial segmented image is suitable for detecting pantograph abnormalities by combining pixel area and connected areas, the image quality of the target segmented image for abnormality detection is improved, thereby improving the recognition accuracy of pantograph abnormalities, and further improving the operation safety of the train.
[0113] Figure 7 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0114] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.
[0115] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.
[0116] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.
[0117] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.
[0118] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.
[0119] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.
[0120] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703 .
[0121] The embodiments of the present disclosure also include a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiments of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiments of the present disclosure.
[0122] When the computer program is executed by the processor 701, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0123] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0124] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).
[0125] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.
[0126] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.
Claims
1. An image processing method for pantograph anomaly detection, include: In response to the image processing instruction, acquiring a pantograph operation image of the pantograph in an abnormal working state, wherein the abnormal working state includes an abnormal environmental state and / or an abnormal state of the pantograph; Using a target image segmentation network to segment the pantograph operation image to obtain an initial segmented image; Performing pixel determination on the initial segmented image based on a pixel area determination rule to obtain a determination result; When the determination result indicates that the initial segmented image satisfies the pixel area determination rule, performing a connected region analysis process on the initial segmented image to obtain an analysis result; When the analysis result indicates that the connected regions of the initial segmented image meet a preset region quantity threshold, the initial segmented image is determined as a target segmented image for anomaly detection.
2. The method according to claim 1, in, Obtain pantograph operation images when the pantograph is in abnormal working state, including: In the case where the abnormal working state is the abnormal environmental state, applying an abnormal working condition environment at the pantograph by using an environmental simulation device; A first image of the pantograph operating under the abnormal working condition is collected, and the first image is used as the pantograph operation image.
3. The method according to claim 2, in, Applying an abnormal working condition environment at the pantograph by using an environmental simulation device includes: Using a spraying device to spray the pantograph to simulate a rainy day environment; and / or Using a smoke device to spray the pantograph to simulate a haze environment; and / or Using a flow field simulation device to blow air to the pantograph to simulate a storm environment; and / or A background simulation device is used to simulate the working background environment of the pantograph.
4. The method according to claim 1, in, Obtain pantograph operation images when the pantograph is in abnormal working state, including: When the abnormal working state is an abnormal state of the pantograph, adjusting the target component of the pantograph to obtain an abnormal pantograph; A second image when the abnormal pantograph is working is collected, and the second image is used as the pantograph operation image.
5. The method according to claim 4, in, The target component of the pantograph is adjusted to obtain an abnormal pantograph, including: Replacing or disassembling a first target component to obtain the abnormal pantograph, wherein the first target component includes at least one of a bow angle and a bow head; and / or The working state of the second target component is adjusted to obtain the abnormal pantograph, wherein the second target component includes a pantograph lifting system.
6. The method according to claim 1, in, The target image segmentation network is trained in the following way: Acquire an image training set, wherein the image training set includes a plurality of pantograph operation sample images and a label image corresponding to each of the pantograph operation sample images, and the label image includes a pixel identifier of the pantograph sample; Inputting the pantograph operation sample image into an initial image segmentation network and outputting a fused image, wherein the initial image segmentation network is generated according to a visual self-supervised model and a feature pyramid network; Inputting the fused image and the label image corresponding to the fused image into the target loss function, and outputting the loss result; The network parameters of the initial image segmentation network are iteratively adjusted according to the loss result to obtain the trained target image segmentation network.
7. The method according to claim 6, in, The pantograph operation sample image is input into the initial image segmentation network, and a fused image is output, including: Processing the pantograph operation sample image using the visual self-supervision model to obtain a first image feature; Processing the first image feature using a first deconvolution layer to obtain an enlarged second image feature; Processing the first image feature using a second deconvolution layer to obtain an enlarged third image feature; Processing the first image feature using an identity mapping network to obtain a fourth image feature; Processing the first image feature using a pooling layer to obtain a reduced fifth image feature, wherein the feature pyramid network includes the first deconvolution layer, the second deconvolution layer, the identity mapping network and the pooling layer; The second image feature, the third image feature, the fourth image feature and the fifth image feature are subjected to feature fusion processing to obtain the fused image, wherein each pixel of the pantograph sample is marked in the fused image.
8. The method according to claim 1, in, Performing pixel determination on the initial segmented image based on a pixel area determination rule to obtain a determination result includes: Calculating the number of pixels of the pantograph in the initial segmented image; When the number of pixels satisfies the first number interval, generating a first determination result, wherein the first determination result indicates that the initial segmented image satisfies the pixel area determination rule; When the number of pixels does not satisfy the first number interval, a second determination result is generated, wherein the determination result includes the first determination result or the second determination result.
9. The method according to claim 1, in, The initial segmented image includes a bow head area, a support rod area and a base area, and the preset area quantity threshold includes a second quantity interval, a third quantity interval and a fourth quantity interval; The connected region analysis is performed on the initial segmented image to obtain the analysis result, including: When the number of components in the bow head region satisfies the second number interval, generating a first analysis result; When the number of components in the support rod region satisfies the third number interval, generating a second analysis result; When the number of components in the base area satisfies the fourth number interval, generating a third analysis result; The analysis result is generated according to the first analysis result, the second analysis result and the third analysis result.
10. An image processing device for pantograph anomaly detection, include: An acquisition module, configured to acquire, in response to an image processing instruction, a pantograph operation image of the pantograph in an abnormal working state, wherein the abnormal working state includes an abnormal environmental state and / or an abnormal pantograph state; A segmentation module, used for performing segmentation processing on the pantograph operation image by using a target image segmentation network to obtain an initial segmented image; A determination module, used for performing pixel determination on the initial segmented image based on a pixel area determination rule to obtain a determination result; An analysis module, configured to perform a connected region analysis process on the initial segmented image to obtain an analysis result when the determination result indicates that the initial segmented image satisfies the pixel area determination rule; The determination module is used to determine the initial segmented image as a target segmented image for anomaly detection when the analysis result indicates that the connected areas of the initial segmented image meet a preset area quantity threshold.
11. A train for pantograph anomaly detection, comprising the image processing device according to claim 10.
12. An electronic device, include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.
