Pantograph state detection method and device thereof

Abstract

The present disclosure provides a pantograph state detection method and device, which can be applied to the technical fields of artificial intelligence and image recognition detection. The pantograph state detection method comprises: identifying position information of a bow head region based on a video stream of a pantograph region; determining motion state information of the pantograph based on the position information and a preset motion rule; generating a start instruction in a case where the motion state information represents an end of bow raising, wherein the start instruction is used to start an edge detection model, and the edge detection model is used to determine displacement information of a contact point and deflection angle information of a carbon slide plate from the start of bow raising to the end of bow raising; and generating a termination instruction in a case where the state information represents a start of bow lowering, wherein the termination instruction is used to terminate the edge detection model.

Description

Technical Field

[0001] This disclosure relates to the fields of rail transit and artificial intelligence technology, specifically to the field of image recognition and detection technology, and more specifically to a method and apparatus for pantograph status detection. Background Technology

[0002] Rail transit trains (with the exception of certain new energy trains) can obtain power from the outside through the sliding contact between the train's pantograph and the overhead contact line. The contact relationship between the pantograph and the overhead contact line, as well as the raising and lowering status of the pantograph head, are crucial to the safety and normal operation of the train. Due to the special contact relationship and structural characteristics of the pantograph and the overhead contact line, it is not convenient to install testing equipment on the pantograph or overhead contact line of the operating train to detect whether the contact relationship between the pantograph and the overhead contact line is normal in real time. Moreover, once a serious abnormal contact occurs between the pantograph and the overhead contact line in the operating train, it is not convenient to climb onto the roof of the train for inspection on the line section. It is necessary to use special attachment equipment to climb onto the roof for inspection, or replace the pantograph to maintain operation, which directly affects the punctuality of the train. Summary of the Invention

[0003] In view of the above problems, this disclosure provides methods, apparatus, equipment, media and procedures for pantograph condition detection.

[0004] According to a first aspect of this disclosure, a pantograph state detection method is provided, comprising: identifying position information of the pantograph head region based on a video stream targeting the pantograph region; determining motion state information of the pantograph based on the position information and preset motion rules; generating a start command when the motion state information indicates the end of pantograph raising, wherein the start command is used to start an edge detection model, the edge detection model being used to determine the displacement information of the contact point and the deflection angle information of the carbon slide plate from the start to the end of pantograph raising; and generating a termination command when the state information indicates the start of pantograph lowering, wherein the termination command is used to terminate the edge detection model.

[0005] According to embodiments of this disclosure, the pantograph status detection method further includes: generating a termination command when it is determined that the motion state information indicates an abnormality in pantograph raising.

[0006] According to embodiments of this disclosure, the video stream includes multiple frames of images, and the position information includes the position information of the bow head region corresponding to each frame of image. Preset motion rules are used to characterize the motion rules for raising and lowering the pantograph. Based on the position information and the preset motion rules, the motion state information of the pantograph is determined, including: determining a movement threshold for the bow head region corresponding to each frame of image according to the preset motion rules; repeatedly performing the following operations for multiple frames of images: determining position change information based on the position information of the bow head region corresponding to a first preset frame image and the position information of the bow head region corresponding to a second preset frame image, wherein the first preset frame image and the second preset frame image are different frames and both belong to the video stream; and determining a change threshold based on the first preset frame image, the second preset frame image, and the movement threshold; and determining the motion state information of the pantograph based on the position change information and the change threshold.

[0007] According to embodiments of this disclosure, the position change information includes position rise information; based on the position change information and a change threshold, the motion state information of the pantograph is determined, including: when it is determined that the position rise information obtained after traversing multiple frames of images all exceeds the change threshold, the motion state information is determined to represent the start of pantograph raising; when pantograph raising is determined to start, a first change state information of the position rise information is determined after a first preset time; when it is determined that the first change state information represents that the position rise information remains unchanged, the motion state information is determined to represent the end of pantograph raising.

[0008] According to embodiments of this disclosure, the position change information further includes position descent information; based on the position change information and the change threshold, the motion state information of the pantograph is determined, including: when it is determined that the position descent information obtained after traversing multiple frames of images all exceeds the change threshold, the motion state information is determined to indicate that pantograph descent has started; when pantograph descent has started, a second change state information of the position descent information is determined after a second preset time; when it is determined that the second change state information indicates that the position descent information remains unchanged, the motion state information is determined to indicate that pantograph descent has ended.

[0009] According to embodiments of this disclosure, identifying the location information of the bow head region based on a video stream targeting the pantograph region includes: inputting the video stream into a bow head recognition model and outputting the location information of the bow head region, wherein the bow head recognition model is obtained by pre-training a lightweight neural network based on a deep learning framework based on different bow head samples.

[0010] According to embodiments of this disclosure, the video stream includes multiple frames of images, and the position information includes the position information of the bow head region corresponding to each frame of images; the pantograph state detection method further includes: when it is determined that the edge detection model is activated, inputting the images corresponding to the start and end of pantograph raising into the edge detection model to perform the following operations: identifying all straight lines in the image and marking all straight lines; identifying the contact line from all straight lines according to a first constraint condition between the pantograph and the contact wire; determining the contact point based on the contact line and the bow head region; determining the displacement information of the contact point based on the contact point and the position information; identifying the straight line parallel to the carbon sliding plate from all straight lines according to a second constraint condition between the pantograph and the carbon sliding plate; determining the deflection angle information of the carbon sliding plate based on the straight line parallel to the carbon sliding plate and the position information; and outputting the displacement information of the contact point and the deflection angle information of the carbon sliding plate.

[0011] According to embodiments of this disclosure, the contact point includes a first contact point and a second contact point. The first contact point is used to characterize the contact point in the image corresponding to the start of bow raising, and the second contact point is used to characterize the contact point in the image corresponding to the end of bow raising. Based on the contact point and position information, determining the displacement information of the contact point includes: determining first position information based on the position information of the bow head region corresponding to the first contact point and the image corresponding to the start of bow raising; determining second position information based on the position information of the second contact point and the image corresponding to the end of bow raising; and determining the displacement information of the contact point based on the first position information and the second position information.

[0012] According to embodiments of this disclosure, the straight line includes a first straight line and a second straight line. The first straight line represents a straight line parallel to the carbon skateboard in the image corresponding to the start of the bow lift, and the second straight line represents a straight line parallel to the carbon skateboard in the image corresponding to the end of the bow lift. Based on the straight line parallel to the carbon skateboard and position information, the deflection angle information of the carbon skateboard is determined, including: determining third position information based on the position information of the bow head region corresponding to the first straight line and the image corresponding to the start of the bow lift; determining fourth position information based on the position information of the bow head region corresponding to the second straight line and the image corresponding to the end of the bow lift; and determining the deflection angle information of the carbon skateboard based on the third and fourth position information.

[0013] A second aspect of this disclosure provides a pantograph state detection device, comprising: an identification module for identifying position information of the pantograph head region based on a video stream targeting the pantograph region; a determination module for determining motion state information of the pantograph based on the position information and preset motion rules; a start module for generating a start command when the determined motion state information indicates the end of pantograph raising, wherein the start command is used to start an edge detection model, the edge detection model is used to determine the displacement information of the contact point and the deflection angle information of the carbon slide plate from the start to the end of pantograph raising; and a first termination module for generating a termination command when the determined state information indicates the start of pantograph lowering, wherein the termination command is used to terminate the edge detection model.

[0014] A third aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the pantograph state detection method described above.

[0015] A fourth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the above-described pantograph status detection method.

[0016] The fifth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described pantograph status detection method.

[0017] According to embodiments of this disclosure, by identifying the positional information of the pantograph head area, the movement of the pantograph head is sensed. By judging the movement of the pantograph head, the pantograph raising and lowering status is detected. Based on the detection results, the edge detection model is linked. When the pantograph raising ends, the edge detection model can be activated in real time to detect contact point displacement and the deflection angle of the carbon sliding plate. This is beneficial for assessing anomalies in the pantograph raising and lowering status, deflection angle, and contact point displacement, and issuing anomaly warnings, providing a reference for the safe operation of the train. When the pantograph lowering begins, the edge detection model is terminated in real time, which can reduce the ineffective consumption of storage and computing resources of the edge detection model. Based on this, the pantograph status detection method provided by embodiments of this disclosure can solve problems such as missed detections, false detections, excessive maintenance, and low efficiency in pantograph detection. Attached Figure Description

[0018] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1The illustration schematically depicts an application scenario of the pantograph status detection method, apparatus, device, medium, and program product according to embodiments of the present disclosure;

[0020] Figure 2 A flowchart illustrating a pantograph status detection method according to an embodiment of the present disclosure is shown schematically.

[0021] Figure 3 This illustration schematically shows a diagram of an image recognition result based on a bow-head recognition model according to an embodiment of the present disclosure;

[0022] Figure 4 A schematic diagram of inter-frame motion intervals according to embodiments of the present disclosure is shown.

[0023] Figure 5 A schematic diagram of a contact line, contact point, and carbon slide plate according to an embodiment of the present disclosure is shown.

[0024] Figure 6 A flowchart illustrating a pantograph status detection method according to another embodiment of the present disclosure is shown schematically;

[0025] Figure 7 This schematic diagram illustrates the functional framework of the pantograph lifting and lowering status monitoring system of a vehicle-mounted PHM pantograph according to an embodiment of the present disclosure.

[0026] Figure 8 A schematic block diagram of a pantograph status detection device according to an embodiment of the present disclosure is shown; and

[0027] Figure 9 A block diagram schematically illustrates an electronic device suitable for implementing a pantograph status detection method according to an embodiment of the present disclosure. Detailed Implementation

[0028] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0029] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0030] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0031] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0032] In the technical solution disclosed herein, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse.

[0033] In implementing this disclosure, it was discovered that with the application of artificial intelligence technology frameworks in the industrial field, a series of real-time monitoring models have emerged based on the detection of pantograph operating status using video data. These real-time monitoring models consume a significant amount of onboard edge device system resources, and they are only meaningful if they can sensitively detect anomalies under the premise that the pantograph is raised and there are no mechanical faults. However, due to the constraints of the onboard environment, the image artificial intelligence hardware and chip resources deployed on the mobile onboard end of the train are relatively limited. How to improve resource utilization, construct a system architecture for onboard fault prediction and health management (PHM) models, and accurately determine whether the pantograph and its associated mechanical movements are normal are technical problems that need to be solved by those skilled in the art.

[0034] Embodiments of this disclosure provide a pantograph state detection method and apparatus. The method includes: identifying position information of the pantograph head region based on a video stream targeting the pantograph region; determining pantograph motion state information based on the position information and preset motion rules; generating a start command when the motion state information indicates the end of pantograph raising, wherein the start command is used to start an edge detection model, the edge detection model is used to determine the displacement information of the contact point and the deflection angle information of the carbon sliding plate from the start to the end of pantograph raising; and generating a termination command when the state information indicates the start of pantograph lowering, wherein the termination command is used to terminate the edge detection model.

[0035] Figure 1 The illustration schematically depicts an application scenario of the pantograph status detection method, apparatus, device, medium, and program product according to embodiments of the present disclosure.

[0036] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0037] Users can interact with server 105 via network 104 using at least one of the first terminal device 101, second terminal device 102, and third terminal device 103 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, second terminal device 102, and third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0038] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to trains, smartphones, tablets, laptops, and desktop computers.

[0039] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0040] It should be noted that the pantograph status detection method provided in this embodiment can generally be executed by server 105. Correspondingly, the pantograph status detection device provided in this embodiment can generally be located in server 105. The pantograph status detection method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the pantograph status detection device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0041] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0042] The following will be based on Figure 1 The described scene, through Figures 2-6 The pantograph status detection method of the disclosed embodiments will be described in detail.

[0043] Figure 2 A flowchart illustrating a pantograph status detection method according to an embodiment of the present disclosure is shown schematically.

[0044] like Figure 2 As shown, the pantograph status detection method 200 of this embodiment includes operations S210 to S240.

[0045] During operation S210, the position information of the pantograph head area is identified based on the video stream targeting the pantograph area.

[0046] According to embodiments of this disclosure, the location information of the bow head region can be the edge coordinate information of the bow head region.

[0047] According to embodiments of this disclosure, a video stream may consist of multiple frames of images.

[0048] According to embodiments of this disclosure, a video stream of the pantograph area can be acquired in real time, and the position information of the pantograph head area can be identified from each frame of the video stream.

[0049] For example, for trains, a PHM onboard host can be used to non-contactly acquire video streams of the pantograph area. The acquired video stream of the pantograph area can be input into a recognition model, and after the pantograph area is identified, the edge coordinate information of the pantograph head area is output.

[0050] During operation S220, the pantograph's motion status information is determined based on position information and preset motion rules.

[0051] According to embodiments of this disclosure, the location information may be the location information of the bow head region corresponding to each frame of image.

[0052] According to embodiments of this disclosure, preset motion rules can be used to characterize the motion rules of the pantograph raising and lowering. For example, these rules could be the speed of the pantograph raising and lowering and the duration of the raising and lowering of the pantograph.

[0053] According to embodiments of this disclosure, the pantograph's motion state information can be used to characterize the pantograph's raising and lowering state. The pantograph's raising and lowering state can include at least one of the following: raising begins, raising ends, raising in progress, raising abnormally, lowering begins, lowering ends, lowering in progress, lowering abnormally, etc.

[0054] According to embodiments of this disclosure, displacement change information between two adjacent frames can be determined based on the position information of the bow head region corresponding to each frame of image; motion displacement information of the pantograph raising and lowering can be determined based on preset motion rules; and motion state information of the pantograph can be determined based on displacement change information and motion displacement information.

[0055] In operation S230, when the motion state information indicates the end of the bow lift, a start command is generated. The start command is used to start the edge detection model, which is used to determine the displacement information of the contact point and the deflection angle information of the carbon slide plate from the start to the end of the bow lift.

[0056] According to embodiments of this disclosure, the pantograph raising / lowering state can be determined based on the pantograph movement state information determined in operation S220. If the pantograph raising / lowering is determined to be complete, an edge detection model is activated. The contact point is the intersection of the carbon sliding plate and the contact wire.

[0057] In operation S240, if the state information indicates that the bow descent has started, a termination instruction is generated, wherein the termination instruction is used to terminate the edge detection model.

[0058] According to embodiments of this disclosure, the pantograph raising / lowering state can be determined based on the pantograph motion state information determined in operation S220. If pantograph lowering is determined to have begun, the edge detection model is terminated.

[0059] According to embodiments of this disclosure, by identifying the positional information of the pantograph head area, the movement of the pantograph head is sensed. By judging the movement of the pantograph head, the pantograph raising and lowering status is detected. Based on the detection results, the edge detection model is linked. When the pantograph raising ends, the edge detection model can be activated in real time to detect contact point displacement and the deflection angle of the carbon sliding plate. This is beneficial for assessing anomalies in the pantograph raising and lowering status, deflection angle, and contact point displacement, and issuing anomaly warnings, providing a reference for the safe operation of the train. When the pantograph lowering begins, the edge detection model is terminated in real time, which can reduce the ineffective consumption of storage and computing resources of the edge detection model. Based on this, the pantograph status detection method provided by embodiments of this disclosure can solve problems such as missed detections, false detections, excessive maintenance, and low efficiency in pantograph detection.

[0060] It should be noted that the pantograph status detection method provided in this disclosure can be applied not only to trains, but also to Tangshan trains, Changchun Railway Vehicles, etc.

[0061] According to embodiments of this disclosure, the pantograph status detection method may further include: generating a termination command when it is determined that motion state information indicates an abnormality in pantograph raising.

[0062] According to embodiments of this disclosure, the pantograph raising / lowering state can be determined based on the pantograph motion state information determined in operation S220. If an abnormal pantograph raising / lowering is determined, the edge detection model is terminated.

[0063] For example, based on the time corresponding to each frame of the image, the pantograph raising time can be determined. By judging whether the pantograph raising time conforms to preset motion rules, it can be determined whether the pantograph raising is abnormal. Based on offline video data, the pantograph raising and lowering times of sample pantographs can be statistically analyzed, and the time threshold ranges for the raising and lowering processes can be determined based on these sample pantograph raising and lowering times. If the pantograph raising time is within the time threshold range, the pantograph raising is considered normal. If the pantograph raising time is outside the time threshold range, the pantograph raising is considered abnormal.

[0064] According to the embodiments of this disclosure, in the event of an abnormal pantograph raising, the edge detection model is terminated in real time, which can reduce the ineffective consumption of storage, computing resources, etc. of the edge detection model.

[0065] Figure 3 The illustration shows a schematic diagram of the image recognition result based on the bow head recognition model according to an embodiment of the present disclosure.

[0066] According to embodiments of this disclosure, identifying the location information of the bow head region based on a video stream targeting the pantograph region may include: inputting the video stream into a bow head recognition model and outputting the location information of the bow head region, wherein the bow head recognition model is obtained by pre-training a lightweight neural network based on a deep learning framework based on different bow head samples.

[0067] According to embodiments of this disclosure, a lightweight neural network that improves the object detection framework can be constructed based on the deep learning framework (PyTorch).

[0068] According to embodiments of this disclosure, the pre-training method may include acquiring bow head samples with different routes, operating conditions, and backgrounds as training samples; labeling each bow head sample with a location information label of the smallest region of the bow head; inputting the training samples into a lightweight neural network and outputting predicted location information; and adjusting the parameters of the lightweight neural network based on the location information labels and the predicted location information to obtain a bow head recognition model.

[0069] According to embodiments of this disclosure, the bow-head recognition model can be converted into an in-vehicle engine using an inference framework (TensorRT) to meet the requirements of real-time in-vehicle operation.

[0070] According to embodiments of this disclosure, after the vehicle-mounted PHM host receives the video stream, it can transmit each frame of the video stream to the bow head recognition model and output the position information of the bow head region for the smallest region of the bow head.

[0071] For example, a camera can be mounted and fixed in the center of the carriage roof, maintaining a certain distance from the pantograph, so that the camera's field of view covers the entire pantograph's movement range during train operation. The camera is then adjusted to face the pantograph directly. Using the onboard network, the video stream is transmitted to the video processing board of the onboard PHM host. After receiving the video stream, the PHM host processes the image and sends it to a lightweight pantograph head recognition model based on PyTorch deep learning, outputting the coordinates of the pantograph head area to complete the localization of the smallest pantograph head.

[0072] like Figure 3 As shown, the identified bow-head region can be a rectangular region, and the position information of the bow-head region can be the coordinate information of the diagonal vertex of the edge of the rectangular region, such as (x1, y1) and (x2, y2).

[0073] According to embodiments of this disclosure, a lightweight model built based on deep learning can accurately identify and locate the coordinates of the smallest area of ​​the pantograph head in real time.

[0074] According to the embodiments of this disclosure, inter-frame filtering can also be performed on the video stream of the pantograph area to accurately identify the position information of all pantograph head areas in the video stream. This can avoid abnormal situations caused by interference from exposure, lighting, etc., in the pantograph head area coordinates, reduce missed detections and false detections, and improve the background anti-interference capability of the recognition model.

[0075] Figure 4 A schematic diagram of inter-frame motion intervals according to an embodiment of the present disclosure is shown.

[0076] According to embodiments of this disclosure, the video stream includes multiple frames of images, and the position information includes the position information of the bow head region corresponding to each frame of image. Preset motion rules are used to characterize the motion rules for raising and lowering the pantograph. Based on the position information and the preset motion rules, the motion state information of the pantograph is determined, including: determining a movement threshold for the bow head region corresponding to each frame of image according to the preset motion rules; repeatedly performing the following operations for multiple frames of images: determining position change information based on the position information of the bow head region corresponding to a first preset frame image and the position information of the bow head region corresponding to a second preset frame image, wherein the first preset frame image and the second preset frame image are different frames and both belong to the video stream; and determining a change threshold based on the first preset frame image, the second preset frame image, and the movement threshold; and determining the motion state information of the pantograph based on the position change information and the change threshold.

[0077] According to embodiments of this disclosure, the preset motion rules may include the inter-frame dynamic range of a normally operating pantograph, the pantograph lowering speed, the pantograph raising speed, and the pantograph raising / lowering time, etc. A movement threshold can be determined based on the inter-frame dynamic range of a normally operating pantograph, the pantograph lowering speed, the pantograph raising speed, and the pantograph raising / lowering time, etc.

[0078] According to embodiments of this disclosure, position change information can be used to characterize inter-frame movement distance information and inter-frame movement direction information.

[0079] For example, when a train is running normally, the amplitude of the pantograph's movement up, down, left, and right oscillates within a fixed range. Based on this, the dynamic range between frames can be set. The movement distance at consecutive intervals of N frames during pantograph raising and lowering can be weighted and averaged to set the inter-frame movement threshold for the pantograph.

[0080] like Figure 4 As shown, the position information of the bow head region corresponding to the first preset frame image can be (x1, y1) and (x2, y2). The position information of the bow head region corresponding to the second preset frame image can be (x11, y11) and (x22, y22). The movement distance Δy1 can be determined based on the coordinates of (x1, y1) and (x11, y11). The movement distance Δy2 can be determined based on the coordinates of (x2, y2) and (x22, y22). Based on Δy1 and Δy2, the position change information is obtained.

[0081] According to embodiments of this disclosure, the first preset frame image may be a frame image in the video stream that is earlier than the current time. In operations that are repeatedly performed on multiple frames, the first preset frame image may remain unchanged, and the second preset frame image may be any frame image in the video stream that follows the first preset frame image.

[0082] Specifically, the number of interval frames can be determined based on the first and second preset frame images; a change threshold can be obtained by multiplying the number of interval frames by a movement threshold; when the position change information represents the upward movement direction and the position change information represents the inter-frame movement distance exceeding the change threshold, the pantograph's motion status information starting from the second preset frame indicates the start of pantograph raising; traversing other frames in the video stream, when multiple position change information represents the inter-frame movement distance continuously increasing, pantograph raising is indicated; when multiple position change information represents the inter-frame movement distance remaining stable, pantograph raising ends. When the position change information represents the downward movement direction and the position change information represents the inter-frame movement distance exceeding the change threshold, the pantograph's motion status information starting from the second preset frame indicates the start of pantograph lowering; traversing other frames in the video stream, when multiple position change information represents the inter-frame movement distance continuously decreasing, pantograph lowering is indicated; when multiple position change information represents the inter-frame movement distance remaining stable, pantograph lowering ends.

[0083] According to embodiments of this disclosure, based on the pantograph's motion rules and the positional relationship of the pantograph head region, the pantograph's motion state between frames in the video stream can be accurately determined, enabling real-time perception of the pantograph's motion state.

[0084] According to embodiments of this disclosure, the position change information includes position rise information; based on the position change information and a change threshold, the motion state information of the pantograph is determined, including: when it is determined that the position rise information obtained after traversing multiple frames of images all exceeds the change threshold, the motion state information is determined to represent the start of pantograph raising; when pantograph raising is determined to start, a first change state information of the position rise information is determined after a first preset time; when it is determined that the first change state information represents that the position rise information remains unchanged, the motion state information is determined to represent the end of pantograph raising.

[0085] According to embodiments of this disclosure, positional increase information can be determined based on the positional information of the bow-head region in the first preset frame image and the positional information of the bow-head region in the second preset frame image. For example, the difference information can be determined based on the positional information of the bow-head region in the first preset frame image and the positional information of the bow-head region in the second preset frame image, and the difference information can be used as the positional increase information.

[0086] According to embodiments of this disclosure, the first preset time can be determined based on the pantograph's operating speed in actual applications. For example, if the operating speed is increased, the first preset time can be lengthened to avoid errors caused by small movement between two frames.

[0087] According to the embodiments of this disclosure, the operation process of the pantograph can be statistically analyzed to determine whether the overall pantograph raising process is within the time threshold range of the pantograph raising process, thereby further determining whether the pantograph raising is abnormal.

[0088] According to another embodiment of this disclosure, a termination command can be generated if it is determined that the pantograph raising is normal and the received train operation command is abnormal.

[0089] According to embodiments of this disclosure, the position change information further includes position descent information; based on the position change information and the change threshold, the motion state information of the pantograph is determined, including: when it is determined that the position descent information obtained after traversing multiple frames of images all exceeds the change threshold, the motion state information is determined to indicate that pantograph descent has started; when pantograph descent has started, a second change state information of the position descent information is determined after a second preset time; when it is determined that the second change state information indicates that the position descent information remains unchanged, the motion state information is determined to indicate that pantograph descent has ended.

[0090] According to embodiments of this disclosure, position descent information can be determined based on the position information of the bow-head region in the first preset frame image and the position information of the bow-head region in the second preset frame image. For example, the difference information can be determined based on the position information of the bow-head region in the first preset frame image and the position information of the bow-head region in the second preset frame image, and the difference information can be used as the position descent information.

[0091] According to embodiments of this disclosure, the second preset time can be determined based on the pantograph's operating speed in actual applications. For example, if the operating speed is increased, the second preset time can be lengthened to avoid errors caused by small movement between two frames.

[0092] According to embodiments of this disclosure, real-time sensing of the pantograph's motion status is achieved based on the pantograph's movement time and the positional relationship of the pantograph head area.

[0093] According to the embodiments of this disclosure, the operation process of the pantograph can be statistically analyzed to determine whether the overall pantograph lowering process is within the time threshold range of the lowering process, thereby further determining whether the lowering is abnormal.

[0094] Figure 5 A schematic diagram of a contact line, contact point, and carbon slide plate according to an embodiment of the present disclosure is shown.

[0095] According to embodiments of this disclosure, the video stream includes multiple frames of images, and the position information includes the position information of the bow head region corresponding to each frame of images; the pantograph state detection method further includes: when it is determined that the edge detection model is activated, inputting the images corresponding to the start and end of pantograph raising into the edge detection model to perform the following operations: identifying all straight lines in the image and marking all straight lines; identifying the contact line from all straight lines according to a first constraint condition between the pantograph and the contact wire; determining the contact point based on the contact line and the bow head region; determining the displacement information of the contact point based on the contact point and the position information; identifying the straight line parallel to the carbon sliding plate from all straight lines according to a second constraint condition between the pantograph and the carbon sliding plate; determining the deflection angle information of the carbon sliding plate based on the straight line parallel to the carbon sliding plate and the position information; and outputting the displacement information of the contact point and the deflection angle information of the carbon sliding plate.

[0096] According to embodiments of this disclosure, identifying and marking all straight lines in an image may include: performing grayscale preprocessing on the images corresponding to the start and end of the arching motion; performing dilation and erosion operations on the grayscale images respectively; subtracting the eroded image from the dilated image; inverting the image pixel by pixel after the above operations; finally applying adaptive binarization to convert the image into a binary image; and performing Hough line detection on the binary image to obtain and mark all straight lines in the image.

[0097] According to embodiments of this disclosure, based on a first constraint condition between the pantograph and the contact wire, identifying the contact wire from all straight lines includes: searching and retaining all straight lines that contact the upper edge of the image within the pantograph head region; the slope of the contact wire varies within a certain range, which can be set to [-1, 1], removing straight lines whose slopes are not within this range, and obtaining the contact wire, as shown below. Figure 5 As shown by the straight line l1.

[0098] According to embodiments of this disclosure, determining the contact point based on the contact line and the pantograph head region may include: determining the contact point by mapping the intersection of the contact line and the upper line of the pantograph head region, such as... Figure 5 As shown in the midpoint A.

[0099] According to embodiments of this disclosure, the first starting position information of the contact point at the start of bow raising and the first ending position information of the contact point at the end of bow raising can be determined based on the position information of the bow head region corresponding to each frame of image; and the displacement information of the contact point can be determined based on the first starting position information and the first ending position information.

[0100] According to embodiments of this disclosure, the carbon slide plate straight line can be obtained based on a straight line parallel to the carbon slide plate, such as... Figure 5As shown in line l2. Based on the position information of the carbon slide plate line and the corresponding bow head area in each frame image, the starting position information of the carbon slide plate at the start of bow lifting and the ending position information of the carbon slide plate at the end of bow lifting are determined; based on the starting position information and the ending position information, the deflection angle information of the carbon slide plate is determined.

[0101] According to embodiments of this disclosure, the displacement information of the contact point determined by image line detection and the deflection angle information of the carbon sliding plate are beneficial for accurately assessing the pantograph's operating status by combining the pantograph's motion state information.

[0102] According to embodiments of this disclosure, the contact point includes a first contact point and a second contact point. The first contact point is used to characterize the contact point in the image corresponding to the start of bow raising, and the second contact point is used to characterize the contact point in the image corresponding to the end of bow raising. Based on the contact point and position information, determining the displacement information of the contact point includes: determining first position information based on the position information of the bow head region corresponding to the first contact point and the image corresponding to the start of bow raising; determining second position information based on the position information of the second contact point and the image corresponding to the end of bow raising; and determining the displacement information of the contact point based on the first position information and the second position information.

[0103] According to embodiments of this disclosure, the first position information of the first contact point can be determined using the position information of the bow head region corresponding to the image at the start of bow raising. The second position information of the second contact point can be determined using the position information of the bow head region corresponding to the image at the end of bow raising. The straight-line distance between the first and second position information is determined and used as the displacement information of the contact point.

[0104] According to embodiments of this disclosure, the displacement information of the contact point can be accurately determined based on the position information of the contact point at the beginning and end of pantograph raising, which is beneficial for accurately evaluating the pantograph's operating status by combining the pantograph's motion state information.

[0105] According to embodiments of this disclosure, the straight line includes a first straight line and a second straight line. The first straight line represents a straight line parallel to the carbon skateboard in the image corresponding to the start of the bow lift, and the second straight line represents a straight line parallel to the carbon skateboard in the image corresponding to the end of the bow lift. Based on the straight line parallel to the carbon skateboard and position information, the deflection angle information of the carbon skateboard is determined, including: determining third position information based on the position information of the bow head region corresponding to the first straight line and the image corresponding to the start of the bow lift; determining fourth position information based on the position information of the bow head region corresponding to the second straight line and the image corresponding to the end of the bow lift; and determining the deflection angle information of the carbon skateboard based on the third and fourth position information.

[0106] According to embodiments of this disclosure, the third position information of the first straight line can be determined using the position information of the bow head region corresponding to the image at the start of bow lifting. The fourth position information of the second straight line can be determined using the position information of the bow head region corresponding to the image at the end of bow lifting. Based on the third and fourth position information, the included angle information of the two straight lines is determined, and this included angle information is used as the deflection angle information of the carbon skateboard.

[0107] According to embodiments of this disclosure, the deflection angle of the carbon sliding plate is accurately determined based on the position information of the carbon sliding plate at the beginning and end of pantograph raising, which is beneficial for accurately evaluating the pantograph's operating status by combining the pantograph's motion state information.

[0108] Figure 6 A flowchart illustrating a pantograph status detection method according to another embodiment of the present disclosure is shown schematically.

[0109] like Figure 6 As shown, the pantograph status detection method 600 of this embodiment may include operations S610 to S660.

[0110] When operating the S610, the video stream is parsed.

[0111] When operating the S620, the bow head recognition model is invoked.

[0112] S630 is used for pantograph status detection.

[0113] When operating S640, start the associated model.

[0114] In operation S650, terminate the associated model.

[0115] When operating the S660, save the output results.

[0116] According to an embodiment of this disclosure, in operation S610, the 25 frames / second video stream from the vehicle-mounted camera is parsed to obtain multiple frames of images. By executing operation S620, the multiple frames of images are input into the pantograph recognition model to identify the position information of the pantograph region. Based on the position information of the pantograph region and preset motion rules, operation S630 is executed to perform pantograph raising / lowering state detection, obtaining the pantograph's motion state information. If the motion state information indicates the end of pantograph raising, operation S640 is executed to start the association model. If the state information indicates the start of pantograph lowering or an abnormal pantograph raising, operation S650 is executed to terminate the association model. The association model may include an edge detection model, which is used to determine the displacement information of the contact point and the deflection angle information of the carbon sliding plate from the start to the end of pantograph raising. The displacement information of the contact point and the deflection angle information of the carbon sliding plate can be used as the output of the association model and saved by operation S660.

[0117] According to embodiments of this disclosure, the saved output results and corresponding video can be sent as early warning information to a third-party client. Upon receiving the early warning information, the third-party client can check whether the pantograph has any mechanical or mechanistic problems. The third-party client can be the client located in the mechanic's and driver's cab. The mechanic and driver can determine whether it is a false alarm based on video playback.

[0118] According to the embodiments of this disclosure, by proposing a multi-threaded / multi-process system architecture, real-time monitoring of the pantograph raising and lowering status is achieved, and the pantograph raising and lowering status detection results are linked with the associated model, thereby reducing the ineffective consumption of system and resources of the artificial intelligence board.

[0119] Figure 7 The diagram schematically illustrates the pantograph lifting status monitoring function framework of a vehicle-mounted PHM pantograph according to an embodiment of the present disclosure.

[0120] like Figure 7 As shown, this disclosure also provides a framework for monitoring the raising and lowering pantograph status of a vehicle-mounted PHM pantograph. This framework mainly includes a technical support layer, a functional layer, and an application layer. The technical support layer may include a camera, a PHM host, Linux, a computer language (Python), and PyTorch. The functional layer may include a pantograph head recognition model, capable of video input, video parsing, model encapsulation, model scheduling, model execution, and result saving. It may also include a pantograph status detection model, capable of detecting the raising and lowering pantograph status, deflection angle, contact point displacement, and result judgment. The functional layer may also include data applications, enabling status analysis, result transmission, linkage with related models, and analysis reports. The application layer can implement pantograph head anomaly result saving, pantograph head status anomaly warning, raising and lowering pantograph anomaly detection, deflection angle anomaly, and contact point anomaly detection.

[0121] Based on the above-described pantograph status detection method, this disclosure also provides a pantograph status detection device. The following will be combined with... Figure 8 The device is described in detail.

[0122] Figure 8 A schematic block diagram of a pantograph status detection device according to an embodiment of the present disclosure is shown.

[0123] like Figure 8 As shown, the pantograph status detection device 800 of this embodiment includes an identification module 810, a determination module 820, a start module 830, and a first termination module 840.

[0124] The identification module 810 is used to identify the position information of the pantograph head region based on the video stream targeting the pantograph region. In one embodiment, the identification module 810 can be used to perform the operation S210 described above, which will not be repeated here.

[0125] The determining module 820 is used to determine the pantograph's motion state information based on position information and preset motion rules. In one embodiment, the determining module 820 can be used to perform the operation S220 described above, which will not be repeated here.

[0126] The startup module 830 is used to generate a startup command when the motion state information indicates the end of the bow lift. The startup command activates the edge detection model, which determines the displacement information of the contact point and the deflection angle information of the carbon slide plate from the start to the end of the bow lift. In one embodiment, the startup module 830 can be used to execute the operation S230 described above, which will not be repeated here.

[0127] The first termination module 840 is used to generate a termination command when the state information indicates the start of bow descent, wherein the termination command is used to terminate the edge detection model. In one embodiment, the first termination module 840 can be used to perform the operation S240 described above, which will not be repeated here.

[0128] According to embodiments of this disclosure, the pantograph status detection device 800 further includes a second termination module.

[0129] The second termination module is used to generate a termination command when the motion state information indicates an abnormality in the bow lift.

[0130] According to embodiments of this disclosure, the video stream includes multiple frames of images, the position information includes the position information of the bow head region corresponding to each frame of image, and the preset motion rules are used to characterize the motion rules for raising and lowering the pantograph.

[0131] The determination module may include a threshold determination unit, a processing unit, and an information determination unit.

[0132] The threshold determination unit is used to determine the movement threshold of the bow head region corresponding to each frame of the image according to the preset motion rules.

[0133] The processing unit is used to repeatedly perform the following operations for multiple frames of images: determine position change information based on the position information of the bow head region corresponding to the first preset frame image and the position information of the bow head region corresponding to the second preset frame image, wherein the first preset frame image and the second preset frame image are different frame images and both belong to the video stream; and determine a change threshold based on the first preset frame image, the second preset frame image and the movement threshold.

[0134] The information determination unit is used to determine the pantograph's motion state information based on position change information and change threshold.

[0135] According to embodiments of this disclosure, the position change information includes position rise information; based on the position change information and a change threshold, the motion state information of the pantograph is determined, including: when it is determined that the position rise information obtained after traversing multiple frames of images all exceeds the change threshold, the motion state information is determined to represent the start of pantograph raising; when pantograph raising is determined to start, a first change state information of the position rise information is determined after a first preset time; when it is determined that the first change state information represents that the position rise information remains unchanged, the motion state information is determined to represent the end of pantograph raising.

[0136] According to embodiments of this disclosure, the position change information further includes position descent information; based on the position change information and the change threshold, the motion state information of the pantograph is determined, including: when it is determined that the position descent information obtained after traversing multiple frames of images all exceeds the change threshold, the motion state information is determined to indicate that pantograph descent has started; when pantograph descent has started, a second change state information of the position descent information is determined after a second preset time; when it is determined that the second change state information indicates that the position descent information remains unchanged, the motion state information is determined to indicate that pantograph descent has ended.

[0137] According to embodiments of this disclosure, identifying the location information of the bow head region based on a video stream targeting the pantograph region includes: inputting the video stream into a bow head recognition model and outputting the location information of the bow head region, wherein the bow head recognition model is obtained by pre-training a lightweight neural network based on a deep learning framework based on different bow head samples.

[0138] According to embodiments of this disclosure, the video stream includes multiple frames of images, and the position information includes the position information of the pantograph head region corresponding to each frame of images; the pantograph state detection device 800 may further include: an edge detection model input module, an edge detection model processing module, and an edge detection model output module.

[0139] The edge detection model input module is used to input the images corresponding to the start and end of the bow lift into the edge detection model when it is determined that the edge detection model has been activated.

[0140] The edge detection model processing module is used to identify and mark all straight lines in the image; identify the contact line from all straight lines based on the first constraint between the pantograph and the contact wire; determine the contact point based on the contact line and the pantograph head area; determine the displacement information of the contact point based on the contact point and its position information; identify the straight line parallel to the carbon sliding plate from all straight lines based on the second constraint between the pantograph and the carbon sliding plate; and determine the deflection angle information of the carbon sliding plate based on the straight line parallel to the carbon sliding plate and its position information.

[0141] The edge detection model output module is used to output the displacement information of the contact point and the deflection angle information of the carbon slide plate.

[0142] According to embodiments of this disclosure, the contact point includes a first contact point and a second contact point. The first contact point is used to characterize the contact point in the image corresponding to the start of bow raising, and the second contact point is used to characterize the contact point in the image corresponding to the end of bow raising. Based on the contact point and position information, determining the displacement information of the contact point includes: determining first position information based on the position information of the bow head region corresponding to the first contact point and the image corresponding to the start of bow raising; determining second position information based on the position information of the second contact point and the image corresponding to the end of bow raising; and determining the displacement information of the contact point based on the first position information and the second position information.

[0143] According to embodiments of this disclosure, the straight line includes a first straight line and a second straight line. The first straight line represents a straight line parallel to the carbon skateboard in the image corresponding to the start of the bow lift, and the second straight line represents a straight line parallel to the carbon skateboard in the image corresponding to the end of the bow lift. Based on the straight line parallel to the carbon skateboard and position information, the deflection angle information of the carbon skateboard is determined, including: determining third position information based on the position information of the bow head region corresponding to the first straight line and the image corresponding to the start of the bow lift; determining fourth position information based on the position information of the bow head region corresponding to the second straight line and the image corresponding to the end of the bow lift; and determining the deflection angle information of the carbon skateboard based on the third and fourth position information.

[0144] According to embodiments of this disclosure, any plurality of modules among the identification module 810, determination module 820, startup module 830, and first termination module 840 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the identification module 810, determination module 820, startup module 830, and first termination module 840 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these methods. Alternatively, at least one of the identification module 810, determination module 820, startup module 830 and first termination module 840 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0145] Figure 9 A block diagram schematically illustrates an electronic device suitable for implementing a pantograph status detection method according to an embodiment of the present disclosure.

[0146] like Figure 9 As shown, an electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0147] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 902 and / or RAM 903. It should be noted that programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0148] According to embodiments of this disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0149] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0150] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903 described above.

[0151] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the item recommendation method provided in the embodiments of this disclosure.

[0152] When the computer program is executed by the processor 901, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0153] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 909, and / or installed from a removable medium 911. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0154] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0155] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0156] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0157] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0158] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for detecting the status of a pantograph, comprising: Based on the video stream targeting the pantograph area, identify the location information of the pantograph head area; Based on the location information and preset motion rules, the motion state information of the pantograph is determined; Upon determining that the motion state information indicates the end of the bow lift, a start command is generated. This start command activates an edge detection model, which determines the displacement information of the contact point and the deflection angle information of the carbon slide plate from the start to the end of the bow lift. If the state information indicates the start of bow descent, a termination instruction is generated, wherein the termination instruction is used to terminate the edge detection model.

2. The method according to claim 1, further comprising: If the motion state information indicates an abnormality in the bow lift, the termination command is generated.

3. The method according to claim 1, wherein, The video stream includes multiple frames of images, the position information includes the position information of the bow head area corresponding to each frame of image, and the preset motion rules are used to characterize the motion rules of the pantograph raising and lowering. The process of determining the pantograph's motion state information based on the location information and preset motion rules includes: Based on the preset motion rules, the movement threshold of the bow head region corresponding to each frame image is determined; The following operations are repeatedly performed on the multiple frames of images: Based on the position information of the bow-head region corresponding to the first preset frame image and the position information of the bow-head region corresponding to the second preset frame image, position change information is determined, wherein the first preset frame image and the second preset frame image are different frame images and both belong to the video stream; and The change threshold is determined based on the first preset frame image, the second preset frame image, and the movement threshold; Based on the location change information and the change threshold, the motion state information of the pantograph is determined.

4. The method according to claim 3, wherein, The location change information includes location ascent information; Determining the pantograph's motion state information based on the location change information and the change threshold includes: If all the position rise information obtained after traversing the multiple frames of images exceeds the change threshold, the motion state information is determined to represent the start of the bow lift. Upon determining that the bow lifting has begun, first change status information of the position rising information is determined after a first preset time. If the first change state information indicates that the position rise information remains unchanged, then the motion state information indicates that the bow lift has ended.

5. The method according to claim 3, wherein, The location change information also includes location descent information; Determining the pantograph's motion state information based on the location change information and the change threshold includes: If all the position descent information obtained after traversing the multiple frames of images exceeds the change threshold, the motion state information is determined to indicate the start of bow descent. Upon determining that the bow descent has begun, a second change state information of the position descent information is determined after a second preset time. If the second change state information indicates that the position descent information remains unchanged, then the motion state information indicates that the bow descent has ended.

6. The method according to claim 1, wherein, The method of identifying the position information of the pantograph head area based on the video stream targeting the pantograph area includes: The video stream is input into the bow head recognition model, which outputs the position information of the bow head region. The bow head recognition model is obtained by pre-training a lightweight neural network based on a deep learning framework based on different bow head samples.

7. The method according to claim 1, wherein, The video stream includes multiple frames of images, and the location information includes the location information of the bow head region corresponding to each frame of image; The method further includes: If the edge detection model is activated, the images corresponding to the start and end of the bow lift are input into the edge detection model to perform the following operations: Identify all straight lines in the image and mark all straight lines; Based on the first constraint condition between the pantograph and the contact wire, the contact wire is identified from all the straight lines; The contact point is determined based on the contact line and the bow head area; Based on the contact point and the position information, determine the displacement information of the contact point; Based on the second constraint condition between the pantograph and the carbon sliding plate, identify the straight line parallel to the carbon sliding plate from all the straight lines; Based on the straight line parallel to the carbon slide plate and the position information, the deflection angle information of the carbon slide plate is determined; Output the displacement information of the contact point and the deflection angle information of the carbon slide plate.

8. The method according to claim 7, wherein, The contact point includes a first contact point and a second contact point. The first contact point is used to characterize the contact point in the image corresponding to the start of the bow lift, and the second contact point is used to characterize the contact point in the image corresponding to the end of the bow lift. Determining the displacement information of the contact point based on the contact point and the position information includes: The first position information is determined based on the position information of the bow head region corresponding to the first contact point and the image at the start of the bow raising; The second position information is determined based on the position information of the bow head region corresponding to the second contact point and the image at the end of the bow raising; The displacement information of the contact point is determined based on the first position information and the second position information.

9. The method according to claim 7, wherein, The straight line includes a first straight line and a second straight line. The first straight line is used to characterize the straight line parallel to the carbon skateboard in the image corresponding to the start of the bow lift, and the second straight line is used to characterize the straight line parallel to the carbon skateboard in the image corresponding to the end of the bow lift. The determination of the deflection angle information of the carbon slide plate based on the straight line parallel to the carbon slide plate and the position information includes: The third position information is determined based on the position information of the bow head region corresponding to the first straight line and the image at the start of the bow raising; The fourth position information is determined based on the position information of the bow head region corresponding to the second straight line and the image at the end of the bow lift; The deflection angle information of the carbon slide plate is determined based on the third position information and the fourth position information.

10. A pantograph status detection device, comprising: The identification module is used to identify the location information of the pantograph head area based on the video stream targeting the pantograph area; The determination module is used to determine the pantograph's motion state information based on the location information and preset motion rules; A startup module is used to generate a startup command when the motion state information indicates the end of the bow lift. The startup command activates an edge detection model, which determines the displacement information of the contact point and the deflection angle information of the carbon slide plate from the start to the end of the bow lift. The first termination module is used to generate a termination instruction when the state information indicates that the bow descent has started, wherein the termination instruction is used to terminate the edge detection model.

Images