Train operation control method and device and electronic equipment

Abstract

The invention provides a train operation control method and device and electronic equipment, which can be applied to the technical field of vehicle control, and the method comprises the following steps: receiving a target video collected from a running train, and carrying out environment recognition on the target video to obtain environment recognition results corresponding to a plurality of target environment types, the target video represents a contact state between a pantograph of the train and a contact network, the target environment type represents a condition of interference on shooting of the target video, the target video comprises at least one video frame, and the environment recognition result comprises a sub-recognition result corresponding to the at least one video frame; for each target environment type, verifying the environment recognition result according to the number of target sub-recognition results representing that the environment interference exists in the sub-recognition results to obtain a verification result; and based on the verification results corresponding to the multiple target environment types, determining an operation strategy for controlling the operation of the vehicle.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle control technology, and more specifically to a train operation control method, device, and electronic equipment. Background Art

[0002] With the rapid development of railway technology, the safe operation of high-speed trains is of paramount importance. Real-time monitoring of the pantograph's status, a key component of high-speed trains, is crucial for ensuring safe train operation. Existing intelligent pantograph video analysis equipment effectively identifies various abnormal pantograph conditions by analyzing real-time images from train video streams. However, in actual operation, strong interference from sunlight and other environmental factors can lead to a high number of false alarms, seriously affecting the accuracy of video analysis results. Summary of the Invention

[0003] In view of the above problems, the present disclosure provides a train operation control method, device and electronic equipment.

[0004] According to a first aspect of the present disclosure, a train operation control method is provided, comprising: the method comprising: receiving a target video captured of a running train, and performing environmental identification on the target video to obtain environmental identification results corresponding to each of a plurality of target environment types, wherein the target video represents the contact state between the pantograph of the train and the contact network, the target environment type represents the situation where there is interference with the captured target video, the target video includes at least one video frame, and the environmental identification result includes a sub-identification result corresponding to the at least one video frame; for each target environment type, the environmental identification result is verified according to the number of target sub-identification results representing the presence of environmental interference in the sub-identification result to obtain a verification result; based on the verification results corresponding to each of the plurality of target environment types, an operation strategy for controlling the vehicle operation is determined.

[0005] According to an embodiment of the present application, for each target environment type, the environment identification result is verified according to the number of target sub-identification results in the sub-identification results that characterize the presence of environmental interference, and the verification result obtained includes: determining a first quantity ratio based on the number of target sub-identification results related to each target environment type and the total number of sub-identification results; when the first quantity ratio is greater than a first preset verification threshold, obtaining a first verification result characterizing that the verification is passed, wherein the first verification result characterizes that there is environmental interference of the target environment type in the target video; when the first quantity ratio is less than or equal to the first preset verification threshold, obtaining a second verification result characterizing that the verification is failed, wherein the second verification result characterizes that there is no environmental interference of the target environment type in the target video.

[0006] According to an embodiment of the present application, for each target environment type, the environment recognition result is verified according to the number of target sub-recognition results that characterize the presence of environmental interference in the sub-recognition results, and the verification result obtained also includes: for each target environment type, determining multiple first sub-recognition results from the target sub-recognition results that characterize the presence of environmental interference in at least one sub-recognition result, wherein the multiple video frames corresponding to the multiple first sub-recognition results are a continuous sequence of video frames in the target video; determining a second quantity ratio according to the number of first sub-recognition results and the total number of sub-recognition results; when the second quantity ratio is greater than a second preset verification threshold, obtaining a third verification result that characterizes the passing of the verification, wherein the first preset verification threshold is higher than the second preset verification threshold; when the second quantity ratio is less than or equal to the second preset verification threshold, obtaining a fourth verification result that characterizes the failing of the verification.

[0007] According to an embodiment of the present application, based on the verification results corresponding to each of the multiple target environment types, determining the operating strategy for controlling the vehicle operation includes: when multiple verification results all fail the verification, using the on-board analysis equipment to process the target video to determine the operating strategy; when there is a verification pass among the multiple verification results, determining the historical operating strategy determined before receiving the target video as the operating strategy.

[0008] According to an embodiment of the present application, a target video is processed using an on-board analysis device to determine an operation strategy, including: identifying the contact status between the pantograph and the contact network in the target video to obtain a contact status identification result; when the contact status identification result represents a contact fault, triggering a train alarm to determine the operation strategy based on the fault type and fault location indicated by the train alarm.

[0009] According to an embodiment of the present application, performing environmental recognition on a target video to obtain environmental recognition results corresponding to each of a plurality of target environment types includes: performing sequence frame extraction on the target video based on preset extraction interval parameters to obtain at least one video frame; and processing at least one video frame using recognition models corresponding to each of the plurality of target environment types to obtain environmental recognition results corresponding to each of the plurality of target environment types.

[0010] According to an embodiment of the present application, the recognition model corresponding to the sunlight interference environment type is obtained through training based on the following operations: obtaining training samples, the training samples include sample video frames and sample labels, and the sample labels are sunlight interference categories or non-sunlight interference categories; inputting the sample video frames into the initial recognition model corresponding to the sunlight interference environment type to obtain sample sub-recognition results; training the initial recognition model based on the sample sub-recognition results and sample labels to obtain a recognition model.

[0011] According to an embodiment of the present application, the target environment type includes at least one of the following: a sunlight interference environment type, a rainy day interference environment type, and a snowy day interference environment type.

[0012] The second aspect of the present application provides a train operation control device, including: a response module, which is used to receive a target video collected from a running train, and perform environmental identification on the target video to obtain environmental identification results corresponding to multiple target environment types, wherein the target video represents the contact state between the pantograph of the train and the contact network, the target environment type represents the situation where there is interference with the shooting of the target video, the target video includes at least one video frame, and the environmental identification result includes a sub-identification result corresponding to the at least one video frame; a verification module, which is used to verify the environmental identification result for each target environment type according to the number of target sub-identification results representing the presence of environmental interference in the sub-identification result to obtain a verification result; a determination module, which is used to determine the operation strategy for controlling the vehicle operation based on the verification results corresponding to each of the multiple target environment types.

[0013] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0014] According to an embodiment of the present application, by performing environmental recognition on a target video, an environmental recognition result corresponding to each of a plurality of target environment types is obtained; for each target environment type, the environmental recognition result is verified based on the number of target sub-recognition results in the sub-recognition result that characterize the presence of environmental interference, to obtain a verification result; based on the verification results corresponding to each of the plurality of target environment types, an operation strategy for controlling vehicle operation is determined. Since environmental recognition is performed on a target video to obtain an environmental recognition result corresponding to each of a plurality of target environment types, and then for each target environment type, the environmental recognition result is verified based on the sub-recognition result of each video frame, the overall environmental interference level of the target video is accurately identified, the environmental recognition false alarm rate is reduced, and corresponding operation strategies are determined for different interference levels and target environment interference types, effectively reducing false alarms caused by environmental interference, thereby significantly improving the accuracy of the contact state analysis between the pantograph and the contact network, improving strategy accuracy and train operation safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0016] Figure 1 A flow chart of a train operation control method according to an embodiment of the present disclosure is shown;

[0017] Figure 2 A flowchart of a process for identifying the type of sunlight interference environment according to an embodiment of the present disclosure is shown;

[0018] Figure 3A A schematic diagram showing a pantograph-catenary arcing scenario according to an embodiment of the present disclosure is shown;

[0019] Figure 3B A schematic diagram showing a system falsely reporting a pantograph-catenary arcing event caused by sunlight interference according to an embodiment of the present disclosure is shown.

[0020] Figure 4 shows a structural block diagram of a train operation control device according to an embodiment of the present application; and

[0021] Figure 5 A block diagram of an electronic device suitable for implementing a train operation control method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0023] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0024] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0025] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0026] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved all comply with relevant laws and regulations, employ necessary confidentiality measures, and do not violate public order and good morals. In the technical solutions disclosed herein, user authorization or consent is obtained before obtaining or collecting user personal information.

[0027] The inventors have discovered that existing pantograph video intelligent analysis equipment can effectively identify various abnormal conditions of the pantograph by acquiring images from the train video stream in real time for analysis. However, in actual operation, strong interference from other environmental types such as sunlight can lead to a large number of false alarms, seriously affecting the accuracy of the video analysis results. For example, the bright pixel blocks formed by the backlight of the sun may be similar to the characteristics of the arcing of the bow network, causing the system to falsely report that the pantograph is in an arcing state. Therefore, before identifying the status of the pantograph, it is necessary to accurately predict whether there is interference in the video, and then filter out the invalid video after the interference to reduce the error rate of video analysis.

[0028] In order to at least partially resolve the technical problems existing in the related art, the present disclosure provides a train operation control method, device, and electronic device. The method includes: receiving a target video captured from a running train, performing environmental recognition on the target video, and obtaining environmental recognition results corresponding to multiple target environment types, wherein the target video represents the contact state between the train's pantograph and the overhead line, the target environment type represents the presence of interference in the captured target video, the target video includes at least one video frame, and the environmental recognition result includes sub-recognition results corresponding to the at least one video frame; for each target environment type, verifying the environmental recognition result based on the number of target sub-recognition results representing the presence of environmental interference in the sub-recognition result to obtain a verification result; and determining an operation strategy for controlling vehicle operation based on the verification results corresponding to each of the multiple target environment types.

[0029] In the technical solution of the present invention, the user information involved (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0030] Figure 1 A flow chart of a train operation control method according to an embodiment of the present disclosure is shown.

[0031] like Figure 1As shown, the train operation control method 100 of this embodiment includes operations S110 to S130.

[0032] In operation S110 , a target video captured of a running train is received, and environment recognition is performed on the target video to obtain environment recognition results corresponding to respective ones of a plurality of target environment types.

[0033] According to an embodiment of the present application, the system architecture of the train includes a video acquisition device, which is used to acquire the target video in real time. The video acquisition device is deployed near the train pantograph and is equipped with a high-resolution camera to ensure that the operating status of the pantograph can be clearly captured. The camera transmits the target video to the analysis device in real time via wired or wireless transmission. The analysis device uses a high-performance computing platform with strong computing power and low power consumption characteristics, and can efficiently process the target video. The frame rate of the target video can be set according to actual needs to ensure the real-time and integrity of the data.

[0034] According to an embodiment of the present application, the target video represents the contact status between the pantograph of the train and the contact network, and the contact status can be a good contact state or an abnormal contact state.

[0035] According to the embodiments of the present application, when the contact is good, the contact network supplies power to the train by contacting the pantograph; the abnormal contact state may be caused by various factors such as excessive contact pressure, insufficient pressure, current interruption, etc., and it is necessary to use analysis equipment to objectively analyze the factors of the target video while ensuring that the target video is not interfered with by the environment.

[0036] According to an embodiment of the present application, the train may be a rail transit vehicle, such as a high-speed rail, a train, etc.

[0037] According to an embodiment of the present application, the target environment type represents a situation in which there is interference with shooting the target video, for example, a sunlight interference environment type.

[0038] According to an embodiment of the present application, the target video includes at least one video frame, and the target video is a video stream.

[0039] According to an embodiment of the present application, a convolutional neural network can be used to perform environmental recognition on a target video to obtain environmental recognition results corresponding to multiple target environment types, and the environmental recognition results include sub-recognition results corresponding to at least one video frame.

[0040] For example, the target video includes video frame A, video frame B, and video frame C, and the target environment type includes type 1, type 2, and type 3. The target video is subjected to environmental recognition to obtain environmental recognition results corresponding to type 1, type 2, and type 3, respectively. The environmental recognition result corresponding to type 1 includes sub-recognition results corresponding to video frame A, video frame B, and video frame C, respectively. For example, for type 1, the sub-recognition result of video frame A is that there is environmental interference, the sub-recognition result of video frame B is that there is environmental interference, and the sub-recognition result of video frame C is that there is no environmental interference.

[0041] In operation S120 , for each target environment type, the environment recognition result is verified according to the number of target sub-recognition results indicating the presence of environmental interference in the sub-recognition results to obtain a verification result.

[0042] According to an embodiment of the present application, the target sub-recognition result is a sub-recognition result indicating the presence of environmental interference.

[0043] According to an embodiment of the present application, the environment recognition result corresponding to each target environment type includes a sub-recognition result corresponding to at least one video frame. The environment recognition result is verified according to the number of target sub-recognition results and the verification threshold to obtain a verification result.

[0044] According to an embodiment of the present application, the verification result is an evaluation of the credibility of the environment recognition result of the target video.

[0045] For example, for type 1, the sub-identification result of video frame A is that there is environmental interference, the sub-identification result of video frame B is that there is environmental interference, and the sub-identification result of video frame C is that there is no environmental interference. The number of target sub-identification results is 2. According to the number of target sub-identification results and the verification threshold, if the verification result is passed, it means that there is type 1 environmental interference in the target video; if the verification result is failed, it means that there is no type 1 environmental interference in the target video.

[0046] In operation S130 , an operation strategy for controlling the operation of the vehicle is determined based on the verification results corresponding to each of the plurality of target environment types.

[0047] According to an embodiment of the present application, based on the verification results corresponding to multiple target environment types, the degree of interference of the environment on the target video is comprehensively evaluated to evaluate the reliability of the information in the target video, and then a corresponding analysis method is adopted to determine the operating strategy for controlling the vehicle operation.

[0048] According to an embodiment of the present application, the operation strategy is a strategy for controlling the safe operation of a train.

[0049] According to the embodiments of the present application, by performing environmental identification on the target video, environmental identification results corresponding to multiple target environment types are obtained, and then for each target environment type, the environmental identification results are verified based on the sub-identification results of each video frame to accurately identify the overall environmental interference level of the target video, reduce the false alarm rate of environmental identification, and determine corresponding operating strategies for different interference levels and target environmental interference types, effectively reducing false alarms caused by environmental interference, thereby greatly improving the accuracy of the contact status analysis between the pantograph and the contact network, and improving the strategy accuracy and train operation safety.

[0050] According to an embodiment of the present application, the target environment type includes at least one of the following: a sunlight interference environment type, a rainy day interference environment type, and a snowy day interference environment type.

[0051] According to an embodiment of the present application, the sunlight interference environment type is a situation where sunlight interferes with the shooting of the target video.

[0052] According to an embodiment of the present application, the rainy day interference environment type is a situation where rain interferes with shooting the target video.

[0053] According to an embodiment of the present application, the snow interference environment type is a situation where snow and fog interfere with the shooting of the target video.

[0054] According to the embodiments of the present application, the present application can accurately identify whether there is interference from multiple target environment types before performing contact status analysis on the target video, and is thus widely applicable to bow head morphology, lighting conditions and railway operating environment, and can be applied to target video analysis tasks in different scenarios.

[0055] According to an embodiment of the present application, for each target environment type, the environment identification result is verified according to the number of target sub-identification results in the sub-identification results that characterize the presence of environmental interference, and the verification result obtained includes: determining a first quantity ratio based on the number of target sub-identification results related to each target environment type and the total number of sub-identification results; when the first quantity ratio is greater than a first preset verification threshold, obtaining a first verification result characterizing that the verification is passed, wherein the first verification result characterizes that there is environmental interference of the target environment type in the target video; when the first quantity ratio is less than or equal to the first preset verification threshold, obtaining a second verification result characterizing that the verification is failed, wherein the second verification result characterizes that there is no environmental interference of the target environment type in the target video.

[0056] According to an embodiment of the present application, for target environment type 1, a first quantity ratio is determined according to a ratio between the number of target sub-recognition results and the total number of sub-recognition results.

[0057] According to an embodiment of the present application, the first preset verification threshold may be 60%.

[0058] According to an embodiment of the present application, when the first number ratio is greater than 60%, a first verification result indicating that the verification has passed is obtained, indicating that the target video has environmental interference of target environment type 1.

[0059] According to an embodiment of the present application, when the first number ratio is less than or equal to 60%, a second verification result indicating verification failure is obtained, indicating that the target video does not have environmental interference of target environment type 1.

[0060] In one embodiment, the first quantity ratio Confidence of target environment type 1 is as shown in formula (1):

[0061] (1);

[0062] Among them, T represents the total number of sub-recognition results, Represents the sub-recognition result corresponding to the t-th video frame, The result of characterization sub-recognition is that there is environmental interference, The result of characterizing the sub-identification is that there is no environmental interference.

[0063] In one embodiment, the verification result D is as shown in formula (2):

[0064] (2);

[0065] in, The first verification result indicating that the verification has passed, The second verification result indicating that the verification failed, Characterizes the first preset verification threshold, Characterizes a first quantity ratio.

[0066] According to an embodiment of the present application, for each target environment type, the environment recognition result is verified according to the number of target sub-recognition results that characterize the presence of environmental interference in the sub-recognition results, and the verification result obtained also includes: for each target environment type, determining multiple first sub-recognition results from the target sub-recognition results that characterize the presence of environmental interference in at least one sub-recognition result, wherein the multiple video frames corresponding to the multiple first sub-recognition results are a continuous sequence of video frames in the target video; determining a second quantity ratio according to the number of first sub-recognition results and the total number of sub-recognition results; when the second quantity ratio is greater than a second preset verification threshold, obtaining a third verification result that characterizes the passing of the verification, wherein the first preset verification threshold is higher than the second preset verification threshold; when the second quantity ratio is less than or equal to the second preset verification threshold, obtaining a fourth verification result that characterizes the failing of the verification.

[0067] According to an embodiment of the present application, the second quantity ratio represents the percentage of frames in which the sub-recognition results of the statistical continuous video frames have environmental interference.

[0068] For example, for target environment type 1, the sub-recognition result of video frame A is that there is environmental interference, the sub-recognition result of video frame B is that there is no environmental interference, the sub-recognition result of video frame C is that there is environmental interference, the sub-recognition result of video frame D is that there is environmental interference, and the sub-recognition result of video frame E is that there is environmental interference. From the 4 target sub-recognition results representing the presence of environmental interference among the 5 sub-recognition results, 3 first sub-recognition results are determined, and the video frames C, video frames D, and video frames E corresponding to the 3 first sub-recognition results are a continuous video frame sequence.

[0069] According to an embodiment of the present application, the second quantity ratio is obtained according to the ratio between the quantity of the first sub-recognition results and the total quantity of the sub-recognition results.

[0070] According to an embodiment of the present application, since the number of the first sub-recognition results is determined based on the proportion of frames in which the sub-recognition results of continuous video frames are those in which environmental interference exists, additional conditional constraints of continuous video frames are added. Therefore, when performing credibility verification, a second preset verification threshold that is smaller than the first preset verification threshold is adopted. For example, the second preset verification threshold can be 55%.

[0071] According to an embodiment of the present application, when the second number ratio is greater than 55%, a third verification result indicating that the verification has passed is obtained, indicating that the target video has environmental interference of target environment type 1.

[0072] According to an embodiment of the present application, when the second number ratio is less than or equal to 55%, a fourth verification result indicating verification failure is obtained, indicating that the target video does not have environmental interference of target environment type 1.

[0073] According to an embodiment of the present application, based on the verification results corresponding to each of the multiple target environment types, determining the operating strategy for controlling the vehicle operation includes: when multiple verification results all fail the verification, using the on-board analysis equipment to process the target video to determine the operating strategy; when there is a verification pass among the multiple verification results, determining the historical operating strategy determined before receiving the target video as the operating strategy.

[0074] According to the embodiment of the present application, multiple verification results are all verification failures, which means that the target video does not have any environmental interference of any target environment type, and the contact status between the pantograph and the contact network captured by the target video is real and reliable. Therefore, the target video is processed using the on-board analysis equipment to determine the operation strategy.

[0075] According to an embodiment of the present application, the system architecture of the train also includes an on-board analysis device, which adopts a high-performance computing platform, has powerful computing capabilities and low power consumption characteristics, and can efficiently process target videos.

[0076] According to an embodiment of the present application, the hardware platform of the in-vehicle analysis equipment has a computing power of 32 trillion operations per second at a peak performance to support the efficient operation of deep learning models. While ensuring computing power, power consumption must be controlled within a reasonable range to adapt to in-vehicle or other resource-constrained operating environments.

[0077] According to an embodiment of the present application, the computing power and power consumption of the vehicle-mounted analysis device satisfy the following relationship: and .in, Indicates the computing power of the on-board analysis equipment, Characterizes the minimum computing power required to run a deep learning model, Indicates the actual power consumption of the vehicle-mounted analysis equipment, The maximum power consumption allowed for the on-board analysis equipment.

[0078] According to an embodiment of the present application, an on-board analysis device is used to process the target video, analyze the contact status, and then determine the operation strategy based on the real-time analysis of the contact status to ensure the safe operation of the train.

[0079] For example, if the contact state is a spark-generating state, an early warning can be issued and a braking strategy can be implemented.

[0080] According to an embodiment of the present application, if there is a verification pass among multiple verification results, it means that there is environmental interference of at least one target environment type in the target video. The information in the target video is based on the video shot under environmental interference, and the real information of the video may be covered. Therefore, there is no need to further analyze the target video, filter out this invalid target video, and determine the historical operation strategy determined before receiving the target video as the operation strategy.

[0081] According to an embodiment of the present application, the train continues to operate based on the historical operation strategy and continues to identify the environment type based on the target video of the next time period received in real time.

[0082] According to the embodiments of the present application, the environmental interference scenes of the target video are accurately identified and filtered. For the target video determined to have environmental interference, the subsequent on-board analysis equipment is temporarily stopped from analyzing the target video, and the interference environment type is recorded. The target video of the next time period is continued to be accepted and identified. After it is determined that the target video does not have environmental interference, the operation of the on-board analysis equipment is resumed, which effectively reduces the false alarm phenomenon caused by environmental interference, thereby greatly improving the accuracy of real-time analysis of the target video to reduce the false alarm rate.

[0083] According to an embodiment of the present application, a target video is processed using an on-board analysis device to determine an operation strategy, including: identifying the contact status between the pantograph and the contact network in the target video to obtain a contact status identification result; triggering a train alarm when the contact status identification result represents a contact fault; and determining an operation strategy based on the fault type and fault location indicated by the train alarm.

[0084] According to an embodiment of the present application, the contact status identification result may be normal contact or contact fault.

[0085] According to an embodiment of the present application, when the contact status identification result may be that the contact is normal, the train can continue to run based on the historical operation strategy.

[0086] According to an embodiment of the present application, when the contact status identification result indicates a contact fault, a train alarm is triggered to prompt staff to handle the fault.

[0087] According to an embodiment of the present application, contact faults represent various abnormal conditions between the train pantograph and the contact network arranged along the railway track, such as structural abnormalities, hanging foreign objects, pantograph tilt, pantograph-net arcing, and dirty screen.

[0088] According to the embodiments of the present application, the fault type and fault location can be extracted from the alarm log, and then the fault type and fault location can be processed using an intelligent decision model to obtain an operation strategy.

[0089] For example, if the fault type is pantograph tilt and the fault location is pantograph 001, the operation strategy is slow operation, and the maintenance personnel are prompted to repair pantograph 001.

[0090] According to an embodiment of the present application, performing environmental recognition on a target video to obtain environmental recognition results corresponding to each of a plurality of target environment types includes: performing sequence frame extraction on the target video based on preset extraction interval parameters to obtain at least one video frame; and processing at least one video frame using recognition models corresponding to each of the plurality of target environment types to obtain environmental recognition results corresponding to each of the plurality of target environment types.

[0091] According to an embodiment of the present application, at least one video frame is extracted from the target video according to a preset extraction interval parameter, for example, 25 video frames are extracted per second.

[0092] According to an embodiment of the present application, the extracted video frame is resized to 224×224 pixels to meet the input requirements of the recognition model; the video frame is then normalized, first converting the image format into a tensor format (Tensor), and the pixel value range in the video frame from [0,255] to [0,1], and then the mean and standard deviation of each channel in the recognition model are set, and the pixel range is changed to [-1,1] to improve the input consistency of the recognition model.

[0093] In one embodiment, the normalization function is shown in formula (3):

[0094] (3);

[0095] in, Represents the pixel value of the i-th row, j-th column, and c-th channel of the t-th video frame, represents the normalized pixel value of the i-th row, j-th column, and c-th channel of the t-th video frame. Characterizes the mean of the c-th channel, Characterizes the standard deviation of channel c.

[0096] According to an embodiment of the present application, each target environment type corresponds one-to-one to a recognition model.

[0097] According to an embodiment of the present application, for each target environment type, at least one preprocessed video frame is processed using a recognition model corresponding to the target environment type to obtain a sub-recognition result corresponding to each of the at least one video frame.

[0098] According to an embodiment of the present application, the recognition model can be a lightweight deep learning model, which is used to analyze the preprocessed video frames. Through its lightweight and efficient network structure, it significantly reduces the consumption of computing resources while ensuring recognition accuracy.

[0099] According to an embodiment of the present application, the recognition model corresponding to the sunlight interference environment type is obtained through training based on the following operations: obtaining training samples, the training samples include sample video frames and sample labels, and the sample labels are sunlight interference categories or non-sunlight interference categories; inputting the sample video frames into the initial recognition model corresponding to the sunlight interference environment type to obtain sample sub-recognition results; training the initial recognition model based on the sample sub-recognition results and sample labels to obtain a recognition model.

[0100] According to an embodiment of the present application, a large number of sample target videos are collected, the sample target videos represent the contact status between the pantograph of the train and the contact network, the sample target videos cover different lighting conditions, seasons and operating environments, and the sample target videos are extracted to obtain sample video frames.

[0101] According to an embodiment of the present application, each sample video frame has a sample label, and a large number of sample video frames are divided into sunlight interference categories or non-sunlight interference categories to construct training samples. Data enhancement algorithms (such as random cropping, flipping, rotation, etc.) can be used to expand the training samples.

[0102] According to an embodiment of the present application, a sample video frame is input into an initial recognition model corresponding to the type of sunlight interference environment to obtain a sample sub-recognition result. The initial recognition model can be constructed based on a lightweight deep learning network (Lightweight Deep Learning Models).

[0103] According to an embodiment of the present application, during the training process, the model parameters are adjusted using an adaptive optimizer (Adaptive Moment Estimation with Weight Decay, AdamW), with minimization of a loss function as the optimization goal. The loss function may be a cross-entropy loss function.

[0104] In one embodiment, the loss function is shown in formula (4):

[0105] (4);

[0106] Among them, M represents the number of sample video frames, Characterize the sample label of the mth sample video frame, Represents the sample sub-recognition result of the mth sample video frame belonging to category n, N=1, n=0 represents the absence of environmental interference, n=1 represents the presence of environmental interference, L represents the loss value,

[0107] According to an embodiment of the present application, cross-validation is used to evaluate the performance of the recognition model to ensure that the recognition model has good recognition accuracy on the test set. The performance indicators of cross-validation can be precision, accuracy, recall and F1 score. Precision and recall are two complementary indicators. A high precision rate means that the model predicts the positive class with high accuracy, but may miss some samples that are actually positive (ie, the recall rate is low). A high recall rate means that the model can cover more actual positive samples, but may mistakenly predict some samples that are actually negative as positive (ie, the precision rate is low). is the true negative (the number of samples correctly predicted as negative), is the false negative (the number of samples incorrectly predicted as negative).

[0108] According to an embodiment of the present application, the last fully connected layer of the recognition model is modified into an output layer suitable for binary classification tasks, and the output feature number of the last fully connected layer is replaced by the output feature number required for binary classification, which is the output feature number, corresponding to the two categories of "sunlight interference scene" and "non-sunlight interference scene".

[0109] According to the embodiments of the present application, the target video analysis of the train pantograph requires real-time processing of a large amount of data. The lightweight recognition model has the characteristics of lightweight and efficient reasoning. The use of the lightweight recognition model can significantly reduce the consumption of computing resources while ensuring recognition accuracy, meeting the real-time requirements; on resource-constrained devices, the lightweight model can run efficiently without the need for additional hardware acceleration equipment. The lightweight deep learning model, due to its efficient design and fast reasoning capabilities, significantly reduces the computational complexity while maintaining a high accuracy rate by optimizing the internal structure. In addition, it also supports a variety of parameter adjustment methods and can flexibly adapt to different application scenarios. These features enable the model to run efficiently on resource-constrained devices, meeting the real-time requirements of pantograph video analysis.

[0110] Figure 2 A flowchart of a process for identifying the type of sunlight interference environment according to an embodiment of the present disclosure is shown.

[0111] like Figure 2 As shown, the process of identifying the type of sunlight interference environment includes steps S201 to S210.

[0112] In step S201, the video acquisition device continuously captures the target video of the running train and sends it to the train, and simultaneously proceeds to step S202 and step S203.

[0113] In step S202 , the capture device on the train continuously receives the target video.

[0114] In step S203, the target video is stored in the ground-side server device.

[0115] In step S204, a training data set of sunlight interference environment type is constructed.

[0116] In step S205 , a recognition model corresponding to the sunlight interference environment type is trained based on the training data set, and the weight of the recognition model is updated.

[0117] In step S206, the trained recognition model corresponding to the sunlight interference environment type is used to process the target video acquired in real time to obtain a sub-recognition result corresponding to at least one video frame.

[0118] In step S207 , the environmental recognition result is verified according to the number of target sub-recognition results indicating the presence of environmental interference in the sub-recognition results to obtain a verification result.

[0119] In step S208, it is determined whether the verification result is passed. If so, the process proceeds to step S209; otherwise, the process proceeds to step S210.

[0120] In step S209, the target video is filtered and the vehicle-mounted analysis device is temporarily stopped from analyzing the target video.

[0121] In step S210 , the target video is processed using the vehicle-mounted analysis device.

[0122] Figure 3A A schematic diagram of a pantograph-catenary arcing scenario according to an embodiment of the present disclosure is shown.

[0123] like Figure 3A As shown, the overhead contact network 301 installed along the track contacts the pantograph 302 on the train to supply power to the train. The pixel area 303 in the captured video frame shows a bright pixel block. Based on the on-board analysis equipment analyzing the target video, the system alarms that the pantograph has arcing. Arcing refers to the arcing phenomenon caused by poor contact or instantaneous disconnection between the pantograph and the overhead contact network.

[0124] Figure 3B A schematic diagram showing a system falsely reporting a pantograph-catenary arcing event caused by sunlight interference according to an embodiment of the present disclosure is shown.

[0125] like Figure 3B As shown, the contact network 304 set along the track contacts the pantograph 305 on the train to supply power to the train. Due to the backlight of the sun, the pixel area 306 in the captured video frame appears as a bright pixel block. The characteristics of the pixel area 306 are similar to the characteristics of the pantograph arcing, which causes the system to falsely report the pantograph arcing.

[0126] Based on the above train operation control method, the present invention also provides a train operation control device. Figure 4 The device is described in detail.

[0127] Figure 4 A structural block diagram of a train operation control device according to an embodiment of the present invention is shown.

[0128] like Figure 4 As shown, the train operation control device 400 of this embodiment includes a response module 410 , a verification module 420 and a determination module 430 .

[0129] Response module 410 is configured to receive a target video captured of a running train and perform environmental recognition on the target video to obtain environmental recognition results corresponding to multiple target environment types. The target video represents the contact state between the train's pantograph and the overhead line, the target environment type represents interference with the captured target video, the target video includes at least one video frame, and the environmental recognition result includes sub-recognition results corresponding to the at least one video frame. In one embodiment, response module 410 may be configured to perform operation S110 described above and will not be further described herein.

[0130] Verification module 420 is configured to verify the environment recognition results for each target environment type based on the number of target sub-recognition results that indicate the presence of environmental interference, thereby obtaining a verification result. In one embodiment, verification module 420 may be configured to perform operation S120 described above, which will not be further described herein.

[0131] The determination module 430 is configured to determine an operation strategy for controlling the vehicle operation based on the verification results corresponding to each of the plurality of target environment types. In one embodiment, the determination module 430 may be configured to execute the operation S130 described above, which will not be described in detail herein.

[0132] According to an embodiment of the present application, the verification module 420 includes a first syndrome module, a second syndrome module, and a third syndrome module.

[0133] The first verification submodule is configured to determine a first quantity ratio according to the quantity of target sub-identification results related to each target environment type and the total quantity of sub-identification results.

[0134] The second verification submodule is configured to obtain a first verification result indicating that the verification has passed when the first number ratio is greater than a first preset verification threshold, wherein the first verification result indicates that the target video has environmental interference of the target environment type.

[0135] The third verification submodule is configured to obtain, when the first number ratio is less than or equal to a first preset verification threshold, a second verification result indicating that the verification fails, wherein the second verification result indicates that the target video does not have environmental interference of the target environment type.

[0136] According to an embodiment of the present application, the verification module 420 further includes a fourth syndrome module, a fifth syndrome module, a sixth syndrome module, and a seventh syndrome module.

[0137] The fourth verification submodule is used to determine, for each target environment type, multiple first sub-recognition results from the target sub-recognition results that represent the presence of environmental interference in at least one sub-recognition result, wherein the multiple video frames corresponding to each of the multiple first sub-recognition results are a continuous video frame sequence in the target video.

[0138] The fifth check submodule is configured to determine a second quantity ratio according to the quantity of the first sub-identification results and the total quantity of the sub-identification results.

[0139] The sixth check submodule is configured to obtain a third check result indicating that the check has passed when the second number ratio is greater than a second preset check threshold, wherein the first preset check threshold is higher than the second preset check threshold.

[0140] The seventh check submodule is configured to obtain a fourth check result indicating that the check has failed when the second number ratio is less than or equal to a second preset check threshold.

[0141] According to an embodiment of the present application, the determination module 430 includes a first determination submodule and a second determination submodule.

[0142] The first determination submodule is used to process the target video using the vehicle-mounted analysis device to determine the operation strategy when multiple verification results all fail the verification.

[0143] The second determining submodule is configured to determine, when one of the multiple verification results passes the verification, a historical operation strategy determined before receiving the target video as the operation strategy.

[0144] According to an embodiment of the present application, the first determining submodule includes a first determining unit, a second determining unit, and a third determining unit.

[0145] The first determining unit is used to identify the contact state between the pantograph and the contact network in the target video to obtain a contact state identification result.

[0146] The second determining unit is configured to trigger a train alarm when the contact state identification result indicates a contact fault.

[0147] The third determining unit is used to determine the operation strategy according to the fault type and fault location indicated by the train alarm.

[0148] According to an embodiment of the present application, the response module 410 includes a first response submodule and a second response submodule.

[0149] The first response submodule is used to extract sequence frames from the target video based on a preset extraction interval parameter to obtain at least one video frame.

[0150] The second response submodule is used to process at least one video frame using the recognition models corresponding to the multiple target environment types respectively, to obtain the environment recognition results corresponding to the multiple target environment types respectively.

[0151] According to embodiments of the present application, any number of modules, submodules, units, and subunits may be combined into one module for implementation, or any one of these modules may be split into multiple modules. Alternatively, partial functionality of one or more of these modules may be combined with partial functionality of other modules and implemented in one module. At least one of the modules, submodules, units, and subunits may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuit, or may be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of these. Alternatively, at least one of the modules, submodules, units, and subunits may be at least partially implemented as a computer program module that, when executed, performs the corresponding function.

[0152] Figure 5 A block diagram of an electronic device suitable for implementing a train operation control method according to an embodiment of the present application is shown.

[0153] like Figure 5 As shown, an electronic device 500 according to an embodiment of the present application includes a processor 501, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 502 or a program loaded from a storage unit 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present application.

[0154] Various programs and data required for the operation of the electronic device 500 are stored in the RAM 503. The processor 501, ROM 502, and RAM 503 are connected to each other via a bus 504. The processor 501 performs various operations of the method flow according to the embodiment of the present application by executing the programs in the ROM 502 and / or RAM 503. It should be noted that the programs may also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 may also perform various operations of the method flow according to the embodiment of the present application by executing the programs stored in one or more memories.

[0155] According to an embodiment of the present application, electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to bus 504. Electronic device 500 may also include one or more of the following components connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN card or modem. Communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 510 as needed, so that computer programs read from the removable media can be installed into storage section 508 as needed.

[0156] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the train operation control method according to the embodiments of this application.

[0157] According to an embodiment of the present application, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, a computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.

[0158] The embodiments of the present application also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is executed in a computer system, the program code is used to enable the computer system to implement the train operation control method provided in the embodiments of the present application.

[0159] The computer program executes the above functions defined in the system / device of the embodiment of the present application when the computer program is executed by the processor 501. According to the embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0160] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 509, and / or installed from a removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0161] In such an embodiment, the computer program can be downloaded and installed from the network via the communication section 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above-mentioned functions defined in the system of the embodiment of the present application are performed. According to the embodiment of the present application, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0162] According to an embodiment of the present application, the program code for executing the computer program provided by the embodiment of the present application can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0163] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as the combination of boxes in the block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or coupled in various ways, and all of these combinations and / or couplings fall within the scope of the present disclosure.

[0164] The above describes the embodiments of the present disclosure. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A train operation control method, characterized in that: The method comprises: Receiving a target video captured from a running train, and performing environment recognition on the target video to obtain environment recognition results corresponding to respective target environment types, wherein the target video represents a contact state between a pantograph of the train and a contact network, the target environment type represents a situation where interference exists in capturing the target video, the target video includes at least one video frame, and the environment recognition result includes a sub-recognition result corresponding to the at least one video frame; For each target environment type, verifying the environment recognition result according to the number of target sub-recognition results indicating the presence of environmental interference in the sub-recognition results to obtain a verification result; An operation strategy for controlling the operation of the vehicle is determined based on the verification results corresponding to each of the plurality of target environment types.

2. The method according to claim 1, characterized in that For each target environment type, verifying the environment recognition result according to the number of target sub-recognition results indicating the presence of environmental interference in the sub-recognition results, to obtain a verification result includes: determining a first quantity ratio according to the number of target sub-identification results associated with each target environment type and the total number of sub-identification results; When the first number ratio is greater than a first preset verification threshold, obtaining a first verification result indicating that the verification has passed, wherein the first verification result indicates that the target video has environmental interference of the target environment type; When the first number ratio is less than or equal to the first preset verification threshold, a second verification result indicating verification failure is obtained, wherein the second verification result indicates that the target video does not have environmental interference of the target environment type.

3. The method according to claim 2, characterized in that For each target environment type, verifying the environment recognition result according to the number of target sub-recognition results indicating the presence of environmental interference in the sub-recognition results, and obtaining the verification result further includes: For each target environment type, determining a plurality of first sub-recognition results from at least one target sub-recognition result indicating the presence of environmental interference, wherein the plurality of video frames corresponding to the plurality of first sub-recognition results are a continuous sequence of video frames in the target video; determining a second quantity ratio according to the quantity of the first sub-identification results and the total quantity of the sub-identification results; When the second number ratio is greater than a second preset verification threshold, obtaining a third verification result indicating that the verification has passed, wherein the first preset verification threshold is higher than the second preset verification threshold; When the second number ratio is less than or equal to the second preset verification threshold, a fourth verification result indicating that the verification fails is obtained.

4. The method according to claim 1, wherein The determining of the operation strategy for controlling the operation of the vehicle based on the verification results corresponding to each of the plurality of target environment types includes: If the plurality of verification results are all verification failures, processing the target video using an onboard analysis device to determine an operation strategy; In the case that one of the multiple verification results passes the verification, a historical operation strategy determined before receiving the target video is determined as the operation strategy.

5. The method according to claim 4, characterized in that The process of processing the target video using the vehicle-mounted analysis device to determine the operation strategy includes: Identifying the contact state between the pantograph and the contact network in the target video to obtain a contact state identification result; If the contact state identification result indicates a contact fault, triggering a train alarm; An operation strategy is determined based on the fault type and fault location indicated by the train alarm.

6. The method according to claim 1, characterized in that The performing environment recognition on the target video to obtain environment recognition results corresponding to the multiple target environment types includes: Based on a preset extraction interval parameter, extracting a sequence frame of the target video to obtain at least one video frame; At least one of the video frames is processed using recognition models corresponding to the multiple target environment types respectively to obtain environment recognition results corresponding to the multiple target environment types respectively.

7. The method according to claim 6, characterized in that The recognition model corresponding to the sunlight interference environment type is obtained through training based on the following operations: Acquire a training sample, where the training sample includes a sample video frame and a sample label, where the sample label is a sunlight interference category or a non-sunlight interference category; Inputting the sample video frame into an initial recognition model corresponding to the sunlight interference environment type to obtain a sample sub-recognition result; The initial recognition model is trained according to the sample sub-recognition results and the sample labels to obtain the recognition model.

8. The method according to claim 1, characterized in that The target environment type includes at least one of the following: Sunlight interference environment type, rainy day interference environment type, snowy day interference environment type.

9. A train operation control device, characterized in that: The device comprises: a response module, configured to receive a target video captured of a running train, and perform environmental recognition on the target video to obtain environmental recognition results corresponding to respective ones of a plurality of target environment types, wherein the target video represents a contact state between a pantograph of the train and a contact network, the target environment type represents a situation where interference exists in capturing the target video, the target video includes at least one video frame, and the environmental recognition result includes a sub-recognition result corresponding to the at least one video frame; a verification module, configured to verify the environment recognition result for each target environment type according to the number of target sub-recognition results indicating the presence of environmental interference in the sub-recognition results, to obtain a verification result; A determination module is used to determine an operation strategy for controlling the operation of the vehicle based on the verification results corresponding to each of the multiple target environment types.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images