Monitoring method and device for railway system

Abstract

The invention provides a monitoring method and device for a railway system. The method comprises the following steps: acquiring sensor data and a real-time monitoring video in the railway system; based on the sensor data and the real-time monitoring video, extracting a key event from the real-time monitoring video; extracting a real video sub-image of a main object corresponding to the key event from the real-time monitoring video; objects and / or areas except the main body object in the real video sub-image are / is set as backgrounds; modeling the background based on a building information model technology to obtain a virtualized virtual background; and synthesizing the real video subimage in the virtual background to obtain a target monitoring video, and displaying the target monitoring video. Therefore, the main object involved in the key event can be highlighted, so that monitoring personnel can quickly identify key information from the target monitoring video.

Description

Technical Field

[0001] The present invention relates to the technical field of railway operation monitoring, and in particular to a monitoring method and device for a railway system. Background Art

[0002] Railway system monitoring is crucial in ensuring the safety of railway operations. However, traditional railway monitoring systems usually monitor in two ways: one is to display the video captured by the monitoring camera in real time, and the other is to display the railway operation status in two or three dimensions through digital simulation.

[0003] For the former, monitoring personnel need to switch to view each video surveillance screen one by one, which is extremely inefficient. Moreover, a single video screen contains a large amount of information, such as trains, pedestrians, road infrastructure, buildings and trees, which often move at the same time, making it difficult for monitoring personnel to quickly extract key information, thus affecting the operating efficiency of the railway monitoring system.

[0004] For the latter, pure digital simulation often ignores some on-site details and cannot provide detailed and true on-site conditions, which makes it difficult for monitoring personnel to fully understand the actual situation. This lack of detailed display also makes it difficult for monitoring personnel to make effective decisions. Summary of the invention

[0005] The present invention provides a monitoring method and device for a railway system, which are used to display monitoring information of the railway system in real time by combining virtuality and reality, so that key information can be displayed intuitively.

[0006] In a first aspect, the present invention provides a monitoring method for a railway system, comprising:

[0007] Obtain sensor data and real-time monitoring video in railway systems;

[0008] Extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video;

[0009] Extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video;

[0010] In the real video sub-image, objects and / or regions other than the main object are set as background;

[0011] Modeling the background based on building information modeling technology to obtain a virtual background after virtualization processing;

[0012] The real video sub-image is synthesized into the virtual background to obtain the target monitoring video and display it.

[0013] Optionally, extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video includes:

[0014] extracting sensor characteristic data including train speed, train acceleration and signal change from the sensor data;

[0015] Extracting video feature data including train movement trajectory, vehicle position and pedestrian position from the real-time monitoring video;

[0016] The sensor feature data and the video feature data are input into a pre-trained key event recognition network to obtain the key events.

[0017] Optionally, after inputting the sensor feature data and the video feature data into a pre-trained key event recognition network to obtain the key event, the method further includes:

[0018] Based on the event type of the key event, the key event is checked by using a preset threshold and / or preset rule of a corresponding type.

[0019] Optionally, extracting a real video sub-image of a subject object corresponding to the key event from the real-time surveillance video includes:

[0020] Based on the building information model technology, an event view corresponding to the key event is formed;

[0021] Determining the subject object from the event view;

[0022] The real video sub-image of the main object is extracted from the real-time monitoring video.

[0023] Optionally, based on the building information model technology, an event view corresponding to the key event is formed, including:

[0024] Based on the building information modeling technology, determining the locations of all the objects;

[0025] Performing spatial analysis on the objects respectively and identifying the relationships between the objects to obtain spatial analysis results and relationship integration results;

[0026] The spatial analysis results and the relationship integration results are integrated to form an event view corresponding to the key time.

[0027] Optionally, the background is modeled based on building information modeling technology to obtain a virtual background after virtualization processing, including:

[0028] Based on the event type of the critical event, determine the corresponding urgency through the preset threshold and / or the preset rule of the corresponding type;

[0029] The background is modeled based on the building information modeling technology, and the model obtained by modeling is fuzzy processed according to the urgency through graphic processing technology to obtain the virtual background.

[0030] In a second aspect, the present invention provides a monitoring device for a railway system, comprising:

[0031] Acquisition module, used to acquire sensor data and real-time monitoring video in the railway system;

[0032] A key event extraction module, used to extract key events from the real-time monitoring video based on the sensor data and the real-time monitoring video;

[0033] A sub-image extraction module, used to extract a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video;

[0034] A background setting module, configured to set objects and / or regions other than the main object in the real video sub-image as background;

[0035] A virtual module, used for modeling the background based on building information modeling technology to obtain a virtual background after virtualization processing;

[0036] The display module is used to synthesize the real video sub-image into the virtual background to obtain the target monitoring video and display it.

[0037] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are performed.

[0038] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, runs the steps of the method provided in the first aspect.

[0039] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, runs the steps of the method provided in the first aspect.

[0040] It can be seen from the above technical solutions that the present invention has the following advantages:

[0041] The present invention provides a monitoring method and device for a railway system, the method comprising: obtaining sensor data and real-time monitoring video in the railway system; extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video; extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video; setting objects and / or areas other than the subject object in the real video sub-image as background; modeling the background based on building information modeling technology to obtain a virtual background after virtualization processing; synthesizing the real video sub-image into the virtual background to obtain a target monitoring video and display it. By using real-time monitoring video to display the subject object, and virtualizing the range other than the subject object in the key event, and calling the two together as target monitoring videos for display, the subject object involved in the key event can be highlighted, so that monitoring personnel can quickly identify key information from the target monitoring video. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0043] Figure 1 A flowchart of a first embodiment of a monitoring method for a railway system according to the present invention;

[0044] Figure 2 A flowchart of a second embodiment of a monitoring method for a railway system according to the present invention;

[0045] Figure 3 The figure is a structural block diagram of an embodiment of a monitoring device for a railway system according to the present invention. DETAILED DESCRIPTION

[0046] The embodiment of the present invention provides a monitoring method and device for a railway system, which is used to display monitoring information of the railway system in real time by combining virtual and real methods, so that key information can be displayed intuitively.

[0047] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Embodiment 1

[0049] See also Figure 1 , Figure 1 This is a flow chart of a first embodiment of a monitoring method for a railway system of the present invention, the method comprising:

[0050] Step S101, obtaining sensor data and real-time monitoring video in the railway system;

[0051] In the embodiment of the present invention, the sensor data is data obtained from various sensors in the railway system (such as temperature sensors, pressure sensors, speed sensors, and position sensors, etc.). These data are usually used to monitor the operating status of the railway system, such as the position, speed, and track condition of the train.

[0052] Real-time monitoring data is the video stream obtained by cameras installed in the railway system. These cameras are used to monitor the operation of the railway system in real time, including the status of trains, changes in signal lights, and the status of tracks.

[0053] Step S102, extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video;

[0054] In an embodiment of the present invention, key events such as abnormal train operation, signal light failure, track problems, etc. are identified and extracted from real-time monitoring videos through event monitoring algorithms or artificial intelligence methods.

[0055] Step S103, extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video;

[0056] In the embodiment of the present invention, the real video sub-image data is the image data or frame data of the main object only related to the key event.

[0057] Step S104, setting objects and / or regions other than the main object in the real video sub-image as background;

[0058] In the embodiment of the present invention, by processing the extracted real video sub-image, other objects and regions except the main object are set as the background, so that the monitoring personnel can focus on the main object of the key event.

[0059] Step S105, modeling the background based on building information modeling technology to obtain a virtual background after virtualization processing;

[0060] The embodiment of the present invention performs three-dimensional modeling of the background through BIM technology, so that the background better meets the actual virtualization requirements.

[0061] Step S106, synthesizing the real video sub-image into the virtual background to obtain the target monitoring video and display it.

[0062] The embodiment of the present invention synthesizes the real video sub-image extracted in step S103 with the virtual background in step S105 to generate a target monitoring video that can seamlessly integrate the virtual background and the main object, and uses it in a real-time monitoring system to allow monitoring personnel to clearly see the main object of the key event, so that monitoring personnel can quickly identify key information from the target monitoring video.

[0063] A monitoring method for a railway system provided by an embodiment of the present invention includes: obtaining sensor data and real-time monitoring video in the railway system; extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video; extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video; setting objects and / or areas other than the subject object in the real video sub-image as background; modeling the background based on building information modeling technology to obtain a virtual background after virtualization processing; synthesizing the real video sub-image into the virtual background to obtain a target monitoring video and display it. By using real-time monitoring video to display the subject object, and virtualizing the range other than the subject object in the key event, and calling the two together as target monitoring videos for display, the subject object involved in the key event can be highlighted, so that monitoring personnel can quickly identify key information from the target monitoring video.

[0064] Embodiment 2

[0065] See also Figure 2 , Figure 2 This is a flow chart of a second embodiment of a monitoring method for a railway system of the present invention, the steps comprising:

[0066] Step S201, obtaining sensor data and real-time monitoring video in the railway system;

[0067] Step S202, extracting sensor characteristic data including train speed, train acceleration and signal change from the sensor data;

[0068] The embodiment of the present invention extracts sensor feature data including train speed, train acceleration and signal changes to facilitate subsequent key event identification.

[0069] Step S203, extracting video feature data including train motion trajectory, vehicle position and pedestrian position from the real-time monitoring video;

[0070] The embodiment of the present invention extracts video feature data from real-time monitoring videos, including the movement trajectory of trains, the positions of vehicles, and the positions of pedestrians, etc., so as to facilitate subsequent key event identification.

[0071] Step S204, inputting the sensor feature data and the video feature data into a pre-trained key event recognition network to obtain the key event;

[0072] The embodiment of the present invention obtains corresponding key event prediction results by inputting sensor feature data and video feature data into a trained key event recognition network.

[0073] Step S205, based on the event type of the key event, the key event is checked by using a preset threshold value and / or preset rule of the corresponding type;

[0074] In order to ensure the accuracy of the key event prediction results, the embodiments of the present invention set different preset thresholds and / or preset rules according to the event types of different key events. For example, when the prediction result obtained by the key event recognition network is train speeding, the real-time speed of the train will be compared with the preset speed threshold to verify the authenticity of the prediction result; when the prediction result obtained by the key event recognition network is abnormal signal status, the signal status of the train will be matched with the signal status fault type in the preset rules to verify the authenticity of the prediction result.

[0075] Step S206, forming an event view corresponding to the key event based on the building information modeling technology;

[0076] In an optional embodiment, based on the building information model technology, an event view corresponding to the key event is formed, including:

[0077] Based on the building information modeling technology, determining the locations of all the objects;

[0078] Performing spatial analysis on the objects respectively and identifying the relationships between the objects to obtain spatial analysis results and relationship integration results;

[0079] The spatial analysis results and the relationship integration results are integrated to form an event view corresponding to the key time.

[0080] In the embodiment of the present invention, a spatial analysis is performed on the virtual model obtained by the building information modeling technology to determine the objects and scope involved in the key events and detect possible abnormal areas. Then, a relationship analysis is performed on the virtual model to determine the relationship between the objects. Finally, the spatial analysis results and the relationship integration results obtained by the analysis are integrated to form a comprehensive event view.

[0081] Step S207, determining the subject object from the event view;

[0082] Step S208, extracting the real video sub-image of the subject object from the real-time monitoring video;

[0083] Step S209, setting objects and / or regions other than the main object in the real video sub-image as background;

[0084] Step S210, based on the event type of the critical event, determine the corresponding urgency through the preset threshold and / or the preset rule of the corresponding type;

[0085] The embodiment of the present invention sets different virtualization degrees according to the urgency of the key event. If the urgency is average, the background blurring degree is low; if the urgency is high, the background blurring degree is high.

[0086] Step S211, modeling the background based on the building information modeling technology, and performing fuzzy processing on the modeling model according to the urgency level through the graphic processing technology to obtain the virtual background;

[0087] In the embodiment of the present invention, when blurring the background, the background is made less noticeable by using effects such as Gaussian blurring or pixelation.

[0088] Step S212, synthesizing the real video sub-image into the virtual background to obtain the target monitoring video and display it.

[0089] A monitoring method for a railway system disclosed in an embodiment of the present invention includes: obtaining sensor data and real-time monitoring video in the railway system; extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video; extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video; setting objects and / or areas other than the subject object in the real video sub-image as background; modeling the background based on building information modeling technology to obtain a virtual background after virtualization processing; synthesizing the real video sub-image into the virtual background to obtain a target monitoring video and display it. By using real-time monitoring video to display the subject object, virtualizing the range other than the subject object in the key event, and performing different degrees of blurring according to the urgency of the key event, and then calling the two together as a target monitoring video for display, the subject object involved in the key event can be highlighted, so that the monitoring personnel can quickly identify key information from the target monitoring video.

[0090] Embodiment 3

[0091] See also Figure 3 , Figure 3 This is a structural block diagram of an embodiment of a monitoring device for a railway system of the present invention, the device comprising:

[0092] An acquisition module 301 is used to acquire sensor data and real-time monitoring video in the railway system;

[0093] A key event extraction module 302, for extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video;

[0094] The sub-image extraction module 303 is used to extract the real video sub-image of the subject object corresponding to the key event from the real-time monitoring video;

[0095] A background setting module 304, configured to set objects and / or regions other than the main object in the real video sub-image as background;

[0096] A virtual module 305 is used to model the background based on building information modeling technology to obtain a virtual background after virtualization processing;

[0097] The display module 306 is used to synthesize the real video sub-image into the virtual background to obtain the target monitoring video and display it.

[0098] In an optional embodiment, the key event extraction module 302 includes:

[0099] A sensor feature data extraction submodule, used to extract sensor feature data including train speed, train acceleration and signal change from the sensor data;

[0100] A video feature data extraction submodule is used to extract video feature data including train motion trajectory, vehicle position and pedestrian position from the real-time monitoring video;

[0101] The key event acquisition submodule is used to input the sensor feature data and the video feature data into a pre-trained key event recognition network to obtain the key event.

[0102] In an optional embodiment, the key event extraction module 302 further includes:

[0103] The verification submodule is used to verify the key event based on the event type of the key event through a preset threshold value and / or preset rule of the corresponding type.

[0104] In an optional embodiment, the sub-image extraction module 303 includes:

[0105] An event view forming submodule, used for forming an event view corresponding to the key event based on the building information modeling technology;

[0106] A subject object determination submodule, used to determine the subject object from the event view;

[0107] The real video sub-image extraction sub-module is used to extract the real video sub-image of the main object from the real-time monitoring video.

[0108] In an optional embodiment, the event view forming submodule includes:

[0109] A position determination unit, used for determining the positions of all the objects based on the building information modeling technology;

[0110] An analysis unit, used to perform spatial analysis on the objects respectively and identify the relationships between the objects, to obtain spatial analysis results and relationship integration results;

[0111] The integration unit is used to integrate the spatial analysis results and the relationship integration results to form an event view corresponding to the key time.

[0112] In an optional embodiment, the virtual module 305 includes:

[0113] An urgency determination submodule, configured to determine a corresponding urgency based on the event type of the critical event by using the preset threshold value and / or the preset rule of the corresponding type;

[0114] The fuzzy processing submodule is used to model the background based on the building information modeling technology, and to perform fuzzy processing on the model obtained by modeling according to the urgency through the graphic processing technology to obtain the virtual background.

[0115] Embodiment 4

[0116] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of a monitoring method for a railway system in any embodiment, including:

[0117] Obtain sensor data and real-time monitoring video in railway systems;

[0118] Extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video;

[0119] Extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video;

[0120] In the real video sub-image, objects and / or regions other than the main object are set as background;

[0121] Modeling the background based on building information modeling technology to obtain a virtual background after virtualization processing;

[0122] The real video sub-image is synthesized into the virtual background to obtain the target monitoring video and display it.

[0123] In an optional embodiment, extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video includes:

[0124] extracting sensor characteristic data including train speed, train acceleration and signal change from the sensor data;

[0125] Extracting video feature data including train movement trajectory, vehicle position and pedestrian position from the real-time monitoring video;

[0126] The sensor feature data and the video feature data are input into a pre-trained key event recognition network to obtain the key events.

[0127] In an optional embodiment, the sensor feature data and the video feature data are input into a pre-trained key event recognition network to obtain the key event, and the method further includes:

[0128] Based on the event type of the key event, the key event is checked by using a preset threshold and / or preset rule of a corresponding type.

[0129] In an optional embodiment, extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video includes:

[0130] Based on the building information model technology, an event view corresponding to the key event is formed;

[0131] Determining the subject object from the event view;

[0132] The real video sub-image of the main object is extracted from the real-time monitoring video.

[0133] In an optional embodiment, based on the building information model technology, an event view corresponding to the key event is formed, including:

[0134] Based on the building information modeling technology, determining the locations of all the objects;

[0135] Performing spatial analysis on the objects respectively and identifying the relationships between the objects to obtain spatial analysis results and relationship integration results;

[0136] The spatial analysis results and the relationship integration results are integrated to form an event view corresponding to the key time.

[0137] In an optional embodiment, the background is modeled based on building information modeling technology to obtain a virtual background after virtualization processing, including:

[0138] Based on the event type of the critical event, determine the corresponding urgency through the preset threshold and / or the preset rule of the corresponding type;

[0139] The background is modeled based on the building information modeling technology, and the model obtained by modeling is fuzzy processed according to the urgency through graphic processing technology to obtain the virtual background.

[0140] Embodiment 5

[0141] An embodiment of the present invention further provides a computer storage medium having a computer program stored thereon. When the computer program is executed by the processor, the steps of implementing a method for monitoring a railway system in any embodiment include:

[0142] Obtain sensor data and real-time monitoring video in railway systems;

[0143] Extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video;

[0144] Extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video;

[0145] In the real video sub-image, objects and / or regions other than the main object are set as background;

[0146] Modeling the background based on building information modeling technology to obtain a virtual background after virtualization processing;

[0147] The real video sub-image is synthesized into the virtual background to obtain the target monitoring video and display it.

[0148] In an optional embodiment, extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video includes:

[0149] extracting sensor characteristic data including train speed, train acceleration and signal change from the sensor data;

[0150] Extracting video feature data including train movement trajectory, vehicle position and pedestrian position from the real-time monitoring video;

[0151] The sensor feature data and the video feature data are input into a pre-trained key event recognition network to obtain the key events.

[0152] In an optional embodiment, the sensor feature data and the video feature data are input into a pre-trained key event recognition network to obtain the key event, and the method further includes:

[0153] Based on the event type of the key event, the key event is checked by using a preset threshold and / or preset rule of a corresponding type.

[0154] In an optional embodiment, extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video includes:

[0155] Based on the building information model technology, an event view corresponding to the key event is formed;

[0156] Determining the subject object from the event view;

[0157] The real video sub-image of the main object is extracted from the real-time monitoring video.

[0158] In an optional embodiment, based on the building information model technology, an event view corresponding to the key event is formed, including:

[0159] Based on the building information modeling technology, determining the locations of all the objects;

[0160] Performing spatial analysis on the objects respectively and identifying the relationships between the objects to obtain spatial analysis results and relationship integration results;

[0161] The spatial analysis results and the relationship integration results are integrated to form an event view corresponding to the key time.

[0162] In an optional embodiment, the background is modeled based on building information modeling technology to obtain a virtual background after virtualization processing, including:

[0163] Based on the event type of the critical event, determine the corresponding urgency through the preset threshold and / or the preset rule of the corresponding type;

[0164] The background is modeled based on the building information modeling technology, and the model obtained by modeling is fuzzy processed according to the urgency through graphic processing technology to obtain the virtual background.

[0165] Embodiment 6

[0166] An embodiment of the present invention further provides a computer program product, on which a computer program is stored. When the computer program is executed by the processor, the steps of implementing a method for monitoring a railway system in any embodiment include:

[0167] Obtain sensor data and real-time monitoring video in railway systems;

[0168] Extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video;

[0169] Extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video;

[0170] In the real video sub-image, objects and / or regions other than the main object are set as background;

[0171] Modeling the background based on building information modeling technology to obtain a virtual background after virtualization processing;

[0172] The real video sub-image is synthesized into the virtual background to obtain the target monitoring video and display it.

[0173] In an optional embodiment, extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video includes:

[0174] extracting sensor characteristic data including train speed, train acceleration and signal change from the sensor data;

[0175] Extracting video feature data including train movement trajectory, vehicle position and pedestrian position from the real-time monitoring video;

[0176] The sensor feature data and the video feature data are input into a pre-trained key event recognition network to obtain the key events.

[0177] In an optional embodiment, the sensor feature data and the video feature data are input into a pre-trained key event recognition network to obtain the key event, and the method further includes:

[0178] Based on the event type of the key event, the key event is checked by using a preset threshold and / or preset rule of a corresponding type.

[0179] In an optional embodiment, extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video includes:

[0180] Based on the building information model technology, an event view corresponding to the key event is formed;

[0181] Determining the subject object from the event view;

[0182] The real video sub-image of the main object is extracted from the real-time monitoring video.

[0183] In an optional embodiment, based on the building information model technology, an event view corresponding to the key event is formed, including:

[0184] Based on the building information modeling technology, determining the locations of all the objects;

[0185] Performing spatial analysis on the objects respectively and identifying the relationships between the objects to obtain spatial analysis results and relationship integration results;

[0186] The spatial analysis results and the relationship integration results are integrated to form an event view corresponding to the key time.

[0187] In an optional embodiment, the background is modeled based on building information modeling technology to obtain a virtual background after virtualization processing, including:

[0188] Based on the event type of the critical event, determine the corresponding urgency through the preset threshold and / or the preset rule of the corresponding type;

[0189] The background is modeled based on the building information modeling technology, and the model obtained by modeling is fuzzy processed according to the urgency through graphic processing technology to obtain the virtual background.

[0190] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0191] In the several embodiments provided in the present application, it should be understood that the methods, devices, electronic devices and storage media disclosed in the present invention can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0192] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a readable storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0195] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A monitoring method for a railway system, characterized in that: include: Obtain sensor data and real-time monitoring video in railway systems; Extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video; Extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video; In the real video sub-image, objects and / or regions other than the main object are set as background; Modeling the background based on building information modeling technology to obtain a virtual background after virtualization processing; The real video sub-image is synthesized into the virtual background to obtain the target monitoring video and display it.

2. The monitoring method of a railway system according to claim 1, characterized in that: Extracting key events from the real-time monitoring video based on the sensor data and the real-time monitoring video includes: extracting sensor characteristic data including train speed, train acceleration and signal change from the sensor data; Extracting video feature data including train movement trajectory, vehicle position and pedestrian position from the real-time monitoring video; The sensor feature data and the video feature data are input into a pre-trained key event recognition network to obtain the key events.

3. The method for monitoring a railway system according to claim 2, characterized in that: After inputting the sensor feature data and the video feature data into a pre-trained key event recognition network to obtain the key event, the method further includes: Based on the event type of the key event, the key event is checked by using a preset threshold and / or preset rule of a corresponding type.

4. The monitoring method of a railway system according to claim 1, characterized in that: Extracting a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video includes: Based on the building information model technology, an event view corresponding to the key event is formed; Determining the subject object from the event view; The real video sub-image of the main object is extracted from the real-time monitoring video.

5. The method for monitoring a railway system according to claim 3, characterized in that: Based on the building information model technology, an event view corresponding to the key event is formed, including: Based on the building information modeling technology, determining the locations of all the objects; Performing spatial analysis on the objects respectively and identifying the relationships between the objects to obtain spatial analysis results and relationship integration results; The spatial analysis results and the relationship integration results are integrated to form an event view corresponding to the key time.

6. The method for monitoring a railway system according to claim 3, characterized in that: The background is modeled based on the building information modeling technology to obtain a virtual background after virtualization processing, including: Based on the event type of the critical event, determine the corresponding urgency through the preset threshold and / or the preset rule of the corresponding type; The background is modeled based on the building information modeling technology, and the model obtained by modeling is fuzzy processed according to the urgency through graphic processing technology to obtain the virtual background.

7. A monitoring device for a railway system, characterized in that: include: An acquisition module is used to obtain sensor data and real-time monitoring videos in the railway system; A key event extraction module, used to extract key events from the real-time monitoring video based on the sensor data and the real-time monitoring video; A sub-image extraction module, used to extract a real video sub-image of a subject object corresponding to the key event from the real-time monitoring video; A background setting module, configured to set objects and / or regions other than the main object in the real video sub-image as background; A virtual module, used for modeling the background based on building information modeling technology to obtain a virtual background after virtualization processing; The display module is used to synthesize the real video sub-image into the virtual background to obtain the target monitoring video and display it.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is executed.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is executed.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images