Intelligent track operation and maintenance method and system based on data twinning technology and medium

Abstract

The invention discloses an intelligent track operation and maintenance method and system based on a data twinning technology and a medium, and belongs to the technical field of data processing.The method comprises the steps that a basic map is established, a plurality of monitoring cameras are used for shooting a plurality of equipment images of the same station equipment, and an equipment model is generated based on the equipment images; based on the position coordinates of the monitoring camera, the equipment model is placed in a basic map, the prediction model is used for receiving equipment operation data, and the operation state of station equipment is marked; if the station equipment is in the abnormal state, the judgment model receives the equipment operation data and judges whether the station equipment is in a fault state or not, and if yes, the operation state of the station equipment is changed into the fault state; in the basic map, marking the equipment models in the normal state, the abnormal state and the fault state as a first color, a second color and a third color respectively; according to the invention, the positions of various abnormal devices in the station can be visually displayed, so that operation and maintenance personnel can be helped to quickly process faults.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and specifically relates to a rail intelligent operation and maintenance method, system and medium based on data twin technology. Background Art

[0002] As a subway subsystem, the AFC system plays an important role in subway operation and management. The traditional maintenance mode is based on post-fault maintenance, that is, after a failure occurs in the equipment, maintenance personnel need to conduct on-site inspections and repairs. The low operation and maintenance efficiency seriously affects station operations. Therefore, it is increasingly important to realize the visualization and informatization of the health of AFC system equipment and components, rapid fault diagnosis, and early status maintenance.

[0003] To solve the above problems, the prior art has proposed the following solutions. For example, Chinese patent document CN116468419A discloses a rail transit big data operation and maintenance decision analysis method, device and storage medium. The method includes a perception layer, which contains a rail transit related basic database. The data acquisition layer obtains fault alarm data and analog data from the perception layer and transmits them to the data management center. The health evaluation model inside the data management center performs real-time health diagnosis. When the health index of the equipment reaches the health index maintenance threshold, the data management center sends the equipment maintenance and disposal suggestion work order to the PMS system. In this way, the warning time can be accurately predicted, which is convenient for guiding the rail transit. Condition-based maintenance of transportation equipment; for example, Chinese patent document CN109299827A discloses a method for predicting urban rail equipment faults based on long short-term memory recurrent neural networks. This method classifies equipment faults, analyzes fault mechanisms, analyzes factors affecting fault characteristics, selects quantifiable factors as model inputs, and then preprocesses the input data. Finally, a VPSO-LSTM equipment fault prediction model is established and the fault prediction performance is tested and analyzed, thereby obtaining a method for predicting urban rail equipment faults, which can predict the possibility of equipment failure after a period of time, and can also analyze the impact of influencing factors on the trouble-free operation time of the equipment.

[0004] However, after determining that the equipment is abnormal, the above method can only indicate the location of the faulty equipment through text. For employees who are inexperienced or unfamiliar with the station, they may not be able to quickly find the location of the faulty equipment based on the text description, which will affect the efficiency of equipment maintenance. Summary of the invention

[0005] To solve the above problems, the present invention provides a rail intelligent operation and maintenance method, system and medium based on data twin technology to realize the visual display of the location of abnormal status equipment.

[0006] In order to achieve the above-mentioned purpose of the invention, the present invention proposes a track intelligent operation and maintenance method based on data twin technology, comprising:

[0007] Establishing a basic map, and locating the position coordinates of the surveillance camera in the basic map;

[0008] Use multiple surveillance cameras to capture multiple device images of the same station device, and establish a generation model and a detection model, wherein the generation model generates a device model based on the device images, and the detection model checks the device model and obtains its integrity;

[0009] If the degree of integrity is less than a first threshold, a reconstruction instruction is generated, and the device model is reconstructed by the reconstruction instruction so that the degree of integrity of the device model is greater than or equal to the first threshold;

[0010] The device model is placed in the basic map based on the location coordinates of the surveillance camera, a device code is assigned to each device model in the basic map, device operation data corresponding to each device code is collected, and a prediction model and a judgment model are established. The prediction model receives the device operation data and marks the operation status of the station equipment;

[0011] The operation state includes a normal state and an abnormal state. If it is the abnormal state, the equipment operation data continues to be input into the judgment model. The judgment model receives the equipment operation data and judges whether the station equipment is in a fault state. If it is, the operation state of the station equipment is changed to the fault state. If not, the operation state is kept as the abnormal state.

[0012] In the basic map, the equipment models in the normal state, the abnormal state and the faulty state are marked as the first color, the second color and the third color respectively, a fault work order is automatically established for the station equipment in the faulty state, and recommended processing measures are generated, and an operation and maintenance plan is automatically generated for the station equipment in the abnormal state and pushed.

[0013] Furthermore, the prediction model determines the operating status of the device including the following steps:

[0014] Setting a first time period and a second time period, wherein the first time period is earlier than the second time period, collecting a plurality of pieces of operation data of the station equipment in the first time period and marking them as normal data, and collecting a plurality of pieces of operation data in the second time period and marking them as abnormal data;

[0015] The normal data and the abnormal data are mixed to generate a first data group and a second data group, and after the prediction model is trained using the first data group, the second data group is input into the prediction model, and the prediction model outputs a first probability that the normal data in the second data group is the abnormal data, and a second probability that the abnormal data is the abnormal data;

[0016] N pieces of the normal data and the abnormal data are extracted from the second data group, and the evaluation value α is calculated based on the first formula, where the first formula is: Wherein, N is the number of abnormal data in the second data group, The first probability of extracting the nth normal data is: To extract the second probability of the nth abnormal data, if the evaluation value is greater than a second threshold, the device is output as the abnormal state, otherwise it is output as the normal state.

[0017] Furthermore, after obtaining the evaluation value, the following steps are also included:

[0018] After the prediction model outputs the evaluation value that is less than the judgment threshold for multiple consecutive times, the change trend of the evaluation value is calculated based on the multiple evaluation values. If the change trend is an increasing trend, the time point when the station equipment enters the abnormal state is predicted based on the increasing trend.

[0019] Furthermore, the first time period and the second time period are both preset durations, and the starting point of the first time period is the time point after the device is put into use, and the end point of the second time period is the current time point.

[0020] Furthermore, generating the device model comprises the following steps:

[0021] The generation model extracts feature points from each of the device images, matches the feature points in different device images, generates a model depth map of each of the device images based on the feature points, the camera postures and camera parameters of each camera, and integrates the model depth maps to generate a point cloud model;

[0022] Select a feature point in the point cloud model and define it as a first base point. Obtain the distance between the first base point and other feature points, select two feature points closest to the first base point and define them as second base points. Connect the first base point and the second base point to obtain a triangular mesh. Repeat this step until all feature points are connected to obtain a basic model.

[0023] Calculate the first area of ​​each triangular mesh in the basic model, set a third threshold, obtain the triangular mesh whose first area is greater than the third threshold, define it as a missing mesh, draw a line segment passing through the center of gravity in the missing mesh, split the missing mesh into polygons, repeat this step until all the missing meshes are split, and obtain the device model.

[0024] Further, determining the completeness of the device model includes the following steps:

[0025] A first length is set, a second length of each edge in each triangular mesh of the device model is identified, an edge whose second length is greater than the first length is defined as a target edge, the triangular mesh containing the target edge is obtained, defined as a rejection target, a second area of ​​the rejection target is obtained, and a first average value μ is calculated based on a second formula, where the second formula is: Among them, S 1 is the sum of the first areas, S 2 is the sum of the second areas, n is the number of the remaining triangular meshes after removing the culling targets;

[0026] The ratio of the first area of ​​the remaining triangular mesh to the first average value after removing the elimination target is calculated, the area of ​​the triangular mesh where the ratio is greater than a fourth threshold is defined as a third area, and the integrity δ of the device model is calculated based on a third formula, wherein the third formula is: Among them, S 3 is the third area.

[0027] Furthermore, establishing the judgment model includes the following steps:

[0028] The judgment model is a neural network model, which collects historical data of each station equipment in the past as a training set, wherein the historical data includes the equipment operation data of the station equipment in the normal state and the fault state. The neural network model is trained using the training set, and the trained neural network model is used as the judgment model.

[0029] Further, placing the device model in the base map comprises the following steps:

[0030] Each monitoring camera obtains the distance between itself and the station equipment, locates a target point between various monitoring cameras photographing the same station equipment based on the distance, and places the equipment model at the position of the target point.

[0031] The present invention also provides a track intelligent operation and maintenance system based on data twin technology, which is used to implement the above-mentioned track intelligent operation and maintenance method based on data twin technology, and the system includes:

[0032] a model generation module, used to establish a basic map, locate the position coordinates of the surveillance camera in the basic map, use multiple surveillance cameras to capture multiple device images of the same station device, establish a generation model and a detection model, wherein the generation model generates a device model based on the device image, and the detection model checks the device model and obtains its integrity. If the integrity is less than a first threshold, a reconstruction instruction is generated, and the device model is reconstructed by the reconstruction instruction so that the integrity of the device model is greater than or equal to the first threshold, and the device model is placed in the basic map based on the position coordinates of the surveillance camera;

[0033] A data collection module, used for assigning a device code to each device model in the basic map, and collecting device operation data corresponding to each device code;

[0034] A state judgment module is used to establish a prediction model and a judgment model. The prediction model receives the equipment operation data and marks the operation state of the station equipment. The operation state includes a normal state and an abnormal state. If it is the abnormal state, the equipment operation data is continuously input into the judgment model. The judgment model receives the equipment operation data and judges whether the station equipment is in a fault state. If it is, the operation state of the station equipment is changed to the fault state. If not, the operation state is retained as the abnormal state.

[0035] The equipment maintenance module is used to mark the equipment models in the normal state, the abnormal state and the faulty state in the basic map as the first color, the second color and the third color respectively, automatically establish a fault work order for the station equipment in the faulty state, generate recommended processing measures, and automatically generate an operation and maintenance plan for the station equipment in the abnormal state and push it.

[0036] The present invention also discloses a computer storage medium, wherein the computer storage medium stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the above-mentioned method.

[0037] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0038] The present invention first establishes a three-dimensional basic map, and then creates some three-dimensional equipment models by taking photos of some station equipment through monitoring cameras, which can reduce the workload of creating real-scene three-dimensional models; then, the completeness of the generated equipment model is confirmed by setting a detection model, and the incomplete equipment model is rebuilt, so that the equipment models existing in the basic map have a high degree of completeness. Finally, each equipment model is placed in the correct position, and data is collected according to the coding of each equipment model, so as to realize the creation of a data twin model.

[0039] After the data twin model is created, the present invention also establishes a prediction model and a judgment model, and then uses the prediction model to determine whether the station equipment is in a normal state, or in an abnormal state where it is about to fail or has already failed. If it is an abnormal state, it is handed over to the judgment model for judgment. The judgment model can ultimately determine whether the station equipment has failed and the type of failure; finally, according to the determined station equipment status, the corresponding color is marked in the data twin model to remind relevant personnel. In addition, the present invention can provide an operation and maintenance plan for equipment in abnormal status, thereby laying a foundation for realizing railway status repair. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flowchart of the steps of intelligent rail operation and maintenance based on data twin technology of the present invention;

[0041] Figure 2 A schematic diagram of the principle of determining a target point according to the present invention;

[0042] Figure 3 This is a structural schematic diagram of a rail intelligent operation and maintenance system based on data twin technology in the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0044] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0045] like Figure 1 As shown in the figure, a rail intelligent operation and maintenance method based on data twin technology includes:

[0046] Step S1: Create a basic map and locate the location coordinates of the surveillance camera in the basic map.

[0047] Step S2: Use multiple surveillance cameras to capture multiple device images of the same station device, and establish a generation model and a detection model. The generation model generates a device model based on the device image, and the detection model checks the device model and obtains its integrity.

[0048] Specifically, the basic map is a three-dimensional map pre-drawn through three-dimensional modeling technology, which includes the distribution of all surveillance cameras in each room, and has placed some pre-made station equipment models; because the basic map is made according to the real scene, it can reflect the actual scene more realistically, but some equipment can be automatically modeled through surveillance cameras, such as entry and exit gates, ticket machines and other equipment in the station. There are many cameras installed here and there are basically no blind spots. Therefore, creating a three-dimensional model by taking photos with surveillance cameras can reduce the manual workload of creating data twin scenes in the early stage.

[0049] Specifically, firstly, the equipment images of the same station equipment, such as the gate, are captured by surveillance cameras at different positions, and the equipment images are analyzed to generate a three-dimensional point cloud model of the gate, and then the equipment model of the gate is generated by the three-dimensional point cloud model. The specific method of generating the equipment model will be introduced later. In addition, after the equipment model is generated, the present invention also sets a detection model to determine the completeness of the produced equipment model, so as to avoid the situation where the generated equipment model has structural defects, resulting in a poor appearance of the model.

[0050] Step S3: If the completeness is less than the first threshold, a reconstruction instruction is generated, and the device model is rebuilt through the reconstruction instruction so that the completeness of the device model is greater than or equal to the first threshold.

[0051] The first threshold is, for example, 90%. If the completeness calculated above is greater than 90%, the device model is considered to be complete. At this time, the creation process of the device model is completed. If the completeness is less than or equal to 90%, the device model is rebuilt so that the completeness of the device model is greater than 90%. The specific reconstruction method will be introduced in detail later.

[0052] Step S4: Place the equipment model in the basic map based on the location coordinates of the surveillance camera, assign a device code to each equipment model in the basic map, collect the equipment operation data corresponding to each equipment code, establish a prediction model and a judgment model, and the prediction model receives the equipment operation data and marks the operation status of the station equipment.

[0053] Step S5: The operating status includes normal status and abnormal status. If it is an abnormal status, the equipment operating data continues to be input into the judgment model. The judgment model receives the equipment operating data and determines whether the station equipment is in a fault state. If so, the operating status of the station equipment is changed to a fault state. If not, the operating status is retained as an abnormal state.

[0054] Here, the placement of the device model in the basic map is determined based on the location coordinates of the surveillance camera. The specific determination method will be introduced later. After completing the placement of all device models in the basic map, encode each device model in each basic map, so that the subsequent collected operation data corresponds to the encoding, that is, the collected device data corresponds to the device model. The operation data of this embodiment includes whole machine status data, module status data, model operation data, transaction data, energy consumption data, network data, software operation data, system operation data, etc. The operation status of each device is determined by establishing a prediction model.

[0055] In the prediction model, the operation data of the equipment is used to determine whether the station equipment is in a normal state or an abnormal state. The abnormal state means that the station equipment is in a state where it may fail, or the station equipment is already in a faulty state. The determined operation state is then bound to the equipment code; after the prediction model determines that the station equipment is already in an abnormal state, the operation data of the station equipment is input into the judgment model again, and the judgment model confirms whether the equipment is already in a faulty state. If it is already in a faulty state, the station equipment is changed from the previously marked abnormal state to a faulty state.

[0056] Step S6: In the basic map, the equipment models in normal state, abnormal state and fault state are marked as the first color, the second color and the third color respectively. A fault work order is automatically created for the station equipment in the fault state, and recommended processing measures are generated. An operation and maintenance plan is automatically generated for the station equipment in the abnormal state and pushed.

[0057] Here, the first color is white, the second color is yellow, and the third color is red. The station equipment in abnormal state is marked in yellow, so that the operation and maintenance personnel can pay attention to it in time when viewing the basic map. The station equipment in fault state is marked in red, making the station equipment more conspicuous in the basic map, which is convenient for relevant personnel to quickly locate it. In addition, since the position of the equipment model in the basic map is the same as the actual scene, when the equipment is abnormal, the relevant personnel can quickly find the location of the equipment according to the basic map, so those who are not familiar with the station environment can also quickly find the location of the faulty equipment.

[0058] In particular, when the present invention detects a fault in the station equipment, it will automatically generate a fault work order and retrieve appropriate treatment measures from the database, thereby helping the operation and maintenance personnel to quickly troubleshoot the fault. For example, when the operation and maintenance personnel log in to the monitoring platform and click on the alarm information, they can view detailed information such as the alarm-related equipment and applications, as well as the preliminary diagnosis and recommended treatment steps of such faults. When the operator successfully solves the fault, the fault alarm disappears, and the fault is recorded in the database for analysis to form an expert knowledge base. In addition, when the station equipment is detected to be in an abnormal state, the state change of the equipment can also be analyzed, and the trend of the equipment status can be dynamically analyzed to achieve the transition from fault repair to status repair.

[0059] The present invention first establishes a three-dimensional basic map, and then creates some three-dimensional equipment models by taking photos of some station equipment through monitoring cameras, which can reduce the workload of creating real-scene three-dimensional models; then, the completeness of the generated equipment model is confirmed by setting a detection model, and the incomplete equipment model is rebuilt, so that the equipment models existing in the basic map have a high degree of completeness. Finally, each equipment model is placed in the correct position, and data is collected according to the coding of each equipment model, so as to realize the creation of a data twin model.

[0060] After the data twin model is created, the present invention also establishes a prediction model and a judgment model, and then uses the prediction model to determine whether the station equipment is in a normal state, or in an abnormal state where it is about to fail or has already failed. If it is an abnormal state, it is handed over to the judgment model for judgment. The judgment model can ultimately determine whether the station equipment has failed and the type of failure; finally, according to the determined station equipment status, the corresponding color is marked in the data twin model to remind relevant personnel. In addition, the present invention can provide an operation and maintenance plan for equipment in abnormal status, thereby laying a foundation for realizing railway status repair.

[0061] It is particularly noteworthy that the present invention can highly visualize the locations of various abnormal equipment in the station, which can help operation and maintenance personnel to quickly handle faults.

[0062] In this embodiment, the prediction model determines the operating status of the device including the following steps:

[0063] A first time period and a second time period are set, the first time period is earlier than the second time period, multiple operation data of the station equipment in the first time period are collected and marked as normal data, and multiple operation data in the second time period are collected and marked as abnormal data.

[0064] Specifically, the duration of the first time period and the second time period are both set to 5 days. The first time period starts at the time when the station equipment is put into use. Because when the equipment is initially put into use, the station equipment is in normal operation, and the collected operation data is generally normal. Therefore, the data at this time is marked as normal data; the second time period ends at the current time point. Because it is necessary to determine the current status of the station equipment, the data in the second time period are uniformly marked as abnormal data, and the reason will be explained later.

[0065] Normal data and abnormal data are mixed to generate a first data group and a second data group. After the prediction model is trained using the first data group, the second data group is input into the prediction model, and the prediction model outputs a first probability that the normal data in the second data group is abnormal data and a second probability that the abnormal data is abnormal data.

[0066] Assume that there are 100 pieces of normal data and abnormal data at present, and mix these 100 pieces of data to generate a first data group and a second data group. There are 100 pieces of data in each of the first data group and the second data group, and these 100 pieces of data are mixed with normal data and abnormal data. This embodiment uses a BP neural network model to establish a prediction model. In other embodiments, a hidden Markov model, a random forest model, etc. can also be used. By inputting the marked normal data and abnormal data into the prediction model for training, the trained prediction model can output a first probability that the normal data is abnormal data and a second probability that the abnormal data is abnormal data.

[0067] N normal data and abnormal data are extracted from the second data group, and the evaluation value α is calculated based on the first formula. The first formula is: Wherein, N is the number of abnormal data in the second data group, is the first probability of extracting the nth normal data, To extract the second probability of the nth abnormal data, if the evaluation value is greater than the second threshold, the device is output as an abnormal state, otherwise it is output as a normal state.

[0068] Here we assume that 30 normal data and 30 abnormal data are extracted from the second data group, then the value of N is 30, and then the first probability and the second probability corresponding to these 30 data are substituted into the first formula to calculate the evaluation value. If the evaluation value is greater than the second threshold, it is considered that the station equipment is in an abnormal state.

[0069] The principle of the first formula is as follows: when the normal data collected in the first time period is close to the abnormal data collected in the second time period, it indicates that the station equipment is also in a normal state in the second time period. After the normal data and abnormal data in the second data group are input into the prediction model, the first probability that the normal data is judged as abnormal data will be higher, and the second probability that the abnormal data is judged as abnormal data will also be higher, because the normal data and abnormal data used to train the model are very similar, which will make the model unable to distinguish them well.

[0070] On the contrary, if the difference between normal data and abnormal data is large, the first probability of normal data being judged as abnormal data will be smaller, and the second probability of abnormal data being judged as abnormal data will be higher, because the distinction between the two is large. In this case, substituting it into the first formula will calculate a larger evaluation value, and a larger evaluation value means that the difference between normal data and abnormal data is large, which means that the current station equipment may be in an abnormal state.

[0071] After obtaining the evaluation value, the present embodiment further includes the following steps:

[0072] After the prediction model outputs evaluation values ​​that are less than the judgment threshold for multiple consecutive times, the change trend of the evaluation values ​​is calculated based on the multiple evaluation values. If the change trend is an increasing trend, the time point when the station equipment enters an abnormal state is predicted based on the increasing trend.

[0073] Specifically, after obtaining an evaluation value in the second time period of this time and the obtained evaluation value is less than the second threshold, the evaluation value is retained, which is defined as evaluation value 1 here. When the next evaluation is performed, evaluation value 2 will be obtained. If evaluation value 2 is still less than the second threshold, evaluation value 2 is retained. When evaluation value 3 is obtained, evaluation value 2 is subtracted from evaluation value 1 to obtain difference 1, and evaluation value 3 is subtracted from evaluation value 2 to obtain difference 2. The average of difference 1 and difference 2 is calculated. If the average value is greater than 0, it indicates that the evaluation value has an increasing trend. Then calculate how many times the average value is added to evaluation value 2 to make the obtained prediction value greater than the second threshold. For example, if the average value is 5, the prediction value obtained after adding evaluation value 3 plus 5 times the average value of 5, that is, adding 25, is greater than the second threshold. Since the second time period is 5 days, it is believed that after 25 days, the station equipment may be abnormal, so 25 days later is used as the time point of the abnormal state.

[0074] In this embodiment, the first time period and the second time period are both preset durations, and the starting point of the first time period is the time point after the device is put into use, and the end point of the second time period is the current time point.

[0075] This implementation of generating a device model includes the following steps:

[0076] The generation model extracts feature points from each device image, matches the feature points in different device images, generates a model depth map for each device image based on the feature points, camera postures and camera parameters of each camera, integrates the model depth maps, and generates a point cloud model.

[0077] Specifically, the camera parameters include the intrinsic and extrinsic parameters of the camera. This embodiment processes all device images through the SFM sparse reconstruction technology to obtain the corresponding monitoring camera posture, camera parameters and sparse point cloud when shooting the device image. The specific extraction process is a prior art and will not be repeated here. After obtaining the sparse point cloud, the StereoBM algorithm in Opecncv is called to perform depth estimation on the sparse point cloud based on the camera parameters, distortion coefficient and baseline length of the monitoring camera to obtain the model depth map. Finally, the Poisson Surface Reconstruction technology is used to densely fuse the depth maps of each model to obtain a complete point cloud model.

[0078] Select a feature point in the point cloud model and define it as the first base point. Get the distance between the first base point and other feature points, select the two feature points closest to the first base point and define them as the second base point. Connect the first base point and the second base point to obtain a triangular mesh. Repeat this step until all feature points are connected and the basic model is obtained.

[0079] Calculate the first area of ​​each triangular mesh in the basic model, set a third threshold, obtain the triangular mesh whose first area is greater than the third threshold, define it as the missing mesh, draw a line segment passing through its center of gravity in the missing mesh, split the missing mesh into polygons, repeat this step until all missing meshes are split, and obtain the device model.

[0080] It should be noted that connecting feature points to form a triangular mesh and presenting the specific shape of a model through the triangular mesh is well known in the art. However, the new solution proposed by the present invention is to obtain the triangular mesh. Specifically, the first area of ​​each triangular mesh is obtained, and the triangular meshes greater than the second threshold are defined as missing meshes. This is because when the area of ​​the triangular mesh is too large, it indicates that the distance between the three feature points constituting the triangular mesh is far, which means that there may be missing feature points in the triangular mesh, which will make the model surface angular. In this case, by adding line segments in the missing mesh to reduce the area of ​​the missing mesh, the smoothness of the model surface can be improved. In this embodiment, the line segments are drawn to pass through the center of gravity of the missing mesh and divide the area of ​​the missing mesh into equal parts.

[0081] In this embodiment, determining the integrity of the device model includes the following steps:

[0082] Set the first length, identify the second length of each edge in each triangular mesh of the device model, define the edge whose second length is greater than the first length as the target edge, obtain the triangular mesh containing the target edge, define it as a rejection target, obtain the second area of ​​the rejection target, and calculate the first average value based on the second formula The second formula is: Among them, S 1 is the sum of the first area, S 2 is the sum of the second area, n is the number of remaining triangle meshes after removing the target;

[0083] Calculate the ratio of the first area of ​​the remaining triangular meshes to the first average value after removing the culled targets, and define the area of ​​the triangular meshes whose ratio is greater than the fourth threshold as the third area;

[0084] Set the model restriction area, obtain the area of ​​the device model outside the model restriction area, and define the area as the fourth area;

[0085] The integrity δ of the device model is calculated based on the fourth formula, which is: Among them, S 3 and S 4 They are the third area and the fourth area respectively.

[0086] After the device model is generated, the second length of each edge of the triangular mesh in the device model is obtained, and the edges greater than the first length are screened out and defined as target edges, and the triangular mesh containing the edge is defined as a rejection target. This is because one of the edges in the triangular mesh is longer, which means that the distance between the two feature points corresponding to the edge is farther, and there may be unrecognized feature points between the two, such as an obtuse triangle. The first area of ​​the obtuse triangle is smaller and may not be recognized and processed in the previous step. Therefore, this step screens out this type of triangular mesh and defines it as a rejection target; then, the first average value is calculated by the second formula. The first average value is the average value of the remaining triangular mesh areas in all triangular meshes after removing the rejection targets.

[0087] If the ratio of the first area of ​​the triangular mesh to the first average value is too large, it means that the area of ​​the triangular mesh is much larger than the overall average value. This triangular mesh may also be an abnormal mesh. Therefore, the area of ​​this type of triangular mesh is obtained and accumulated to obtain the total area of ​​this type of triangular mesh, that is, the third area.

[0088] By using the third formula, the remaining area after removing the second area and the third area from the first area is compared with the total area of ​​the triangular mesh, thereby obtaining the completeness of the device model.

[0089] The present embodiment establishes the judgment model including the following steps:

[0090] The judgment model is a neural network model. The historical data of each station equipment in the past is collected as a training set. The historical data includes the equipment operation data of the station equipment in normal state and fault state. The training set is used to train the neural network model, and the trained neural network model is used as the judgment model.

[0091] Here, the BP neural network is used as the basis for establishing the judgment model. Here, the normal equipment operation data is marked as 1. If there are 5 types of faults 1-5, the corresponding equipment operation data is marked as 2-6. By inputting the marked equipment operation data into the neural network model for training, a judgment model that can be used to judge the fault status and type of station equipment can be obtained. For example, all 100 pieces of equipment operation data collected this time are input into the judgment model, and the judgment model can output the labels of these 100 pieces of data. If all the labels are 2, or most of them are 2, it is considered that the station equipment is in a fault state, and the fault type is 1. The specific neural network can be established through python or Matlab, which is well known to those skilled in the art.

[0092] In this embodiment, placing the device model in the basic map includes the following steps:

[0093] Each surveillance camera obtains the distance between itself and the station equipment, locates the target point between the surveillance cameras shooting the same station equipment based on the distance, and places the equipment model at the position of the target point.

[0094] Specifically, the distance between the surveillance camera itself and the station equipment can be obtained by a monocular ranging algorithm. The monocular ranging algorithm is an existing technology and will not be introduced here. Figure 2 ,exist Figure 2 There are surveillance cameras A1-A4 in the system. The distances between the device model and surveillance cameras A1-A4 are L1-L4 respectively, obtained through a single visual calculation algorithm. Since the device model is generated by surveillance cameras A1-A4, the device model is in the area surrounded by surveillance cameras A1-A4. Then, surveillance cameras A1-A4 are connected in sequence to obtain the area contour, and a suitable target point D is found in the area contour. The distances between target point D and surveillance cameras A1-A4 are L1-L4 respectively. In this way, the position of target point D is determined. When determining, the constraint function of the target point D and the coordinates of each surveillance camera A1-A4 can be established in combination with the distance L1-L4. The specific function can be easily inferred by those skilled in the art based on the above introduction of the present invention, and will not be repeated here.

[0095] like Figure 3 As shown, the present invention also provides a track intelligent operation and maintenance system based on data twin technology, which is used to implement the above-mentioned track intelligent operation and maintenance method based on data twin technology, and the system includes:

[0096] A model generation module is used to establish a basic map, locate the position coordinates of the surveillance camera in the basic map, use multiple surveillance cameras to capture multiple device images of the same station device, establish a generation model and a detection model, the generation model generates a device model based on the device image, the detection model checks the device model and obtains its integrity, and if the integrity is less than a first threshold, generates a reconstruction instruction, reconstructs the device model through the reconstruction instruction, so that the integrity of the device model is greater than or equal to the first threshold, and places the device model in the basic map based on the position coordinates of the surveillance camera;

[0097] The data collection module is used to assign a device code to each device model in the basic map and collect the device operation data corresponding to each device code;

[0098] The state judgment module is used to establish a prediction model and a judgment model. The prediction model receives the equipment operation data and marks the operation state of the station equipment. The operation state includes a normal state and an abnormal state. If it is an abnormal state, the equipment operation data continues to be input into the judgment model. The judgment model receives the equipment operation data and judges whether the station equipment is in a fault state. If it is, the operation state of the station equipment is changed to a fault state. If not, the operation state is retained as an abnormal state.

[0099] The equipment maintenance module is used to mark the equipment models in normal state, abnormal state and fault state as the first color, second color and third color respectively in the basic map, automatically create a fault work order for the station equipment in the fault state, generate recommended processing measures, and automatically generate an operation and maintenance plan for the station equipment in the abnormal state and push it.

[0100] The present invention also discloses a computer storage medium, wherein the computer storage medium stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the above method.

[0101] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0102] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0103] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0104] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A rail intelligent operation and maintenance method based on data twin technology, characterized in that: include: Establishing a basic map, and locating the position coordinates of the surveillance camera in the basic map; Use multiple surveillance cameras to capture multiple device images of the same station device, and establish a generation model and a detection model, wherein the generation model generates a device model based on the device images, and the detection model checks the device model and obtains its integrity; If the degree of integrity is less than a first threshold, a reconstruction instruction is generated, and the device model is reconstructed by the reconstruction instruction so that the degree of integrity of the device model is greater than or equal to the first threshold; The device model is placed in the basic map based on the location coordinates of the surveillance camera, a device code is assigned to each device model in the basic map, device operation data corresponding to each device code is collected, and a prediction model and a judgment model are established. The prediction model receives the device operation data and marks the operation status of the station equipment; The operation state includes a normal state and an abnormal state. If it is the abnormal state, the equipment operation data continues to be input into the judgment model. The judgment model receives the equipment operation data and judges whether the station equipment is in a fault state. If it is, the operation state of the station equipment is changed to the fault state. If not, the operation state is kept as the abnormal state. In the basic map, the equipment models in the normal state, the abnormal state and the faulty state are marked as the first color, the second color and the third color respectively, a fault work order is automatically established for the station equipment in the faulty state, and recommended processing measures are generated, and an operation and maintenance plan is automatically generated for the station equipment in the abnormal state and pushed.

2. The method according to claim 1, characterized in that The prediction model determines the operating status of the device including the following steps: Setting a first time period and a second time period, wherein the first time period is earlier than the second time period, collecting a plurality of pieces of operation data of the station equipment in the first time period and marking them as normal data, and collecting a plurality of pieces of operation data in the second time period and marking them as abnormal data; The normal data and the abnormal data are mixed to generate a first data group and a second data group, and after the prediction model is trained using the first data group, the second data group is input into the prediction model, and the prediction model outputs a first probability that the normal data in the second data group is the abnormal data, and a second probability that the abnormal data is the abnormal data; N pieces of the normal data and the abnormal data are extracted from the second data group, and the evaluation value α is calculated based on the first formula, where the first formula is: Wherein, N is the number of abnormal data in the second data group, The first probability of extracting the nth normal data is: To extract the second probability of the nth abnormal data, if the evaluation value is greater than a second threshold, the device is output as the abnormal state, otherwise it is output as the normal state.

3. The method according to claim 2, characterized in that After obtaining the evaluation value, the following steps are also included: After the prediction model outputs the evaluation value that is less than the judgment threshold for multiple consecutive times, the change trend of the evaluation value is calculated based on the multiple evaluation values. If the change trend is an increasing trend, the time point when the station equipment enters the abnormal state is predicted based on the increasing trend.

4. The method according to claim 2, characterized in that: The first time period and the second time period are both preset time periods, and the starting point of the first time period is the time point after the device is put into use, and the end point of the second time period is the current time point.

5. The method according to claim 1, characterized in that Generating the device model includes the following steps: The generation model extracts feature points from each of the device images, matches the feature points in different device images, generates a model depth map of each of the device images based on the feature points, the camera postures and camera parameters of each camera, and integrates the model depth maps to generate a point cloud model; Select a feature point in the point cloud model and define it as a first base point. Obtain the distance between the first base point and other feature points, select two feature points closest to the first base point and define them as second base points. Connect the first base point and the second base point to obtain a triangular mesh. Repeat this step until all feature points are connected to obtain a basic model. Calculate the first area of ​​each triangular mesh in the basic model, set a third threshold, obtain the triangular mesh whose first area is greater than the third threshold, define it as a missing mesh, draw a line segment passing through the center of gravity in the missing mesh, split the missing mesh into polygons, repeat this step until all the missing meshes are split, and obtain the device model.

6. The method according to claim 5, characterized in that Determining the completeness of the device model includes the following steps: Set a first length, identify a second length of each edge in each triangular mesh of the device model, define an edge whose second length is greater than the first length as a target edge, obtain the triangular mesh containing the target edge, define it as a rejection target, obtain a second area of ​​the rejection target, and calculate a first average value based on a second formula The second formula is: Wherein, S1 is the sum of the first areas, S2 is the sum of the second areas, and n is the number of the remaining triangular meshes after removing the elimination targets; The ratio of the first area of ​​the remaining triangular mesh to the first average value after removing the elimination target is calculated, the area of ​​the triangular mesh where the ratio is greater than a fourth threshold is defined as a third area, and the integrity δ of the device model is calculated based on a third formula, wherein the third formula is: Wherein, S3 is the third area.

7. The method according to claim 1, characterized in that Establishing the judgment model includes the following steps: The judgment model is a neural network model, which collects historical data of each station equipment in the past as a training set, wherein the historical data includes the equipment operation data of the station equipment in the normal state and the fault state. The neural network model is trained using the training set, and the trained neural network model is used as the judgment model.

8. The method according to claim 1, characterized in that Placing the device model in the base map comprises the following steps: Each monitoring camera obtains the distance between itself and the station equipment, locates a target point between various monitoring cameras photographing the same station equipment based on the distance, and places the equipment model at the position of the target point.

9. A rail intelligent operation and maintenance system based on data twin technology, used to implement the method according to any one of claims 1 to 8, characterized in that: include: a model generation module, used to establish a basic map, locate the position coordinates of the surveillance camera in the basic map, use multiple surveillance cameras to capture multiple device images of the same station device, establish a generation model and a detection model, wherein the generation model generates a device model based on the device image, and the detection model checks the device model and obtains its integrity. If the integrity is less than a first threshold, a reconstruction instruction is generated, and the device model is reconstructed by the reconstruction instruction so that the integrity of the device model is greater than or equal to the first threshold, and the device model is placed in the basic map based on the position coordinates of the surveillance camera; A data collection module, used for assigning a device code to each device model in the basic map, and collecting device operation data corresponding to each device code; A state judgment module is used to establish a prediction model and a judgment model. The prediction model receives the equipment operation data and marks the operation state of the station equipment. The operation state includes a normal state and an abnormal state. If it is the abnormal state, the equipment operation data is continuously input into the judgment model. The judgment model receives the equipment operation data and judges whether the station equipment is in a fault state. If it is, the operation state of the station equipment is changed to the fault state. If not, the operation state is retained as the abnormal state. The equipment maintenance module is used to mark the equipment models in the normal state, the abnormal state and the faulty state in the basic map as the first color, the second color and the third color respectively, automatically establish a fault work order for the station equipment in the faulty state, generate recommended processing measures, and automatically generate an operation and maintenance plan for the station equipment in the abnormal state and push it.

10. A computer storage medium, characterized in that: The computer storage medium stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the method according to any one of claims 1 to 8.

Images