Urban rail communication machine room three-dimensional live-action operation and maintenance method and device, electronic equipment and medium

Abstract

The invention provides an urban rail communication machine room three-dimensional live-action operation and maintenance method and device, electronic equipment and a medium, and the method comprises the steps: inputting the position features of position data into a first multi-layer perceptron model in a neural radiation field model, and obtaining density data and information enhancement features; and inputting the direction features and the information enhancement features of the direction data into a second multilayer perceptron model to obtain color data, and performing voxel rendering in combination with the density data and the color data to generate a high-fidelity three-dimensional real scene model. The real scene model provides a visual scene view for operation and maintenance personnel. And performing equipment fault detection on the panoramic image to obtain an equipment fault detection result, and displaying a three-dimensional live-action operation and maintenance result in the three-dimensional live-action model according to the result. According to the mode of combining fault detection with the three-dimensional real scene model, operation and maintenance personnel can quickly position fault equipment in a visual three-dimensional scene and know the fault condition in time, so that the fault detection efficiency and the response speed are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of rail transit, in particular to a three-dimensional real scene operation and maintenance method and device for a city rail communication machine room, an electronic device and a medium. BACKGROUND

[0002] The communication machine room of the urban rail transit station integrates key core equipment of each sub-professional of the communication system, and the safety management level of the machine room is required to be high, and the number of internal equipment of the cabinet is large, the types are various, the wiring is complex, and the operation and maintenance professional difficulty is required to be high. At present, the inspection of the machine room mainly adopts two ways of manual periodic inspection and interface data monitoring. In the manual inspection, the equipment of the machine room is periodically inspected by manual, whether the equipment is abnormal is checked, and then maintenance is guided. In the interface data monitoring, the internal parameters of the equipment are collected and monitored based on Modbus, TCP / UDP (Transmission Control Protocol / User Datagram Protocol), Internet of Things protocol and other ways through the centralized alarm system, and then the faults are analyzed. The existing operation and maintenance mode has the defects of low fault detection efficiency, high operation and maintenance cost, and untimely fault discovery.

[0003] With the wide successful application of three-dimensional reconstruction, VR (Virtual Reality), deep learning and other technologies in urban modeling, automatic driving and other fields, the communication machine room operation and maintenance urgently needs to apply three-dimensional reconstruction, image recognition, artificial intelligence and other technologies to realize the real scene visualization, intelligentization, automation and low-cost operation and maintenance of the communication machine room, and solve the problems of low efficiency and high cost of the on-site inspection and operation and maintenance of the communication machine room. SUMMARY

[0004] The present application provides a three-dimensional real scene operation and maintenance method and device for a city rail communication machine room, which solves the defects of low fault detection efficiency, high operation and maintenance cost and untimely fault discovery of the existing operation and maintenance mode.

[0005] The present application provides a three-dimensional real scene operation and maintenance method for a city rail communication machine room, comprising the following steps: acquire a panoramic image of the city rail communication machine room, and determine camera pose data of the panoramic image, the camera pose data comprising position data and direction data of the panoramic image; input the position feature of the position data into a first multi-layer perception machine model in a neural radiation field model, obtain density data and information enhancement features output by the first multi-layer perception machine model, input the direction feature of the direction data and the information enhancement features into a second multi-layer perception machine model in the neural radiation field model, and obtain color data output by the second multi-layer perception machine model; According to the density data and the color data, voxel rendering is performed to obtain a three-dimensional real scene model; Device fault detection is performed on the panoramic image to obtain a device fault detection result, and according to the device fault detection result, a three-dimensional real scene operation and maintenance result is displayed in the three-dimensional real scene model.

[0006] According to the present application, a three-dimensional real scene operation and maintenance method for a city rail communication machine room is provided, and the training step of the neural radiance field model comprises: Sample panoramic images of the city rail communication machine room and label colors of the sample panoramic images are obtained, and sample camera pose data of the sample panoramic images is determined; the sample camera pose data comprises sample position data and sample direction data of the sample panoramic images; Sample position features of the sample position data are input into a first multi-layer perception machine model in an initial neural radiance field model to obtain predicted density data and predicted information enhancement features output by the first multi-layer perception machine model; Sample direction features of the sample direction data and the predicted information enhancement features are input into a second multi-layer perception machine model in the initial neural radiance field model to obtain predicted color data output by the second multi-layer perception machine model; According to the predicted density data and the predicted color data, voxel rendering is performed to obtain a predicted rendering result; Based on the color in the predicted rendering result and the label color, a target loss is determined, and the initial neural radiance field model is trained based on the target loss to obtain the neural radiance field model.

[0007] According to the present application, a three-dimensional real scene operation and maintenance method for a city rail communication machine room is provided, and according to the predicted density data and the predicted color data, voxel rendering is performed to obtain a predicted rendering result, which comprises: According to the cumulative opacity of the sampling points from the starting point to the ending point of the ray, the predicted density data and the predicted color data, voxel rendering is performed to obtain the predicted rendering result; The ray is a light path in a three-dimensional real scene scene mapped by a camera corresponding to the initial neural radiance field model.

[0008] According to the present application, a three-dimensional real scene operation and maintenance method for a city rail communication machine room is provided, and the device fault detection comprises device type detection, indicator light type detection and fault state detection; The device fault detection on the panoramic image to obtain a device fault detection result comprises: The device type detection is performed on the panoramic image to obtain a device type detection result; According to the device type in the device type detection result, image segmentation is performed on the panoramic image to obtain a device region image, and according to the device type, the indicator light type detection is performed on the device region image to obtain an indicator light type detection result; According to the indicator light type in the indicator light type detection result, image segmentation is performed on the device region image to obtain an indicator light region image, and according to the device type and the indicator light type, the fault state detection is performed on the indicator light region image to obtain the device fault detection result.

[0009] According to the device type, the indicator light type detection is performed on the device region image to obtain an indicator light type detection result, including: According to the device type, a target indicator light type detection model is selected from a plurality of indicator light type detection models; The device region image is input into the target indicator light type detection model to obtain the indicator light type detection result output by the target indicator light type detection model; The training step of each indicator light type detection model in the plurality of indicator light type detection models includes: A sample device region image and a label indicator light type of the sample device region image are obtained; the label indicator light type includes a power indicator light, a health state indicator light, a network port connection state indicator light, a data transmission state indicator light and a rate indicator light; The sample device region image is input into each indicator light type detection model to obtain a predicted indicator light type output by each indicator light type detection model; Based on the predicted indicator light type and the label indicator light type, each indicator light type detection model is trained respectively to obtain the plurality of indicator light type detection models.

[0010] According to the device type and the indicator light type, the fault state detection is performed on the indicator light region image to obtain the device fault detection result, including: According to the device type and the indicator light type, a target device state detection model is selected from a plurality of device state detection models; The indicator light region image is input into the target device state detection model to obtain the device fault detection result output by the target device state detection model; The training step of each device state detection model in the plurality of device state detection models includes: An indicator light area image is acquired, and a label device state of the indicator light area image is acquired; the label device state includes a normal operation state, an unused state, and a fault state; The indicator light area image is input into the device state detection model, and a predicted device state output by the device state detection model is obtained. According to the predicted device state and the label device state, the device state detection model is trained respectively, and the plurality of device state detection models are obtained.

[0011] According to the present application, a three-dimensional real scene operation and maintenance method for a city rail communication machine room is provided, and the camera pose data of the panoramic image is determined, including: The panoramic image is input into a motion recovery structure, and the camera pose data output by the motion recovery structure is obtained.

[0012] The present application also provides a three-dimensional real scene operation and maintenance device for a city rail communication machine room, including the following units: An acquisition unit is configured to acquire a panoramic image of a city rail communication machine room and determine camera pose data of the panoramic image, wherein the camera pose data includes position data and direction data of the panoramic image. An input unit is configured to input position features of the position data into a first multi-layer perception machine model in a neural radiation field model, obtain density data and information enhancement features output by the first multi-layer perception machine model, and input direction features of the direction data and the information enhancement features into a second multi-layer perception machine model in the neural radiation field model, to obtain color data output by the second multi-layer perception machine model. A voxel rendering unit is configured to perform voxel rendering according to the density data and the color data, to obtain a three-dimensional real scene model. A three-dimensional real scene operation and maintenance unit is configured to perform device fault detection on the panoramic image, to obtain a device fault detection result, and display a three-dimensional real scene operation and maintenance result in the three-dimensional real scene model according to the device fault detection result.

[0013] The present application also provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the three-dimensional real scene operation and maintenance method for a city rail communication machine room as described above.

[0014] The present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the three-dimensional real scene operation and maintenance method for a city rail communication machine room as described above.

[0015] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the three-dimensional real scene operation and maintenance method of the urban rail communication machine room according to any one of the above.

[0016] The three-dimensional real scene operation and maintenance method, device, electronic equipment and medium provided by the application input the position features of the position data into a first multi-layer perception model in a neural radiance field model to obtain density data and information enhancement features. This process utilizes the powerful representation capability of the neural radiance field model and can accurately reconstruct the structural information of a three-dimensional scene. Then, the direction features of the direction data and the information enhancement features are input into a second multi-layer perception model to obtain color data, and the density data and the color data are combined for voxel rendering to generate a high-fidelity three-dimensional real scene model. This real scene model not only contains rich geometric information but also has a realistic visual effect, providing an intuitive scene view for operation and maintenance personnel. On this basis, equipment fault detection is performed on the panoramic image to obtain equipment fault detection results, and three-dimensional real scene operation and maintenance results are displayed in the three-dimensional real scene model according to the results. This combination of fault detection and three-dimensional real scene model enables operation and maintenance personnel to quickly locate fault equipment in an intuitive three-dimensional scene and promptly understand the fault situation, thereby significantly improving the fault detection efficiency and response speed. At the same time, the automatic processing process reduces manual intervention, reduces operation and maintenance costs, and improves the overall efficiency and quality of operation and maintenance work. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0018] Figure 1 is one of the flowcharts of the three-dimensional real scene operation and maintenance method of the urban rail communication machine room provided by the application.

[0019] Figure 2 is a structural schematic diagram of the neural radiance field model provided by the application.

[0020] Figure 3 is a schematic diagram of the image update acquisition device provided by the application.

[0021] Figure 4 is another flowchart of the three-dimensional real scene operation and maintenance method of the urban rail communication machine room provided by the application.

[0022] Figure 5 is a schematic diagram of the fault detection method provided by the application.

[0023] Figure 6 is a structural schematic view of a three-dimensional real scene operation and maintenance device for a city rail communication machine room provided by the present application.

[0024] Figure 7 is a structural schematic view of an electronic device provided by the present application. DETAILED DESCRIPTION

[0025] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0026] The terms "first", "second" and the like in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second" and the like are generally of a kind.

[0027] Figure 1 is one of flow schematic views of a three-dimensional real scene operation and maintenance method for a city rail communication machine room provided by the present application, as shown in the figure, the method comprises steps 110, 120, 130 and 140. Figure 1

[0028] Step 110, acquiring a panoramic image of a city rail communication machine room, and determining camera pose data of the panoramic image, the camera pose data comprising position data and direction data of the panoramic image.

[0029] Specifically, the panoramic image of the city rail communication machine room can be acquired, wherein the panoramic image of the city rail communication machine room refers to the panoramic image of the city rail communication machine room including the overall environment and internal cabinet equipment of the machine room. This panoramic image can be photographed by a panoramic camera or an image acquisition device, and can fully display the internal layout and equipment state of the machine room, providing intuitive visual information for monitoring and management of the machine room.

[0030] Among them, image data acquisition can be carried out by manual, robot, unmanned aerial vehicle and the like, and a high-definition camera is selected as the image acquisition device, which is manually held or fixedly installed on a movable device such as a robot or an unmanned aerial vehicle, and image data with the same size is acquired.

[0031] ​Then, camera pose data of the panoramic image can be determined, which refers to information describing the position and orientation of the camera in three-dimensional space. For example, the panoramic image can be input into Structure from Motion (SFM) to obtain camera pose data output by the Structure from Motion. The process of applying Structure from Motion to calculate image pose data mainly includes steps such as feature point extraction, feature point matching, geometric verification, initialization, image registration and triangulation, and bundle adjustment. First, feature points are extracted from the input panoramic image, which are usually points with significant information in the panoramic image, such as corner points, SIFT (Scale-Invariant Feature Transform) or ORB (Oriented FAST and Rotated BRIEF) local feature points. Then, the descriptors of these feature points are calculated, and matching is performed between different images to find matching point pairs by distance measurement. Then, the relative poses between panoramic images are estimated using the geometric relationship between matching points, such as the fundamental matrix or essential matrix, and the accuracy of matching points is verified by methods such as RANSAC (Random Sample Consensus), and incorrect matching points are removed. In the initialization stage, a suitable image pair is selected as the initial model for initial camera pose estimation. Subsequently, new panoramic images are gradually added, and the camera pose of the new image is estimated by solving the PnP (Perspective-n-Point) problem, and the new image is registered to the current model. At the same time, the three-dimensional coordinates of the feature points are calculated by triangulation using the known camera pose and two-dimensional coordinates of the feature points. Finally, in order to optimize the camera pose and the position of the three-dimensional points, the reprojection error of the feature points is minimized, and a bundle adjustment is performed using a nonlinear optimization algorithm. This process can recover the three-dimensional structure of the scene and the motion state of the camera from a series of images, thereby realizing the calculation of camera pose data.

[0032] The camera pose data includes position data and orientation data of the panoramic image. The position data is used to determine the specific position of the camera in space, helping to calculate the relative position relationship between the camera and the objects in the scene. The orientation data is used to determine the orientation of the camera, helping to calculate the field of view direction and angle of view change of the camera.

[0033] In step 120, the position features of the position data are input into a first multi-layer perceptron model in the neural radiance field model to obtain density data and information enhancement features output by the first multi-layer perceptron model. The orientation features of the orientation data and the information enhancement features are input into a second multi-layer perceptron model in the neural radiance field model to obtain color data output by the second multi-layer perceptron model.

[0034] Specifically, Figure 2 is a structural diagram of the neural radiance field model provided by the present application, as shown in Figure 2 The neural radiance field network is composed of two multilayer perceptrons (MLP) as shown in the figure, taking position data (x) x, y, z ) and direction data (d) θ, φ ) as input, and density data σ and color data c as output. For example, the position data (x) x, y, z ) and the direction data (d) θ, φ ) are respectively encoded with position information, mapped to a high-dimensional space, to obtain γ(x) x, y, z ) and γ(d) θ, φ ). Then the position feature γ(x) x, y, z is input into the first MLP to obtain the density data σ and an information enhancement feature. The information enhancement feature refers to information that can enhance the subsequent processing process obtained through nonlinear transformation and feature extraction of the network in the MLP processing process. It can be understood that the information enhancement feature can provide more rich context information, which helps to improve the representation ability of the network to the scene and the rendering quality.

[0035] Further, the direction feature of the direction data and the information enhancement feature are input into the second multilayer perceptron model in the neural radiance field model to obtain the color data output by the second multilayer perceptron model. For example, the direction feature γ(d) θ, φ ) and the information enhancement feature are input into the second MLP to obtain the color data c ) through training. The color data refers to the color information of a point in a three-dimensional space. It is usually a three-dimensional vector representing the color value of the point in the RGB (red, green, blue) color space. The density data refers to the volume density of a point in a three-dimensional space, which is used to describe the opacity or the density of the matter of the point.

[0036] In step 130, according to the density data and the color data, voxel rendering is performed to obtain a three-dimensional real scene model.

[0037] Specifically, after obtaining the density data and the color data, voxel rendering can be performed according to the density data and the color data to obtain a three-dimensional real scene model. Specifically, the voxel rendering technology utilizes the density data and the color data, integrates each sampling point along a ray path, and calculates the final color and opacity of each pixel. In this way, a high-quality three-dimensional real scene model observed from different perspectives can be generated. That is, by using the voxel rendering technology, the density and color data can be synthesized into a panoramic image to obtain a complete three-dimensional real scene model.

[0038] At step 140, device fault detection is performed on the panoramic image to obtain a device fault detection result, and according to the device fault detection result, a three-dimensional real scene operation and maintenance result is displayed in the three-dimensional real scene model.

[0039] Specifically, after obtaining the panoramic image, device fault detection can be performed on the panoramic image to obtain a device fault detection result. Here, the device fault detection includes device type detection, indicator light type detection, and fault state detection.

[0040] For example, when a fault state is detected, the fault detection result and fault-related information are standardized, including fault device images, device names, device-related information such as system to which the device belongs, abnormal indicator light names, fault state conditions, fault occurrence times, etc.

[0041] Further, according to the device fault detection result, a three-dimensional real scene operation and maintenance result can be displayed in the three-dimensional real scene model. Here, an information transmission interface can be used to set a fault information transmission protocol to transmit the device fault detection result obtained by performing device fault detection on the panoramic image to an operation and maintenance-related system such as a communication system centralized alarm system. Then, a three-dimensional model viewing tool is applied to provide three-dimensional visualization monitoring and fault alarm information prompting for inspection and operation and maintenance personnel, etc.

[0042] Among them, the three-dimensional model viewing tools such as NeRFStudio and SIBR_viewer are applied to provide monitoring of the three-dimensional real scene model of the neural radiation field for inspection and operation and maintenance users, and to view the three-dimensional real scene information inside the urban rail communication machine room.

[0043] When there is a device fault, the centralized alarm and other operation and maintenance systems actively send fault alarm information to the communication machine room three-dimensional real scene monitoring software through a data interface. The monitoring software establishes data connection with various external operation and maintenance systems, receives alarm information in real time and performs data standardization processing.

[0044] The monitoring software not only displays alarm information to the operation and maintenance user, but also automatically adjusts the display perspective of the three-dimensional real scene model viewing tool according to the position information of the fault equipment in the alarm information, thereby providing the operation and maintenance personnel with a convenient real-time status viewing function of the fault equipment.

[0045] The method provided by the embodiment of the application inputs the position feature of the position data into a first multi-layer perception model in a neural radiance field model to obtain density data and information enhancement features. This process utilizes the powerful representation capability of the neural radiance field model and can accurately reconstruct the structural information of a three-dimensional scene. Then, the direction feature of the direction data and the information enhancement features are input into a second multi-layer perception model to obtain color data. The density data and the color data are combined for voxel rendering to generate a high-fidelity three-dimensional real scene model. This real scene model not only contains rich geometric information but also has a realistic visual effect, thereby providing an intuitive scene view for the operation and maintenance personnel. On this basis, panoramic images are subjected to equipment fault detection to obtain equipment fault detection results, and three-dimensional real scene operation and maintenance results are displayed in the three-dimensional real scene model according to the results. This combination of fault detection and the three-dimensional real scene model enables the operation and maintenance personnel to quickly locate the fault equipment in the intuitive three-dimensional scene and timely understand the fault condition, thereby significantly improving the fault detection efficiency and response speed. Meanwhile, the automatic processing process reduces manual intervention, reduces operation and maintenance costs, and improves the overall efficiency and quality of operation and maintenance work.

[0046] Based on the above embodiment, the training step of the neural radiance field model comprises: In step 210, sample panoramic images of the urban rail communication machine room and label colors of the sample panoramic images are obtained, and sample camera pose data of the sample panoramic images is determined. The sample camera pose data comprises sample position data and sample direction data of the sample panoramic images. In step 220, sample position features of the sample position data are input into a first multi-layer perception model in an initial neural radiance field model to obtain predicted density data and predicted information enhancement features output by the first multi-layer perception model. In step 230, sample direction features of the sample direction data and the predicted information enhancement features are input into a second multi-layer perception model in the initial neural radiance field model to obtain predicted color data output by the second multi-layer perception model. In step 240, voxel rendering is performed according to the predicted density data and the predicted color data to obtain a predicted rendering result. In step 250, a target loss is determined based on the color in the predicted rendering result and the label color, and the initial neural radiance field model is trained based on the target loss to obtain the neural radiance field model.

[0047] Specifically, first, a sample panoramic image (a training data set) of a city rail communication machine room is acquired, and a label color of the sample panoramic image is determined, and sample camera pose data of the sample panoramic image is determined. Here, the sample camera pose data includes sample position data and sample direction data of the sample panoramic image. Figure 3 is a schematic diagram of an image updating and collecting device provided by the present application, as Figure 3 As shown in the figure, the image updating and collecting device is installed on the equipment cabinet in the machine room, and mobile sliding rails, motors, cameras and other equipment are used to realize periodic image collection of the cabinet, which is used for updating the training data set.

[0048] Among them, the image collected needs to keep consistent with the camera pose when the corresponding image in the pre-trained image data set is collected, and the camera is controlled to collect images and update them to the training data set synchronously, so that the latest collected image replaces the original image, and the naming of the new and old images is kept consistent.

[0049] In an embodiment, the sample panoramic image, the sample position data and the sample direction data can be integrated into the LLFF (Local Light Field Fusion) format to form a training data set for pre-training of a neural radiance field model corresponding to the neural radiance field model. Further, based on the updated training data set, the initial neural radiance model is trained, the model network parameters are iteratively updated, and the updated neural radiance field model of the city rail communication machine room is obtained.

[0050] Then, the sample position features of the sample position data can be input into the first multi-layer perception model in the neural radiance field model to obtain the predicted density data and the predicted information enhancement features output by the first multi-layer perception model. The sample direction features of the sample direction data and the predicted information enhancement features are input into the second multi-layer perception model in the neural radiance field model to obtain the predicted color data output by the second multi-layer perception model.

[0051] Further, after obtaining the predicted density data and the predicted color data, voxel rendering can be performed according to the predicted density data and the predicted color data to obtain a predicted rendering result.

[0052] Finally, based on the color in the predicted rendering result and the label color, a target loss is determined, and the initial neural radiance field model after training is used as the neural radiance field model based on the target loss.

[0053] Here, according to the color in the predicted rendering result and the label color of the sample panoramic image, the formula of the target loss is as follows: In the formula, denotes a target loss, for predicting colors in a rendered result, for a label color of a sample panoramic image.

[0054] The method provided by the embodiment of the present application first separates spatial position features and direction features by using a double-branch multi-layer perception architecture, a first MLP converts sample position data into density data and information enhancement features, a second MLP combines direction features and information enhancement features to predict color data, and this decoupling design effectively improves the modeling capability of the model for complex lighting and material changes. The predicted density data and the predicted color data are rendered into a predicted rendered result by voxel rendering, which not only retains the geometric details of a three-dimensional scene, but also ensures the accuracy of perspective-related optical effects (such as specular reflection).

[0055] Secondly, in the training process, the loss function optimization by comparing the colors in the predicted rendered result with the label colors forces the neural radiance field model to learn the implicit radiance field representation of the scene, and this end-to-end supervision enables the model to automatically capture high-frequency geometric features (such as the edges of computer room equipment) and low-frequency appearance features (such as wall textures) without the need for explicit construction of a three-dimensional mesh or a point cloud, thereby improving the realistic rendering effect.

[0056] Based on the above embodiment, step 240 comprises: Step 241, voxel rendering is performed according to the cumulative opacity of the sampling points from the starting point to the ending point of the ray, the predicted density data and the predicted color data, to obtain the predicted rendered result. The ray is a light path in the three-dimensional real scene scene mapped by the camera corresponding to the initial neural radiance field model.

[0057] Specifically, voxel rendering can be performed according to the cumulative opacity of the sampling points from the starting point to the ending point of the ray, the predicted density data and the predicted color data, to obtain the predicted rendered result, and the formula is as follows: In the formula, denotes a predicted rendered result, c denotes predicted color data, σ denotes predicted density data, r denotes a distance on a camera projection ray, d denotes a direction of the camera projection ray, t denotes a distance between a sampling point on the camera projection ray and a camera optical center, denotes a cumulative opacity of the sampling points from to denotes a differential distance, denotes a starting point of the ray, ​End point of a ray, which is a light path in a three-dimensional real scene mapped by a camera corresponding to an initial neural radiance field model Here, the voxel rendering method is to synthesize and calculate the color values of the pixel coordinates of the camera projection ray in the scene.

[0058] Based on the above embodiment, the device fault detection includes device type detection, indicator light type detection, and fault state detection. In step 140, the panoramic image is subjected to device fault detection to obtain a device fault detection result, including: In step 141, the device type detection is performed on the panoramic image to obtain a device type detection result. In step 142, according to the device type in the device type detection result, the panoramic image is subjected to image segmentation to obtain a device region image, and according to the device type, the indicator light type detection is performed on the device region image to obtain an indicator light type detection result. In step 143, according to the indicator light type in the indicator light type detection result, the device region image is subjected to image segmentation to obtain an indicator light region image, and according to the device type and the indicator light type, the fault state detection is performed on the indicator light region image to obtain the device fault detection result.

[0059] Specifically, the device fault detection includes device type detection, indicator light type detection, and fault state detection, and accordingly, the device type detection can be performed on the panoramic image to obtain a device type detection result. Here, the panoramic image can be input into a device type detection model to obtain a device type detection result output by the device type detection model. The device type detection model is obtained by training a lightweight target detection algorithm such as YOLO. For example, the devices such as servers, storage devices, and switches in the historically stored images are labeled to form a training data set and a test data set, the device type detection model is trained by using the training data set, and the test data set is used for testing to ensure that the model detection accuracy is not less than 95%.

[0060] Then, the panoramic image can be image segmented according to the device type in the device type detection result to obtain a device region image, and the device region image can be indicator light type detection according to the device type to obtain an indicator light type detection result. The device region image is a rectangular or irregular polygon image block containing only a single device and its adjacent structure (such as a fixed support, a panel indicator light, and other components directly associated with the device) in the original panoramic image, which is accurately cropped or extracted from the panoramic image according to the instance-level mask generated by the device type detection result, and the background pixels are set to zero or transparent, so that the image region is one-to-one corresponding to the corresponding device in geometry, color and semantics, and can be directly used for subsequent defect detection, attribute identification or visualization superposition tasks.

[0061] The device type includes servers, storage devices, switches, and the like, which are not specifically limited in the embodiments of the present application.

[0062] The indicator light type detection result is a structured information set output after identifying the indicator light and its base in the device region image. The indicator light type detection result includes a type identifier and position information, wherein the type identifier is one of the predefined categories such as "red light-on", "yellow light-on", "green light-off", or "faulty light"; and the position information is a pixel-level coordinate given in the form of a rectangular frame or a mask, which is used to accurately locate the area of the indicator light in the device region image.

[0063] Then, the device region image can be image segmented according to the indicator light type in the indicator light type detection result to obtain an indicator light region image, and the indicator light region image can be fault state detection according to the device type and the indicator light type to obtain a device fault detection result. The indicator light region image is a local image block containing only a single indicator light and its adjacent light-emitting surface or lampshade, which is extracted from the cropped device region image according to the pixel-level mask or the minimum circumscribed rectangle given by the indicator light type detection result, and the background pixels are set to zero or transparent, so that the image region is uniquely corresponding to the indicator light in geometry, color and semantics, and can be directly used for subsequent fault state determination.

[0064] Based on the above embodiments, the indicator light type detection of the device region image according to the device type in step 142 to obtain an indicator light type detection result includes: Step 1421, selecting a target indicator light type detection model from a plurality of indicator light type detection models according to the device type; Step 1422, inputting the device region image into the target indicator light type detection model to obtain the indicator light type detection result output by the target indicator light type detection model. The training step of each indicator light type detection model in the plurality of indicator light type detection models comprises: Obtaining a sample device region image and a label indicator light type of the sample device region image; the label indicator light type comprises a power indicator light, a health status indicator light, a network port connection state indicator light, a data transmission state indicator light and a rate indicator light; Inputting the sample device region image into each indicator light type detection model to obtain a predicted indicator light type output by each indicator light type detection model; Training each indicator light type detection model based on the predicted indicator light type and the label indicator light type to obtain the plurality of indicator light type detection models.

[0065] Specifically, considering that different device types correspond to different indicator lights, and the indicator light detection models corresponding to the indicator lights are also different, therefore, a target indicator light type detection model can be selected from the plurality of indicator light type detection models according to the device type.

[0066] Then, the device region image can be input into the target indicator light type detection model to obtain an indicator light type detection result output by the target indicator light type detection model.

[0067] The training step of each indicator light type detection model in the plurality of indicator light type detection models comprises: First, a sample device region image and a label indicator light type of the sample device region image are obtained, wherein the label indicator light type comprises a power indicator light, a health status indicator light, a network port connection state indicator light, a data transmission state indicator light and a rate indicator light, etc., which are not limited by the embodiments of the present application.

[0068] Then, after obtaining the sample device region image, the sample device region image can be input into each indicator light type detection model to obtain a predicted indicator light type output by each indicator light type detection model.

[0069] Finally, each indicator light type detection model is trained based on the predicted indicator light type and the label indicator light type to obtain the plurality of indicator light type detection models.

[0070] It can be understood that the greater the difference between the predicted indicator light type and the label indicator light type, the greater the first loss; the smaller the difference between the predicted indicator light type and the label indicator light type, the smaller the first loss.

[0071] In an embodiment, the indicator light type detection model is trained by a lightweight target detection algorithm such as YOLO. The indicator light types in historical stored images of a certain type of device, such as power indicator lights, health status indicator lights, network connection status indicator lights, data transmission status indicator lights, and rate indicator lights, are labeled to form a training data set and a test data set. The training data set is applied to train the indicator light type detection model, and the test data set is applied to test, so as to ensure that the model detection accuracy is not less than 90%.

[0072] Based on the above embodiment, the fault state detection on the indicator light area image according to the device type and the indicator light type in step 143 to obtain the device fault detection result includes: Step 1431, selecting a target device state detection model from a plurality of device state detection models according to the device type and the indicator light type; Step 1432, inputting the indicator light area image into the target device state detection model to obtain the device fault detection result output by the target device state detection model; The training steps of each device state detection model in the plurality of device state detection models include: Obtaining an indicator light area image and a labeled device state of the indicator light area image; the labeled device state includes a normal operation state, an unused state, and a fault state; Inputting the indicator light area image into each device state detection model to obtain a predicted device state output by each device state detection model; Training each device state detection model according to the predicted device state and the labeled device state to obtain the plurality of device state detection models.

[0073] Specifically, considering that different device types and indicator light types correspond to different device states, correspondingly, different device types and indicator light types correspond to different device state detection models, therefore, a target device state detection model can be selected from a plurality of device state detection models according to the device type and the indicator light type.

[0074] Then, inputting the indicator light area image into the target device state detection model to obtain the device fault detection result output by the target device state detection model.

[0075] Here, the training steps of each device state detection model in the plurality of device state detection models include: First, an indicator light area image and a labeled device state of the indicator light area image are obtained, the labeled device state includes a normal operation state, an unused state, and a fault state, etc., which are not specifically limited by the embodiments of the present application.

[0076] The indicator light area image is input into each device state detection model to obtain a predicted device state output by each device state detection model. Here, the preset device state includes a normal operation state, an unused state, a fault state, and the like, and embodiments of the present application do not make specific limitations thereto.

[0077] Then, after obtaining the predicted device state, each device state detection model can be trained according to the predicted device state and the labeled device state, to obtain a plurality of device state detection models.

[0078] It can be understood that the greater the difference between the predicted device state and the labeled device state, the greater the second loss, and the smaller the difference between the predicted device state and the labeled device state, the smaller the second loss.

[0079] It can be understood that there are multiple device state detection models, which are selected according to the device type and the indicator light type corresponding to the to-be-detected indicator light area image. The device state detection model is a lightweight target detection algorithm such as YOLO, which is obtained by training. According to the display state rules corresponding to each type of device and each type of indicator light contained therein, the various operation states such as normal operation, non-use, and fault of the historically stored indicator light images are labeled to form a training data set and a test data set. The training data set is applied to train the fault state detection model, and the test data set is applied to test, to ensure that the model detection accuracy is not less than 90%.

[0080] Based on any of the above embodiments, Figure 4 is a flowchart of a second city rail communication machine room three-dimensional real scene operation and maintenance method provided by the present application, Figure 5 is a schematic diagram of a fault detection method provided by the present application, such as Figure 4 , Figure 5As shown, first, a sample panoramic image of the urban rail communication machine room is obtained through image data acquisition, and pose data calculation is performed based on the sample panoramic image. The obtained pose data and the sample panoramic image are sent to the fault detection together, wherein the density data and the color data output by the neural radiation field model pre-trained by the MLP model are used for voxel rendering. In the model training update stage, the updated sample is iterated continuously to output a deployable three-dimensional real scene model; then, in the real-time stage, the machine room equipment cabinet panoramic image is captured in real time by the cabinet image acquisition module. The equipment type detection model is used to identify the equipment type of the panoramic image to obtain the equipment type detection results of switches, storage devices, servers and the like. According to this, the corresponding device area image is segmented; then, according to the equipment type, the dedicated indicator light type detection model is selected from the model library as the target indicator light type detection model, and the device area image is input to obtain the indicator light type detection results containing the power indicator light and the hard disk indicator light. Then, the indicator light area image is cropped in the region; finally, the fault state detection is performed based on the indicator light area image to obtain the fault detection result. The result is sent to the communication system centralized alarm through the transmission interface to trigger the machine room equipment fault alarm. At the same time, the three-dimensional real scene monitoring terminal in the machine room realizes visual presentation through the three-dimensional real scene model monitoring, realizing the closed loop from data acquisition, model training, real-time detection to alarm display.

[0081] The urban rail communication machine room three-dimensional real scene operation and maintenance device provided by the present application is described below. The urban rail communication machine room three-dimensional real scene operation and maintenance device described below can be correspondingly referred to the urban rail communication machine room three-dimensional real scene operation and maintenance method described above.

[0082] Based on any of the above embodiments, the present application provides an urban rail communication machine room three-dimensional real scene operation and maintenance device, Figure 6 The urban rail communication machine room three-dimensional real scene operation and maintenance device provided by the present application is described below. The urban rail communication machine room three-dimensional real scene operation and maintenance device described below can be correspondingly referred to the urban rail communication machine room three-dimensional real scene operation and maintenance method described above. Figure 6 As shown, the device comprises: The acquisition unit 610 is configured to acquire a panoramic image of the urban rail communication machine room and determine camera pose data of the panoramic image, wherein the camera pose data comprises position data and direction data of the panoramic image. The input unit 620 is configured to input the position feature of the position data into a first multi-layer perception machine model in a neural radiation field model to obtain density data and information enhancement features output by the first multi-layer perception machine model, and input the direction feature of the direction data and the information enhancement features into a second multi-layer perception machine model in the neural radiation field model to obtain color data output by the second multi-layer perception machine model. The voxel rendering unit 630 is configured to perform voxel rendering according to the density data and the color data to obtain a three-dimensional real scene model. The three-dimensional real scene operation unit 640 is configured to perform equipment fault detection on the panoramic image to obtain an equipment fault detection result, and display a three-dimensional real scene operation result in the three-dimensional real scene model according to the equipment fault detection result.

[0083] The device provided by the embodiment of the present application inputs the position features of the position data into a first multi-layer perception model in the neural radiance field model to obtain density data and information enhancement features. This process utilizes the powerful representation capability of the neural radiance field model to accurately reconstruct the structural information of a three-dimensional scene. Then, the direction features of the direction data and the information enhancement features are input into a second multi-layer perception model to obtain color data. The density data and the color data are combined to perform voxel rendering to generate a high-fidelity three-dimensional real scene model. This real scene model not only contains rich geometric information but also has a realistic visual effect, providing an intuitive scene view for operation personnel. On this basis, the panoramic image is subjected to equipment fault detection to obtain equipment fault detection results, and three-dimensional real scene operation results are displayed in the three-dimensional real scene model according to these results. This combination of fault detection and the three-dimensional real scene model enables operation personnel to quickly locate fault equipment in an intuitive three-dimensional scene and timely understand the fault condition, thereby significantly improving the fault detection efficiency and response speed. At the same time, the automatic processing process reduces manual intervention, reduces operation cost, and improves the overall efficiency and quality of operation work.

[0084] Based on any of the above embodiments, a training unit is further included, and the training unit specifically includes: A sample image acquisition unit is configured to acquire a sample panoramic image of the urban rail communication machine room and a label color of the sample panoramic image, and determine sample camera pose data of the sample panoramic image. The sample camera pose data includes sample position data and sample direction data of the sample panoramic image. A first prediction unit is configured to input sample position features of the sample position data into a first multi-layer perception model in an initial neural radiance field model to obtain predicted density data and predicted information enhancement features output by the first multi-layer perception model. A second prediction unit is configured to input sample direction features of the sample direction data and the predicted information enhancement features into a second multi-layer perception model in the initial neural radiance field model to obtain predicted color data output by the second multi-layer perception model. A predicted rendering result determination unit is configured to perform voxel rendering according to the predicted density data and the predicted color data to obtain a predicted rendering result. A training unit is configured to determine a target loss based on colors in the predicted rendering result and the label color, and train the initial neural radiance field model based on the target loss to obtain the neural radiance field model.

[0085] According to any one of the above embodiments, the determining predicted rendering result unit is specifically configured to: perform voxel rendering according to the accumulated opacity of the sampling points from the start point to the end point of the ray, the predicted density data and the predicted color data, to obtain the predicted rendering result; The ray is a light path in the three-dimensional real scene corresponding to the camera mapping of the initial neural radiance field model.

[0086] According to any one of the above embodiments, the device fault detection includes device type detection, indicator light type detection and fault state detection; The three-dimensional real scene operation and maintenance unit 640 specifically includes: A device type detection unit configured to perform the device type detection on the panoramic image to obtain a device type detection result; An indicator light type detection unit configured to perform image segmentation on the panoramic image according to the device type in the device type detection result to obtain a device region image, and perform the indicator light type detection on the device region image according to the device type to obtain an indicator light type detection result; A fault state detection unit configured to perform image segmentation on the device region image according to the indicator light type in the indicator light type detection result to obtain an indicator light region image, and perform the fault state detection on the indicator light region image according to the device type and the indicator light type to obtain the device fault detection result.

[0087] According to any one of the above embodiments, the indicator light type detection unit is specifically configured to: select a target indicator light type detection model from a plurality of indicator light type detection models according to the device type; input the device region image into the target indicator light type detection model to obtain the indicator light type detection result output by the target indicator light type detection model; The training steps of each indicator light type detection model in the plurality of indicator light type detection models include: obtain a sample device region image and a label indicator light type of the sample device region image; the label indicator light type includes a power indicator light, a health status indicator light, a network port connection state indicator light, a data transmission state indicator light and a rate indicator light; input the sample device region image into each indicator light type detection model to obtain a predicted indicator light type output by each indicator light type detection model; Based on the predicted indicator light type and the label indicator light type, the respective indicator light type detection model is trained respectively to obtain the plurality of indicator light type detection models.

[0088] Based on any of the above embodiments, the fault state detection on the indicator light area image according to the device type and the indicator light type obtains the device fault detection result, including: According to the device type and the indicator light type, a target device state detection model is selected from a plurality of device state detection models; The indicator light area image is input into the target device state detection model to obtain the device fault detection result output by the target device state detection model; The fault state detection unit is specifically configured to: An indicator light area image and a label device state of the indicator light area image are obtained, and the label device state includes a normal operation state, an unused state and a fault state; The indicator light area image is input into the respective device state detection model to obtain a predicted device state output by the respective device state detection model; According to the predicted device state and the label device state, the respective device state detection model is trained respectively to obtain the plurality of device state detection models.

[0089] Based on any of the above embodiments, the acquisition unit 610 is specifically configured to: The panoramic image is input into a motion restoration structure to obtain camera pose data output by the motion restoration structure.

[0090] Figure 7 is a structural schematic diagram of an electronic device provided by the application, as Figure 7As shown, the electronic device can include a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 complete mutual communication through the communications bus 740. The processor 710 can invoke a logical instruction in the memory 730 to execute the urban rail communication machine room three-dimensional real scene operation and maintenance method, which includes: acquiring a panoramic image of an urban rail communication machine room and determining camera pose data of the panoramic image, the camera pose data including position data and direction data of the panoramic image; inputting position features of the position data into a first multi-layer perception machine model in a neural radiation field model to obtain density data and information enhancement features output by the first multi-layer perception machine model, inputting direction features of the direction data and the information enhancement features into a second multi-layer perception machine model in the neural radiation field model to obtain color data output by the second multi-layer perception machine model; performing voxel rendering according to the density data and the color data to obtain a three-dimensional real scene model; performing equipment fault detection on the panoramic image to obtain an equipment fault detection result, and displaying a three-dimensional real scene operation and maintenance result in the three-dimensional real scene model according to the equipment fault detection result.

[0091] In addition, the logical instruction in the memory 730 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0092] In another aspect, the present application also provides a computer program product comprising a computer program, which can be stored on a non-transitory computer-readable storage medium, and the computer program can be executed by a processor to enable a computer to perform the urban rail communication machine room three-dimensional real scene operation and maintenance method provided by the above method. The method comprises: acquiring a panoramic image of an urban rail communication machine room, and determining camera pose data of the panoramic image, wherein the camera pose data comprises position data and direction data of the panoramic image; inputting position features of the position data into a first multi-layer perception model in a neural radiance field model to obtain density data and information enhancement features output by the first multi-layer perception model; inputting direction features of the direction data and the information enhancement features into a second multi-layer perception model in the neural radiance field model to obtain color data output by the second multi-layer perception model; performing voxel rendering according to the density data and the color data to obtain a three-dimensional real scene model; performing equipment fault detection on the panoramic image to obtain an equipment fault detection result; and displaying a three-dimensional real scene operation and maintenance result in the three-dimensional real scene model according to the equipment fault detection result.

[0093] In another aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program can be executed by a processor to implement the urban rail communication machine room three-dimensional real scene operation and maintenance method provided by the above method. The method comprises: acquiring a panoramic image of an urban rail communication machine room, and determining camera pose data of the panoramic image, wherein the camera pose data comprises position data and direction data of the panoramic image; inputting position features of the position data into a first multi-layer perception model in a neural radiance field model to obtain density data and information enhancement features output by the first multi-layer perception model; inputting direction features of the direction data and the information enhancement features into a second multi-layer perception model in the neural radiance field model to obtain color data output by the second multi-layer perception model; performing voxel rendering according to the density data and the color data to obtain a three-dimensional real scene model; performing equipment fault detection on the panoramic image to obtain an equipment fault detection result; and displaying a three-dimensional real scene operation and maintenance result in the three-dimensional real scene model according to the equipment fault detection result.

[0094] The device embodiments described above are merely illustrative, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0095] Those skilled in the art can clearly understand the implementation of the embodiments by means of software and necessary general hardware platforms through the description of the above embodiments, and the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each embodiment or some parts of the embodiment.

[0096] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A three-dimensional real scene operation and maintenance method for a city rail communication machine room, characterized in that, The method comprises the following steps: acquire a panoramic image of a city rail communication machine room, and determine camera pose data of the panoramic image, the camera pose data comprising position data and direction data of the panoramic image; input position features of the position data into a first multi-layer perception model in a neural radiance field model to obtain density data and information enhancement features output by the first multi-layer perception model, and input direction features of the direction data and the information enhancement features into a second multi-layer perception model in the neural radiance field model to obtain color data output by the second multi-layer perception model; perform voxel rendering according to the density data and the color data to obtain a three-dimensional real scene model; perform equipment fault detection on the panoramic image to obtain an equipment fault detection result, and display a three-dimensional real scene operation and maintenance result in the three-dimensional real scene model according to the equipment fault detection result.

2. The urban rail communication machine room three-dimensional real scene operation and maintenance method according to claim 1, characterized in that, The training step of the neural radiance field model comprises the following steps: acquire sample panoramic images of the city rail communication machine room and label colors of the sample panoramic images, and determine sample camera pose data of the sample panoramic images; the sample camera pose data comprises sample position data and sample direction data of the sample panoramic images; input sample position features of the sample position data into a first multi-layer perception model in an initial neural radiance field model to obtain predicted density data and predicted information enhancement features output by the first multi-layer perception model; input sample direction features of the sample direction data and the predicted information enhancement features into a second multi-layer perception model in the initial neural radiance field model to obtain predicted color data output by the second multi-layer perception model; perform voxel rendering according to the predicted density data and the predicted color data to obtain a predicted rendering result; determine a target loss based on colors in the predicted rendering result and the label colors, and train the initial neural radiance field model based on the target loss to obtain the neural radiance field model.

3. The urban rail communication room three-dimensional real scene operation and maintenance method according to claim 2, characterized in that, The voxel rendering according to the predicted density data and the predicted color data to obtain a predicted rendering result comprises the following steps: perform voxel rendering according to cumulative opacity of sampling points from a starting point to an ending point of a ray, the predicted density data and the predicted color data to obtain the predicted rendering result; the ray is a light path in a three-dimensional real scene mapped by a camera corresponding to the initial neural radiance field model.

4. The urban rail communication machine room three-dimensional real scene operation and maintenance method according to any one of claims 1 to 3, characterized in that, The equipment fault detection comprises equipment type detection, indicator light type detection and fault state detection; The equipment fault detection on the panoramic image to obtain an equipment fault detection result comprises the following steps: perform the equipment type detection on the panoramic image to obtain an equipment type detection result; perform image segmentation on the panoramic image according to equipment types in the equipment type detection result to obtain an equipment region image, and perform the indicator light type detection on the equipment region image according to the equipment types to obtain an indicator light type detection result; According to the indicator light type in the indicator light type detection result, the device region image is image segmented to obtain an indicator light region image, and according to the device type and the indicator light type, the indicator light region image is subjected to the fault state detection to obtain the device fault detection result.

5. The urban rail communication machine room three-dimensional real scene operation and maintenance method according to claim 4, characterized in that, The indicator light type detection of the device region image according to the device type comprises: According to the device type, a target indicator light type detection model is selected from a plurality of indicator light type detection models; The device region image is input into the target indicator light type detection model to obtain the indicator light type detection result output by the target indicator light type detection model; The training step of each indicator light type detection model in the plurality of indicator light type detection models comprises: A sample device region image and a label indicator light type of the sample device region image are obtained; the label indicator light type comprises a power indicator light, a health state indicator light, a network port connection state indicator light, a data transmission state indicator light and a rate indicator light; The sample device region image is input into each indicator light type detection model to obtain a predicted indicator light type output by each indicator light type detection model; Based on the predicted indicator light type and the label indicator light type, each indicator light type detection model is trained respectively to obtain the plurality of indicator light type detection models.

6. The urban rail communication machine room three-dimensional real scene operation and maintenance method according to claim 4, characterized in that, The fault state detection of the indicator light region image according to the device type and the indicator light type comprises: According to the device type and the indicator light type, a target device state detection model is selected from a plurality of device state detection models; The indicator light region image is input into the target device state detection model to obtain the device fault detection result output by the target device state detection model; The training step of each device state detection model in the plurality of device state detection models comprises: An indicator light region image and a label device state of the indicator light region image are obtained; the label device state comprises a normal operation state, an unused state and a fault state; The indicator light region image is input into each device state detection model to obtain a predicted device state output by each device state detection model; According to the predicted device state and the label device state, each device state detection model is trained respectively to obtain the plurality of device state detection models.

7. The urban rail communication machine room three-dimensional real scene operation and maintenance method according to any one of claims 1 to 3, characterized in that, The determination of the camera pose data of the panoramic image comprises: The panoramic image is input into a motion recovery structure to obtain the camera pose data output by the motion recovery structure.

8. A three-dimensional real scene operation and maintenance device for a city rail communication machine room, characterized in that, Comprise: An acquisition unit is configured to acquire a panoramic image of a city rail communication machine room and determine camera pose data of the panoramic image, wherein the camera pose data comprises position data and direction data of the panoramic image; An acquisition unit is configured to acquire a panoramic image of a city rail communication machine room and determine camera pose data of the panoramic image, wherein the camera pose data comprises position data and direction data of the panoramic image; The input unit is configured to input position features of the position data into a first multi-layer perception model in the neural radiance field model to obtain density data and information enhancement features output by the first multi-layer perception model, and input direction features of the direction data and the information enhancement features into a second multi-layer perception model in the neural radiance field model to obtain color data output by the second multi-layer perception model. The voxel rendering unit is configured to perform voxel rendering according to the density data and the color data to obtain a three-dimensional real scene model. The three-dimensional real scene operation and maintenance unit is configured to perform equipment fault detection on the panoramic image to obtain an equipment fault detection result, and display a three-dimensional real scene operation and maintenance result in the three-dimensional real scene model according to the equipment fault detection result.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the urban rail communication machine room three-dimensional real scene operation and maintenance method in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the urban rail communication machine room three-dimensional real scene operation and maintenance method in any one of claims 1 to 7.

Images