Indoor distribution system fault location method, device, equipment and readable medium

Abstract

The embodiment of the present invention relates to the field of wireless network maintenance technology and discloses a method for locating faults in an indoor distributed system. The method includes: obtaining alarm information, determining faulty device information based on the alarm information; determining the indoor distributed system corresponding to the faulty device information as the system to be checked; determining a three-dimensional model corresponding to the system to be checked, the three-dimensional model including multiple three-dimensional sampling points, one or more three-dimensional sampling points corresponding to a device in the system to be checked; marking the fault location in the three-dimensional model based on the alarm information; obtaining a field device image sent by an AR device associated with the system to be checked, fusing the marked three-dimensional model with the field device image to obtain an enhanced system image; and displaying the enhanced system image through the AR device. Through the above methods, the embodiment of the present invention improves the accuracy of fault location in the indoor distributed system and reduces the cost of fault maintenance.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of wireless network maintenance, and in particular to a method, apparatus, device and readable medium for locating faults in an indoor distribution system. Background Art

[0002] With the rapid development of the Internet, indoor distribution systems are being used more and more to improve the quality of wireless networks and signal coverage depth.

[0003] The indoor distribution system includes a variety of different devices scattered in various places. Therefore, when a fault occurs, it is necessary to locate the fault of each device in the indoor distribution system, so as to conduct on-site inspection and repair of the specific device fault.

[0004] In the existing technology, when locating a fault in an indoor distribution system, the first step is generally to obtain fault information, then obtain the equipment distribution and configuration data of the indoor distribution system according to the as-built drawings, and manually identify and match the real-life photos of the fault site obtained by the operation and maintenance personnel with the as-built drawings marked with the configuration information of the indoor distribution system, so as to determine the location of the faulty equipment in the real scene.

[0005] On the one hand, this approach results in low efficiency and accuracy in manual matching and identification, as well as high storage costs and inconvenient access to drawings. Furthermore, the indoor distribution system may be located at a later stage due to various reasons, such as modifications, which may cause the actual scene to no longer match the as-built drawings. Furthermore, concealed fault maintenance is difficult to accurately locate based on the drawings, and the damage costs caused by inaccurate positioning are difficult to control. All of these factors result in low accuracy and high positioning costs for indoor distribution system fault location. Summary of the Invention

[0006] In view of the above problems, an embodiment of the present invention provides a method for locating faults in an indoor distribution system, which is used to solve the problems in the prior art of low accuracy and high cost of locating faults in an indoor distribution system.

[0007] According to one aspect of an embodiment of the present invention, a method for locating a fault in an indoor distribution system is provided, the method comprising:

[0008] Obtaining alarm information, and determining faulty device information based on the alarm information;

[0009] Determine the indoor distribution system corresponding to the faulty device information as the system to be checked;

[0010] Determine a three-dimensional model corresponding to the system to be checked, where the three-dimensional model includes a plurality of three-dimensional sampling points, and one or more of the three-dimensional sampling points corresponds to a device in the system to be checked;

[0011] Marking the fault location in the three-dimensional model according to the alarm information;

[0012] Acquire a real-world device image sent by an AR device associated with the system to be checked, and fuse the marked three-dimensional model with the real-world device image to obtain an enhanced system image;

[0013] The enhanced system image is displayed through the AR device.

[0014] In an optional embodiment, the method further comprises:

[0015] Obtaining a scene video sequence corresponding to the system to be checked, wherein the scene video sequence includes multiple frames of depth maps;

[0016] Determine the depth values ​​of the two-dimensional sampling points contained in the depth map of each frame;

[0017] Obtaining camera position parameters corresponding to the scene video sequence;

[0018] Perform back projection based on the depth value of the two-dimensional sampling point and the camera position parameter to obtain the coordinates of the three-dimensional sampling point corresponding to the depth map of each frame;

[0019] The three-dimensional model corresponding to the system to be checked is determined according to the coordinates of the three-dimensional sampling points corresponding to the depth map of each frame.

[0020] In an optional manner, the camera position parameters include a camera intrinsic parameter matrix, a camera rotation matrix, and a camera translation matrix;

[0021] The method further comprises: determining a plurality of feature points in the scene video sequence, and respectively obtaining coordinates of two-dimensional sampling points corresponding to each feature point in the depth map of each frame;

[0022] The camera position parameters and the coordinates of the three-dimensional sampling points corresponding to each of the two-dimensional sampling points are determined by the bundle adjustment algorithm according to the following formula:

[0023]

[0024] Wherein, λ is the preset scale value, K is the camera intrinsic parameter matrix, R is the camera rotation matrix, t is the camera translation matrix, (u i ,v i ) is the coordinate of the two-dimensional sampling point in the depth map corresponding to the three-dimensional sampling point i, (x i ,y i ,z i ) is the coordinate of the three-dimensional sampling point i.

[0025] In an optional embodiment, the method further comprises:

[0026] Acquire at least one acquisition location identifier corresponding to the three-dimensional model, where the acquisition location identifier corresponds to one or more three-dimensional sampling points;

[0027] Obtaining system configuration information of the system to be checked, wherein the system configuration information includes at least one device location identifier;

[0028] Matching the at least one collection location identifier with the at least one device location identifier;

[0029] Each of the collection location identifiers is associated with the device location identifier that matches it.

[0030] In an optional manner, the faulty device information includes a device location identifier of the faulty device, and the method further includes:

[0031] Acquire a three-dimensional sampling point corresponding to a collection position associated with a device location identifier of the faulty device as the fault location;

[0032] The three-dimensional sampling points are marked in the three-dimensional model.

[0033] In an optional manner, the method further comprises:

[0034] Determine the disparity of the three-dimensional sampling points in the depth map of each frame;

[0035] Determine the depth error of each 3D sampling point according to the disparity of each 3D sampling point in two adjacent depth maps;

[0036] Delete the three-dimensional sampling points whose depth error is greater than a preset error threshold.

[0037] In an optional manner, the method further comprises:

[0038] Determining the color values ​​of the three-dimensional sampling points in each of the depth maps;

[0039] Determine the color error of each three-dimensional sampling point according to the color value of the three-dimensional sampling point in two adjacent depth maps;

[0040] Determining the confidence level of each of the three-dimensional sampling points according to the color error;

[0041] The three-dimensional sampling points whose confidence level is less than a preset confidence threshold are deleted.

[0042] According to another aspect of an embodiment of the present invention, a device for locating a fault in an indoor distribution system is provided, comprising:

[0043] An alarm acquisition module is used to obtain alarm information and determine faulty device information based on the alarm information;

[0044] A system positioning module, configured to determine the indoor distribution system corresponding to the faulty device information as the system to be checked;

[0045] a model determination module, configured to determine a three-dimensional model corresponding to the system to be checked, wherein the three-dimensional model includes a plurality of three-dimensional sampling points, and one or more of the three-dimensional sampling points corresponds to a device in the system to be checked;

[0046] A fault location module, configured to mark the fault location in the three-dimensional model according to the alarm information;

[0047] a model matching module, configured to obtain a physical device image sent by an AR device associated with the system to be checked, and fuse the marked three-dimensional model with the physical device image to obtain an enhanced system image;

[0048] An image display module displays the enhanced system image through the AR device.

[0049] According to another aspect of an embodiment of the present invention, there is provided an indoor distribution system fault location device, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0050] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the indoor distribution system fault location method as described in any one of the aforementioned embodiments.

[0051] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which at least one executable instruction is stored. When the executable instruction is executed on an indoor distributed system fault locating device / apparatus, the indoor distributed system fault locating device / apparatus performs the operation of the indoor distributed system fault locating method as described in any of the aforementioned embodiments.

[0052] An embodiment of the present invention obtains alarm information and determines faulty device information based on the alarm information. Then, the indoor distribution system corresponding to the faulty device information is identified as the system to be troubleshooted. A three-dimensional model corresponding to the system to be troubleshooted is then determined. The three-dimensional model includes multiple three-dimensional sampling points, one or more of which corresponds to a device in the system to be troubleshooted. The fault location is marked in the three-dimensional model based on the alarm information. Finally, an image of the actual device is obtained from an AR device associated with the system to be troubleshooted. The marked three-dimensional model is fused with the actual device image to obtain an enhanced system image. The enhanced system image is then displayed using the AR device.

[0053] This is different from the existing technology that manually compares equipment installation information and fault alarm information in construction drawings. The present invention can mark the fault location based on the three-dimensional model corresponding to the indoor distribution system, and fuse the marked three-dimensional model with the actual scene image and display it to on-site troubleshooting personnel through AR equipment, thereby improving the accuracy and efficiency of fault location at indoor distribution sites and reducing the cost of positioning and troubleshooting.

[0054] The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to more clearly understand the technical means of the embodiments of the present invention, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the drawings. In the drawings:

[0056] Figure 1 A schematic diagram showing a flow chart of a fault location method for an indoor distribution system provided by an embodiment of the present invention is shown;

[0057] Figure 2 A schematic structural diagram of a fault location device for an indoor distribution system provided by an embodiment of the present invention is shown;

[0058] Figure 3 A schematic structural diagram of a fault location device for an indoor distribution system provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0059] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0060] Figure 1 The flowchart of the embodiment of the method for locating the fault of the indoor distribution system of the present invention is shown. The method is executed by a computer processing device. The specific computer processing device may include a laptop computer, a mobile phone, etc. Figure 1 As shown, the method includes at least steps 110-160.

[0061] Step 110: Acquire alarm information, and determine faulty device information based on the alarm information.

[0062] The alarm information is obtained by the preset network management center and may include the name of the system where the fault occurred, the time of the fault, the longitude and latitude of the system distribution, the name and number of devices installed in the system, the name of the device that generated the alarm, and detailed alarm type.

[0063] The faulty device information may include the device fault type, device fault occurrence time, device name, device system identifier, device installation location, etc.

[0064] Step 120: Determine the indoor distribution system corresponding to the faulty device information as the system to be checked.

[0065] First of all, fault location must first be located in the specific indoor distribution system. The indoor distribution system mainly consists of two parts: the donor signal source from various network standards and the signal distribution system. Among them, the donor signal source includes base stations, base station remote equipment, wireless or wired relay equipment, etc. The indoor signal distribution system consists of active devices, passive devices, antennas, cables, etc.

[0066] The specific faulty device may be an antenna or a BBU, etc. Therefore, after receiving the alarm information of the corresponding device, the indoor distribution system associated with the faulty device identifier is obtained as the system to be checked.

[0067] Step 130: Determine a three-dimensional model corresponding to the system to be checked, where the three-dimensional model includes a plurality of three-dimensional sampling points, and one or more of the three-dimensional sampling points corresponds to a device in the system to be checked.

[0068] The 3D model is a digital reproduction of the real scene of the system to be checked, which includes the location and installation information of each device in the system to be checked. To identify a device, at least one 3D sampling point corresponding to it in the 3D model must be determined.

[0069] To determine the three-dimensional model, we can first use a motion camera to collect a series of continuous depth maps of the system to be inspected (i.e., a video sequence), and then perform back-projection based on each pixel point in the depth map to determine its corresponding three-dimensional point coordinates in space, thereby constructing a three-dimensional model based on all the three-dimensional points corresponding to the system to be inspected.

[0070] The steps for determining the three-dimensional model may include: Figure 2 Steps 1301-1305 are shown.

[0071] Step 1301: Obtain a scene video sequence corresponding to the system to be checked, where the scene video sequence includes multiple frames of depth maps.

[0072] First, the scene video sequence is captured by a motion camera. During the capture, each system to be checked corresponds to a scene video sequence.

[0073] Step 1302: Determine the depth values ​​of the two-dimensional sampling points included in the depth map of each frame.

[0074] A depth map is a two-dimensional image in which each pixel has two-dimensional coordinates and depth information.

[0075] Step 1303: Obtain camera position parameters corresponding to the scene video sequence.

[0076] First, the camera position parameters here include at least the camera intrinsic parameter matrix, camera rotation matrix and camera translation matrix of the camera that captures the scene sequence, which are used to determine the position and posture of the camera coordinate system in the world coordinate system.

[0077] According to the position parameters of the camera when capturing the scene video sequence, the coordinates of the two-dimensional points in each depth map contained in the captured video sequence can be back-projected to obtain the corresponding three-dimensional points in space.

[0078] The specific process of obtaining the camera position parameters corresponding to the scene video sequence may further include steps 13031-13032:

[0079] Step 13031: Determine multiple feature points in the scene video sequence, and obtain the coordinates of the two-dimensional sampling points corresponding to each feature point in the depth map of each frame.

[0080] Each point in the scene video sequence corresponds to a two-dimensional sampling point in the depth map of each frame. It is easy to understand that since the video sequence is composed of a series of continuously moving points, the two-dimensional sampling point coordinates and depth values ​​corresponding to the same point in the video in different depth maps are constantly changing during the movement.

[0081] Feature points can be determined based on the equipment composition of each system to be checked, and can be used to calibrate the coordinate points of each device in the system to be located, such as the coordinate points of the installation locations of BBU, RRU, 1 / 2 feeder, 7 / 8 feeder, ceiling antenna, etc.

[0082] Step 13032: Determine the camera position parameters and the coordinates of the three-dimensional sampling points corresponding to each of the two-dimensional sampling points using a bundle adjustment algorithm according to the following formula based on the coordinates of the two-dimensional sampling points:

[0083]

[0084] Wherein, λ is the preset scale value, K is the camera intrinsic parameter matrix, R is the camera rotation matrix, t is the camera translation matrix, (u i ,v i ) is the coordinate of the two-dimensional sampling point in the depth map corresponding to the three-dimensional sampling point i, (xi ,y i ,z i ) is the coordinate of the three-dimensional sampling point i.

[0085] The bundle adjustment algorithm optimizes a function to minimize reprojection error. Since two images have many pairs of feature points, the projection errors of all feature points are summed, squared, and multiplied by 1 / 2. This constructs a nonlinear least-squares problem. The poses and spatial points involved are used as the optimization targets (i.e., the objects to be solved). By minimizing the reprojection error, an optimized result is obtained: the coordinates of the 3D points and the camera position parameters that minimize the reprojection error.

[0086] Step 1304: performing back-projection based on the depth values ​​of the two-dimensional sampling points and the camera position parameters to obtain coordinates of the three-dimensional sampling points corresponding to the depth map of each frame.

[0087] Projection is the process of mapping a point in space to a pixel in an image. Back-projection, or 3D sampling points, is the process of determining the position of each device in the field system in 3D space by reversing the position of the points in the collected 2D depth map.

[0088] Considering that there may be movement of the camera position or noise in the actual sampling process, it is necessary to screen the three-dimensional points and eliminate the points with large reprojection errors among the three-dimensional sampling points.

[0089] Furthermore, after determining the coordinates of the three-dimensional sampling points, the three-dimensional sampling points need to be screened and duplicate points and error points removed. Optionally, steps 130410 to 130412 may be included.

[0090] Step 130410: Determine the disparity of the three-dimensional sampling points in the depth map of each frame.

[0091] Parallax is the derivative of depth. Parallax reflects the different distances (i.e., depths) of the 3D sampling points from the camera. A larger parallax indicates a 3D sampling point is closer to the camera.

[0092] Step 130411: Determine the depth error of each 3D sampling point according to the disparity of the 3D sampling point in two adjacent depth maps.

[0093] First, the difference in disparity between the three-dimensional sampling points in two adjacent depth maps is determined.

[0094] When a point is in constant motion, the depth values ​​of the corresponding sampling points in different frames should be close. Therefore, the average value of the disparity difference of each 3D sampling point in each adjacent frame can be used as the depth error corresponding to the 3D sampling point.

[0095] Therefore, if the parallax of a 3D sampling point in consecutive frames changes greatly, it may indicate that the acquisition of the point has been disturbed or there is an error in the acquired data, and the 3D sampling point needs to be eliminated.

[0096] Step 130412: Delete the three-dimensional sampling points whose depth error is greater than a preset error threshold.

[0097] The preset error threshold can be based on the construction of the historical three-dimensional model.

[0098] It may also include steps 130420 to 130423.

[0099] Step 130420: Determine the color values ​​of the three-dimensional sampling points in each depth map.

[0100] The color value refers to the RGB value of each 3D sampling point in the depth map.

[0101] Step 130421: Determine the color error of each 3D sampling point according to the color value of the 3D sampling point in the depth map of two adjacent frames.

[0102] Similar to the calculation of the depth error in step 130411, firstly, the difference between the color values ​​of each of the three-dimensional sampling points in the depth maps of two adjacent frames is determined.

[0103] The color values ​​of the sampling points corresponding to a point in different frames should be close during the continuous movement of the point. Therefore, the average value of the color value difference of each 3D sampling point in each adjacent frame can be used as the color error corresponding to the 3D sampling point.

[0104] Therefore, if the color value of a 3D sampling point changes greatly in consecutive frames, it may indicate that the acquisition of the point has been disturbed or there is an error in the acquired data, and the 3D sampling point needs to be eliminated.

[0105] Step 130422: Determine the confidence level of each of the three-dimensional sampling points based on the color error.

[0106] According to the preset metric calculation formula, the confidence level is calculated based on the proportional relationship between the color error and the confidence level.

[0107] Step 130423: Delete the three-dimensional sampling points whose confidence level is less than a preset confidence threshold.

[0108] Step 1305: Determine the three-dimensional model corresponding to the system to be checked according to the coordinates of the three-dimensional sampling points corresponding to the depth map of each frame.

[0109] The coordinates of the 3D sampling points determined in the previous steps only include the 3D coordinates of the feature points. In order to make the 3D model closer to the actual scene, that is, to include all points in the indoor distributed system and obtain a 3D model that is completely restored 1:1 with reality, dense reconstruction is also required based on the 3D sampling points that determined the feature points.

[0110] Specifically, dense reconstruction refers to converting the coordinates of the two-dimensional sampling points of the remaining large number of unmatched indoor distribution systems collected by the depth camera according to the pose parameters of the camera coordinate system obtained in step 1303 to obtain the three-dimensional coordinates of each non-feature point.

[0111] Finally, the three-dimensional coordinates of all points in the indoor distributed sites are fused to obtain a dense point cloud corresponding to the site to be checked as the three-dimensional model.

[0112] Optionally, after determining the dense point cloud including all sampling points, in order to completely restore the scene to facilitate the location of indoor distributed base station faults, the indoor distributed base station fault problems in the network management can also be marked on the dense point cloud. Because each 3D point corresponds to many 2D points on the image, and the 2D points on the image already carry relevant color information, these color information can be directly assigned to the 3D points to obtain a dense textured scene, and the size of the scene is 1:1 with the actual scene.

[0113] After obtaining a 1:1 dense point cloud model of the actual scene, the fault location can be marked according to the alarm information.

[0114] Step 140: Mark the fault location in the three-dimensional model according to the alarm information.

[0115] It is easy to understand that before fault location can be performed, field sampling is required. This involves matching the data collected by the depth camera with the construction drawings and the field scene configuration information for completion acceptance. This involves associating each sampling point in the 3D model with the field equipment information.

[0116] Step 140 may further include steps 1401-1404:

[0117] Step 1401: Acquire at least one acquisition location identifier corresponding to the three-dimensional model, where the acquisition location identifier corresponds to one or more three-dimensional sampling points.

[0118] Collection location identifier: If a system to be checked includes at least one device, at least one or more three-dimensional sampling points are needed to locate the device, and one device corresponds to a collection area, and the collection area corresponds to a collection location identifier.

[0119] For example, in an indoor distribution system A, there are three devices: the first BBU, the second BBU, and the third BBU. Each device corresponds to a specific collection location identifier, such as A001, A002, and A003. A001 can correspond to 10 three-dimensional sampling points.

[0120] Step 1402: Obtain system configuration information of the system to be checked, wherein the system configuration information includes at least one device location identifier.

[0121] The system configuration information may be the equipment information installed in the indoor distribution site collected when the indoor distribution system is completed and delivered, specifically including the types of equipment included in various indoor distribution systems such as main equipment, remote BBU, RRU, feeder, antenna, etc., the number of various devices, and the installation location of each device.

[0122] Each installation location corresponds to a specific device location identifier, which is used to indicate the latitude and longitude of the current location, as well as the device type, device name, and system identifier of the device installed at the latitude and longitude.

[0123] Step 1403: Match the at least one collection location identifier with the at least one device location identifier.

[0124] Step 1404: Associating each of the collection location identifiers with the device location identifier that matches it.

[0125] After the device information corresponding to each three-dimensional point is determined, association can be performed based on the device location identifier corresponding to the faulty device in the alarm information.

[0126] The process of marking the fault location according to the alarm information further includes steps 1411-1412:

[0127] Step 1411: Acquire a three-dimensional sampling point corresponding to the acquisition position associated with the device location identifier of the faulty device as the fault location.

[0128] The fault location may correspond to one or more three-dimensional sampling points.

[0129] Step 1412: Mark the three-dimensional sampling points in the three-dimensional model.

[0130] The specific marking form may be in text form, such as displaying the device identification, device name, and device type corresponding to a certain three-dimensional sampling point above the three-dimensional sampling point.

[0131] Step 150: Acquire a real device image sent by an AR device associated with the system to be checked, fuse the marked three-dimensional model with the real device image, and obtain an enhanced system image.

[0132] Fusion is performed by superimposing a three-dimensional model on the actual device image for display, so that the superposition of the actual device image and the three-dimensional model image can be seen through the AR device. The user of the AR device can find the corresponding point in the actual device image through the device location marked in the three-dimensional model.

[0133] Optionally, before fusion, a display message can be sent to the AR device to guide the user to align the manually determined feature points in the actual device image with the feature points in the three-dimensional model, so that the three-dimensional model can be superimposed on the actual device image.

[0134] Specifically, after obtaining the actual device image, the user can be guided to move the AR device according to the feature point positions in the actual device image reported by the AR device, so as to align the three-dimensional model with the actual device image, and then overlay and fuse them.

[0135] Step 160: Display the enhanced system image through the AR device.

[0136] Fusion is the display of a three-dimensional model based on the actual device image, so that the overlay of the actual device image and the three-dimensional model image can be seen through the AR device. The user of the AR device can find the corresponding point in the actual device image through the device location marked in the three-dimensional model.

[0137] In an optional embodiment, after step 160, the geographical location of the faulty device may be further determined based on the alarm information and sent to an AR device or an operation and maintenance personnel device associated with the system to be checked.

[0138] In another optional method, a fault solution is determined based on the alarm information, marked and annotated in the enhanced system image, and then sent back to the AR device. This allows maintenance personnel to quickly locate the faulty device and, with the guidance of remote experts, can further identify the fault solution after a BBU fault is marked in the enhanced system image. This further improves the troubleshooting speed of the indoor distribution system and saves manpower and time.

[0139] Figure 2 FIG. 1 shows a schematic diagram of the structure of an embodiment of a fault location device for an indoor distribution system according to the present invention. Figure 2 As shown, the device 200 includes: an alarm acquisition module 210 , a system positioning module 220 , a model determination module 230 , a fault positioning module 240 , a model matching module 250 , and an image display module 260 .

[0140] An alarm acquisition module 210 is configured to acquire alarm information and determine faulty device information based on the alarm information;

[0141] The system positioning module 220 is used to determine the indoor distribution system corresponding to the faulty device information as the system to be checked;

[0142] A model determination module 230 is configured to determine a three-dimensional model corresponding to the system to be checked, wherein the three-dimensional model includes a plurality of three-dimensional sampling points, and one or more of the three-dimensional sampling points corresponds to a device in the system to be checked;

[0143] A fault location module 240 is configured to mark a fault location in the three-dimensional model according to the alarm information;

[0144] The model matching module 250 is configured to obtain a physical device image sent by an AR device associated with the system to be inspected, and fuse the marked three-dimensional model with the physical device image to obtain an enhanced system image.

[0145] The image display module 260 displays the enhanced system image through the AR device.

[0146] In an optional manner, the fault location module 240 is further configured to:

[0147] Acquire at least one acquisition location identifier corresponding to the three-dimensional model, where the acquisition location identifier corresponds to one or more three-dimensional sampling points;

[0148] Obtaining system configuration information of the system to be checked, wherein the system configuration information includes at least one device location identifier;

[0149] Matching the at least one collection location identifier with the at least one device location identifier;

[0150] Each of the collection location identifiers is associated with the device location identifier that matches it.

[0151] In an optional manner, the fault location module 240 is further configured to:

[0152] Acquire a three-dimensional sampling point corresponding to a collection position associated with a device location identifier of the faulty device as the fault location;

[0153] The three-dimensional sampling points are marked in the three-dimensional model.

[0154] In an optional manner, the model determination module 260 is further configured to:

[0155] Obtaining a scene video sequence corresponding to the system to be checked, wherein the scene video sequence includes multiple frames of depth maps;

[0156] Determine the depth values ​​of the two-dimensional sampling points contained in the depth map of each frame;

[0157] Obtaining camera position parameters corresponding to the scene video sequence;

[0158] Perform back projection based on the depth value of the two-dimensional sampling point and the camera position parameter to obtain the coordinates of the three-dimensional sampling point corresponding to the depth map of each frame;

[0159] The three-dimensional model corresponding to the system to be checked is determined according to the coordinates of the three-dimensional sampling points corresponding to the depth map of each frame.

[0160] In an optional manner, the model determination module 260 is further configured to:

[0161] Determine a plurality of feature points in the scene video sequence, and respectively obtain coordinates of two-dimensional sampling points corresponding to each feature point in the depth map of each frame;

[0162] The camera position parameters and the coordinates of the three-dimensional sampling points corresponding to each of the two-dimensional sampling points are determined by the bundle adjustment algorithm according to the following formula:

[0163]

[0164] Wherein, λ is the preset scale value, K is the camera intrinsic parameter matrix, R is the camera rotation matrix, t is the camera translation matrix, (u i ,v i ) is the coordinate of the two-dimensional sampling point in the depth map corresponding to the three-dimensional sampling point i, (x i ,y i ,z i ) is the coordinate of the three-dimensional sampling point i.

[0165] In an optional manner, the model determination module 260 is further configured to:

[0166] Determining the color values ​​of the three-dimensional sampling points in each of the depth maps;

[0167] Determine the color error of each three-dimensional sampling point according to the color value of the three-dimensional sampling point in two adjacent depth maps;

[0168] Determining the confidence level of each of the three-dimensional sampling points according to the color error;

[0169] The three-dimensional sampling points whose confidence level is less than a preset confidence threshold are deleted.

[0170] In an optional manner, the model determination module 260 is further configured to:

[0171] Determining the color values ​​of the three-dimensional sampling points in each of the depth maps;

[0172] Determine the color error of each three-dimensional sampling point according to the color value of the three-dimensional sampling point in two adjacent depth maps;

[0173] Determining the confidence level of each of the three-dimensional sampling points according to the color error;

[0174] The three-dimensional sampling points whose confidence level is less than a preset confidence threshold are deleted.

[0175] The specific working process of the fault location device of the indoor distribution system according to the embodiment of the present invention is the same as the specific process steps of the fault location method of the indoor distribution system described above, and will not be repeated here.

[0176] The fault location device of the indoor distribution system of an embodiment of the present invention marks the fault location based on the three-dimensional model corresponding to the indoor distribution system, and fuses the marked three-dimensional model with the real scene image and displays it to on-site troubleshooting personnel through AR equipment, thereby improving the accuracy and efficiency of fault location of indoor distribution sites and reducing the cost of positioning and troubleshooting.

[0177] Figure 3 The schematic diagram of the structure of an embodiment of a fault location device for an indoor distribution system of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the fault location device for an indoor distribution system.

[0178] like Figure 3 As shown, the fault location device of the indoor distribution system may include: a processor (processor) 402 , a communications interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0179] Processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408. Communication interface 404 is used to communicate with other devices, such as clients or other server network elements. Processor 402 is used to execute program 410, which may specifically perform the steps described in the aforementioned embodiment of the fault location method for an indoor distributed system.

[0180] Specifically, the program 410 may include program code including computer-executable instructions.

[0181] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the fault location device of the indoor distribution system may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0182] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0183] The program 410 may be specifically called by the processor 402 to enable the fault location device of the indoor distribution system to perform the following operations:

[0184] Obtaining alarm information, and determining faulty device information based on the alarm information;

[0185] Determine the indoor distribution system corresponding to the faulty device information as the system to be checked;

[0186] Determine a three-dimensional model corresponding to the system to be checked, where the three-dimensional model includes a plurality of three-dimensional sampling points, and one or more of the three-dimensional sampling points corresponds to a device in the system to be checked;

[0187] Marking the fault location in the three-dimensional model according to the alarm information;

[0188] Acquire a real-world device image sent by an AR device associated with the system to be checked, and fuse the marked three-dimensional model with the real-world device image to obtain an enhanced system image;

[0189] The enhanced system image is displayed through the AR device.

[0190] In an optional manner, the program 410 may be specifically called by the processor 402 to enable the fault location device of the indoor distribution system to perform the following operations:

[0191] Obtaining a scene video sequence corresponding to the system to be checked, wherein the scene video sequence includes multiple frames of depth maps;

[0192] Determine the depth values ​​of the two-dimensional sampling points contained in the depth map of each frame;

[0193] Obtaining camera position parameters corresponding to the scene video sequence;

[0194] Perform back projection based on the depth value of the two-dimensional sampling point and the camera position parameter to obtain the coordinates of the three-dimensional sampling point corresponding to the depth map of each frame;

[0195] The three-dimensional model corresponding to the system to be checked is determined according to the coordinates of the three-dimensional sampling points corresponding to the depth map of each frame.

[0196] In an optional manner, the program 410 may be specifically called by the processor 402 to enable the fault location device of the indoor distribution system to perform the following operations:

[0197] Determine a plurality of feature points in the scene video sequence, and respectively obtain coordinates of two-dimensional sampling points corresponding to each feature point in the depth map of each frame;

[0198] The camera position parameters and the coordinates of the three-dimensional sampling points corresponding to each of the two-dimensional sampling points are determined by the bundle adjustment algorithm according to the following formula:

[0199]

[0200] Wherein, λ is the preset scale value, K is the camera intrinsic parameter matrix, R is the camera rotation matrix, t is the camera translation matrix, (u i ,v i ) is the coordinate of the two-dimensional sampling point in the depth map corresponding to the three-dimensional sampling point i, (x i ,y i ,z i ) is the coordinate of the three-dimensional sampling point i.

[0201] In an optional manner, the program 410 may be specifically called by the processor 402 to enable the fault location device of the indoor distribution system to perform the following operations:

[0202] Acquire at least one acquisition location identifier corresponding to the three-dimensional model, where the acquisition location identifier corresponds to one or more three-dimensional sampling points;

[0203] Obtaining system configuration information of the system to be checked, wherein the system configuration information includes at least one device location identifier;

[0204] Matching the at least one collection location identifier with the at least one device location identifier;

[0205] Each of the collection location identifiers is associated with the device location identifier that matches it.

[0206] In an optional manner, the program 410 may be specifically called by the processor 402 to enable the fault location device of the indoor distribution system to perform the following operations:

[0207] Acquire a three-dimensional sampling point corresponding to a collection position associated with a device location identifier of the faulty device as the fault location;

[0208] The three-dimensional sampling points are marked in the three-dimensional model.

[0209] In an optional manner, the program 410 may be specifically called by the processor 402 to enable the fault location device of the indoor distribution system to perform the following operations:

[0210] Determine the disparity of the three-dimensional sampling points in the depth map of each frame;

[0211] Determine the depth error of each 3D sampling point according to the disparity of each 3D sampling point in two adjacent depth maps;

[0212] Delete the three-dimensional sampling points whose depth error is greater than a preset error threshold.

[0213] In an optional manner, the program 410 may be specifically called by the processor 402 to enable the fault location device of the indoor distribution system to perform the following operations:

[0214] Determining the color values ​​of the three-dimensional sampling points in each of the depth maps;

[0215] Determine the color error of each three-dimensional sampling point according to the color value of the three-dimensional sampling point in two adjacent depth maps;

[0216] Determining the confidence level of each of the three-dimensional sampling points according to the color error;

[0217] The three-dimensional sampling points whose confidence level is less than a preset confidence threshold are deleted.

[0218] The specific working process of the fault location device of the indoor distribution system in the embodiment of the present invention is the same as the specific process steps of the fault location method of the indoor distribution system described above, and will not be repeated here.

[0219] The fault location device of the indoor distribution system of an embodiment of the present invention marks the fault location based on the three-dimensional model corresponding to the indoor distribution system, and fuses the marked three-dimensional model with the real scene image and displays it to on-site troubleshooting personnel through an AR device, thereby improving the accuracy and efficiency of fault location of indoor distribution sites and reducing the cost of positioning and troubleshooting.

[0220] An embodiment of the present invention provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is run on a fault location device / apparatus of an indoor distribution system, the fault location device / apparatus of the indoor distribution system executes the fault location method of the indoor distribution system in any of the above-mentioned method embodiments.

[0221] The executable instructions may be specifically used to enable the fault location device / apparatus of the indoor distribution system to perform the following operations:

[0222] Obtaining alarm information, and determining faulty device information based on the alarm information;

[0223] Determine the indoor distribution system corresponding to the faulty device information as the system to be checked;

[0224] Determine a three-dimensional model corresponding to the system to be checked, where the three-dimensional model includes a plurality of three-dimensional sampling points, and one or more of the three-dimensional sampling points corresponds to a device in the system to be checked;

[0225] Marking the fault location in the three-dimensional model according to the alarm information;

[0226] Acquire a real-world device image sent by an AR device associated with the system to be checked, and fuse the marked three-dimensional model with the real-world device image to obtain an enhanced system image;

[0227] The enhanced system image is displayed through the AR device.

[0228] In an optional manner, the executable instructions are further used to enable the fault location device / apparatus of the indoor distribution system to perform the following operations:

[0229] Obtaining a scene video sequence corresponding to the system to be checked, wherein the scene video sequence includes multiple frames of depth maps;

[0230] Determine the depth values ​​of the two-dimensional sampling points contained in the depth map of each frame;

[0231] Obtaining camera position parameters corresponding to the scene video sequence;

[0232] Perform back projection based on the depth value of the two-dimensional sampling point and the camera position parameter to obtain the coordinates of the three-dimensional sampling point corresponding to the depth map of each frame;

[0233] The three-dimensional model corresponding to the system to be checked is determined according to the coordinates of the three-dimensional sampling points corresponding to the depth map of each frame.

[0234] In an optional manner, the executable instructions are further used to enable the fault location device / apparatus of the indoor distribution system to perform the following operations:

[0235] Determine a plurality of feature points in the scene video sequence, and respectively obtain coordinates of two-dimensional sampling points corresponding to each feature point in the depth map of each frame;

[0236] The camera position parameters and the coordinates of the three-dimensional sampling points corresponding to each of the two-dimensional sampling points are determined by the bundle adjustment algorithm according to the following formula:

[0237]

[0238] Wherein, λ is the preset scale value, K is the camera intrinsic parameter matrix, R is the camera rotation matrix, t is the camera translation matrix, (u i ,v i ) is the coordinate of the two-dimensional sampling point in the depth map corresponding to the three-dimensional sampling point i, (x i ,y i ,z i ) is the coordinate of the three-dimensional sampling point i.

[0239] In an optional manner, the executable instructions are further used to enable the fault location device / apparatus of the indoor distribution system to perform the following operations:

[0240] Acquire at least one acquisition location identifier corresponding to the three-dimensional model, where the acquisition location identifier corresponds to one or more three-dimensional sampling points;

[0241] Obtaining system configuration information of the system to be checked, wherein the system configuration information includes at least one device location identifier;

[0242] Matching the at least one collection location identifier with the at least one device location identifier;

[0243] Each of the collection location identifiers is associated with the device location identifier that matches it.

[0244] In an optional manner, the executable instructions are further used to enable the fault location device / apparatus of the indoor distribution system to perform the following operations:

[0245] Acquire a three-dimensional sampling point corresponding to a collection position associated with a device location identifier of the faulty device as the fault location;

[0246] The three-dimensional sampling points are marked in the three-dimensional model.

[0247] In an optional manner, the executable instructions are further used to enable the fault location device / apparatus of the indoor distribution system to perform the following operations:

[0248] Determine the disparity of the three-dimensional sampling points in the depth map of each frame;

[0249] Determine the depth error of each 3D sampling point according to the disparity of each 3D sampling point in two adjacent depth maps;

[0250] Delete the three-dimensional sampling points whose depth error is greater than a preset error threshold.

[0251] In an optional manner, the executable instructions are further used to enable the fault location device / apparatus of the indoor distribution system to perform the following operations:

[0252] Determining the color values ​​of the three-dimensional sampling points in each of the depth maps;

[0253] Determine the color error of each three-dimensional sampling point according to the color value of the three-dimensional sampling point in two adjacent depth maps;

[0254] Determining the confidence level of each of the three-dimensional sampling points according to the color error;

[0255] The three-dimensional sampling points whose confidence level is less than a preset confidence threshold are deleted.

[0256] The specific working process of the computer-readable storage medium in the embodiment of the present invention is the same as the specific process steps of the fault location method of the indoor distribution system mentioned above, and will not be repeated here.

[0257] The computer-readable storage medium of an embodiment of the present invention marks the fault location based on the three-dimensional model corresponding to the indoor distribution system, and fuses the marked three-dimensional model with the real scene image and displays it to on-site troubleshooting personnel through an AR device, thereby improving the accuracy and efficiency of fault location at indoor distribution sites and reducing the cost of positioning and troubleshooting.

[0258] An embodiment of the present invention provides a computer program, which can be called by a processor to cause an indoor distributed system fault location device to execute the indoor distributed system fault location method in any of the above method embodiments.

[0259] An embodiment of the present invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are run on a computer, the computer executes the indoor distribution system fault location method in any of the above method embodiments.

[0260] The algorithm or demonstration provided herein are not inherently relevant to any particular computer, virtual system or other equipment. Various general-purpose systems may also be used together with the teachings based on this. According to the above description, it is apparent that the structure required for constructing this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0261] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0262] Similarly, it should be understood that in order to streamline the present invention and facilitate understanding of one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0263] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed so far can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) can be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0264] It should be noted that the above embodiments illustrate rather than limit the invention, and that alternative embodiments may be devised by a person skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments should not be understood as limiting the order of execution unless otherwise specified.

Claims

1. A method for locating faults in an indoor distribution system, characterized in that: The method comprises: Obtaining alarm information, and determining faulty device information based on the alarm information; Determine the indoor distribution system corresponding to the faulty device information as the system to be checked; Determine a three-dimensional model corresponding to the system to be checked, where the three-dimensional model includes a plurality of three-dimensional sampling points, and one or more of the three-dimensional sampling points corresponds to a device in the system to be checked; Marking the fault location in the three-dimensional model according to the alarm information; Acquire a real-world device image sent by an AR device associated with the system to be troubleshooted, fuse the marked three-dimensional model with the real-world device image, and obtain an enhanced system image; wherein, before the fusion is performed, send a display message to the AR device to instruct the user to align manually determined feature points in the real-world device image with feature points in the three-dimensional model, so that the three-dimensional model is superimposed on the real-world device image; after acquiring the real-world device image, instruct the user to move the AR device according to the positions of the feature points in the actual device image reported by the AR device, so as to align the three-dimensional model with the real-world device image, and then perform superposition and fusion; Displaying the enhanced system image through the AR device; The geographic location of the faulty device is determined based on the alarm information and sent to the AR device or the operation and maintenance personnel device associated with the system to be checked.

2. The method according to claim 1, characterized in that The determining of the three-dimensional model corresponding to the system to be checked further includes: Obtaining a scene video sequence corresponding to the system to be checked, wherein the scene video sequence includes multiple frames of depth maps; Determine the depth values ​​of the two-dimensional sampling points contained in the depth map of each frame; Obtaining camera position parameters corresponding to the scene video sequence; Perform back projection based on the depth value of the two-dimensional sampling point and the camera position parameter to obtain the coordinates of the three-dimensional sampling point corresponding to the depth map of each frame; The three-dimensional model corresponding to the system to be checked is determined according to the coordinates of the three-dimensional sampling points corresponding to the depth map of each frame.

3. The method according to claim 2, characterized in that The camera position parameters include camera intrinsic parameter matrix, camera rotation matrix and camera translation matrix; The obtaining of camera position parameters corresponding to the scene video sequence further includes: Determine a plurality of feature points in the scene video sequence, and respectively obtain coordinates of two-dimensional sampling points corresponding to each feature point in the depth map of each frame; The camera position parameters and the coordinates of the three-dimensional sampling points corresponding to each of the two-dimensional sampling points are determined by the bundle adjustment algorithm according to the following formula: Wherein, λ is the preset scale value, K is the camera intrinsic parameter matrix, R is the camera rotation matrix, t is the camera translation matrix, (u i ,v i ) is the coordinate of the two-dimensional sampling point in the depth map corresponding to the three-dimensional sampling point i, (x i ,y i ,z i ) is the coordinate of the three-dimensional sampling point i.

4. The method according to claim 1 or 2, characterized in that Before marking the fault location in the three-dimensional model according to the alarm information, the method further includes: Acquire at least one acquisition location identifier corresponding to the three-dimensional model, where the acquisition location identifier corresponds to one or more three-dimensional sampling points; Obtaining system configuration information of the system to be checked, wherein the system configuration information includes at least one device location identifier; Matching the at least one collection location identifier with the at least one device location identifier; Each of the collection location identifiers is associated with the device location identifier that matches it.

5. The method according to claim 1, wherein The faulty device information includes a device location identifier of the faulty device, and marking the fault location in the three-dimensional model according to the alarm information further includes: Acquire a three-dimensional sampling point corresponding to a collection position associated with a device location identifier of the faulty device as the fault location; The three-dimensional sampling points are marked in the three-dimensional model.

6. The method according to claim 2, characterized in that Before determining the three-dimensional model corresponding to the system to be checked according to the three-dimensional sampling points corresponding to the depth map of each frame, the method further includes: Determine the disparity of the three-dimensional sampling points in the depth map of each frame; Determine the depth error of each 3D sampling point according to the disparity of each 3D sampling point in two adjacent depth maps; Delete the three-dimensional sampling points whose depth error is greater than a preset error threshold.

7. The method according to claim 2, characterized in that Before determining the three-dimensional model corresponding to the system to be checked according to the three-dimensional sampling points corresponding to the depth map of each frame, the method further includes: Determine the color value of the three-dimensional sampling point in the depth map of each frame; Determine the color error of each three-dimensional sampling point according to the color value of the three-dimensional sampling point in two adjacent depth maps; Determining the confidence level of each of the three-dimensional sampling points according to the color error; The three-dimensional sampling points whose confidence level is less than a preset confidence threshold are deleted.

8. A fault location device for an indoor distribution system, characterized in that: The device comprises: An alarm acquisition module is used to obtain alarm information and determine faulty device information based on the alarm information; A system positioning module, configured to determine the indoor distribution system corresponding to the faulty device information as the system to be checked; a model determination module, configured to determine a three-dimensional model corresponding to the system to be checked, wherein the three-dimensional model includes a plurality of three-dimensional sampling points, and one or more of the three-dimensional sampling points corresponds to a device in the system to be checked; A fault location module, configured to mark the fault location in the three-dimensional model according to the alarm information; a model matching module for acquiring an actual device image sent by an AR device associated with the system to be troubleshooted, fusing the marked three-dimensional model with the actual device image to obtain an enhanced system image; wherein, before fusing, a display message is sent to the AR device to instruct the user to align manually determined feature points in the actual device image with feature points in the three-dimensional model, thereby superimposing the three-dimensional model on the actual device image; after acquiring the actual device image, the user is instructed to move the AR device based on the positions of the feature points in the actual device image reported by the AR device, thereby aligning the three-dimensional model with the actual device image, and then performing superimposition and fusion; The image display module displays the enhanced system image through the AR device; determines the geographical location of the faulty device based on the alarm information, and sends it to the AR device or the operation and maintenance personnel device associated with the system to be checked.

9. An indoor distribution system fault location device, characterized in that: include: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the indoor distribution system fault location method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores at least one executable instruction. When the executable instruction is executed on the indoor distributed system fault locating device / apparatus, the indoor distributed system fault locating device executes the operation of the indoor distributed system fault locating method according to any one of claims 1 to 7.

Images