Cable fault location method, device and electronic equipment based on augmented reality technology
Through augmented reality technology and edge folding algorithm, the problems of multi-cable mixed cables, positioning accuracy error and high computing power occupancy in cable fault location are solved, and the rapid identification and maintenance of cable faults are realized, reducing computing resource consumption.
Patent Information
- Application Number
- CN202410467685.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-18
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-04-18
AI Technical Summary
In the cable fault location, the prior art has problems such as the complexity of multiple cable equipment, resulting in difficulty in identifying faults, large error in underground cable positioning accuracy and high 3D model real-time rendering computing power.
Using augmented reality technology, the three-dimensional underground cable model is fused with environmental images through head-mounted display devices, providing cable laying position images, realizing accurate positioning and rapid identification of faulty cable equipment, and simplifying the three-dimensional model through edge folding algorithm to reduce computing power Occupancy.
It realizes timely identification and maintenance of faulty cable equipment, reduces the difficulty of troubleshooting in cable trenches, and reduces the computing resources required for three-dimensional model rendering.
Smart Images

Figure CN118314211B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a cable fault location method, device, and electronic device based on augmented reality technology. Background Art
[0002] Locating cable faults is an important technology for inspecting underground cables and maintaining cable faults. At present, the common method for locating cable faults is: first, determine the buried area of the faulty cable based on longitude and latitude or drawings. Then, the inspectors dig cable trenches in the buried area based on experience. Finally, the inspectors inspect and perform fault maintenance on each cable device in the cable trench in turn.
[0003] However, it is found in practice that when the above method is used to locate cable faults, the following technical problems often occur:
[0004] First, since there are usually a large number of cable devices in the cable trench, it is difficult to identify the faulty cable devices in time if the faults are removed one by one, which makes it difficult to maintain the faulty cable devices in time;
[0005] Second, because underground cables are buried underground and there are usually precision errors in longitude and latitude or drawings, it is difficult for inspection personnel to accurately locate the buried position and direction of underground cables on the ground in a timely manner based on longitude and latitude or drawings, making it difficult to dig out the location of the faulty cable in time when digging the cable trench, making it difficult to maintain the faulty cable in time;
[0006] Third, if the underground cables are located on the ground by superimposing the three-dimensional model on the real space, when the model is more complex, a lot of computing power is required to render the three-dimensional model in real time, which results in a large amount of computing power resources being occupied.
[0007] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention
[0008] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.
[0009] Some embodiments of the present disclosure propose a cable fault location method, device, and electronic device based on augmented reality technology to solve one or more of the technical problems mentioned in the above background technology section.
[0010] In a first aspect, some embodiments of the present disclosure provide a cable fault location method based on augmented reality technology, the method comprising: in response to receiving a user task query request for a target user, sending a cable fault inspection task information set corresponding to the user task query request to a head-mounted display device corresponding to the target user for selection by the target user; in response to receiving task selection information for each cable fault inspection task information in the cable fault inspection task information set, acquiring a current environment image, user posture information, historical user posture information and a three-dimensional underground cable model; based on the current environment image, constructing a three-dimensional environment image; matching the user posture information and the historical user posture information to obtain task location matching information; in response to determining the above The task position matching information satisfies the preset matching conditions, the three-dimensional environment image and the three-dimensional underground cable model are fused to obtain a cable laying position image, and the cable laying position image is sent to the head-mounted display device for the target user to dig cable trenches and repair faulty cables; in response to receiving the cable equipment image sent by the head-mounted display device, based on the three-dimensional underground cable model, the cable equipment image is fault identified to obtain a cable equipment fault information group; based on the cable equipment fault information group, the cable equipment image is updated to obtain a cable equipment fault mark image, and the head-mounted display device is controlled to display the cable equipment fault mark image, so that the target user can locate and repair the cable fault.
[0011] In a second aspect, some embodiments of the present disclosure provide a cable fault locating device based on augmented reality technology, the device comprising: a sending unit, configured to, in response to receiving a user task query request for a target user, send a cable fault inspection task information set corresponding to the above user task query request to a head-mounted display device corresponding to the above target user for selection by the above target user; an acquiring unit, configured to, in response to receiving task selection information for each cable fault inspection task information in the above cable fault inspection task information set, acquire a current environment image, user posture information, historical user posture information and a three-dimensional underground cable model; a constructing unit, configured to construct a three-dimensional environment image based on the above current environment image; a matching unit, configured to match the above user posture information and the above historical user posture information to obtain task position matching information; and a fusing and sending unit. The unit is configured to, in response to determining that the task position matching information satisfies a preset matching condition, fuse the three-dimensional environment image and the three-dimensional underground cable model to obtain a cable laying position image, and send the cable laying position image to the head-mounted display device, so that the target user can dig a cable trench and repair the faulty cable; the fault identification unit is configured to, in response to receiving the cable equipment image sent by the head-mounted display device, perform fault identification processing on the cable equipment image based on the three-dimensional underground cable model to obtain a cable equipment fault information group; the update and control unit is configured to update the cable equipment image based on the cable equipment fault information group to obtain a cable equipment fault mark image, and control the head-mounted display device to display the cable equipment fault mark image, so that the target user can locate and repair the cable fault.
[0012] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.
[0013] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.
[0014] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the cable fault location method based on augmented reality technology of some embodiments of the present disclosure, the faulty cable equipment can be identified in time, so as to maintain the faulty cable equipment in time. Specifically, the reason why it is difficult to maintain the faulty cable equipment in time is that since there are usually a large number of cable equipment in the cable trench, it is difficult to identify the faulty cable equipment in time if the faults are removed one by one. Based on this, the cable fault location method based on augmented reality technology of some embodiments of the present disclosure, first, in response to receiving a user task query request for a target user, the cable fault inspection task information set corresponding to the user task query request is sent to the head-mounted display device corresponding to the target user for selection by the target user. Thus, each fault cable inspection task to be executed can be obtained, so that the target user can select the task to be executed. Secondly, in response to receiving the task selection information for each cable fault inspection task information in the cable fault inspection task information set, the current environment image, user posture information, historical user posture information and three-dimensional underground cable model are obtained. Thus, the relevant information of the task to be executed by the target user can be obtained, so as to locate the underground cable to be inspected. Then, based on the current environment image, a three-dimensional environment image is constructed. Thus, a three-dimensional environment image of the target user's location can be obtained. Afterwards, the user posture information and the historical user posture information are matched to obtain task position matching information. Thus, the error between the current location of the target user and the buried location of the faulty cable can be obtained. Next, in response to determining that the task position matching information meets the preset matching condition, the three-dimensional environment image and the three-dimensional underground cable model are fused to obtain a cable laying location image, and the cable laying location image is sent to the head-mounted display device for the target user to dig a cable trench and repair the faulty cable. Thus, when the target user has reached the vicinity of the buried location of the faulty cable, the target user can locate the underground cable on the ground surface according to the location of the three-dimensional underground cable model. Then, in response to receiving the cable equipment image sent by the head-mounted display device, the cable equipment image is subjected to fault identification processing based on the three-dimensional underground cable model to obtain a cable equipment fault information group. Thus, by identifying the cable equipment image taken after the cable trench is excavated, each cable equipment fault information can be quickly obtained. Finally, based on the cable equipment fault information group, the cable equipment image is updated to obtain a cable equipment fault mark image, and the head mounted display device is controlled to display the cable equipment fault mark image, so that the target user can locate and repair the cable fault. Thus, each marked cable equipment fault information can be displayed through the head mounted display device, so that the user can maintain the faulty cable equipment in time.Therefore, the cable fault location method based on augmented reality technology in some embodiments of the present disclosure can accurately locate the buried position of the underground cable by superimposing the three-dimensional underground cable model and the three-dimensional environment image, and can timely identify each faulty cable device by matching and identifying the underground cable model and the underground cable image. Therefore, the inspection personnel can timely maintain the faulty cable device. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.
[0016] Figure 1 is a flow chart of some embodiments of a cable fault location method based on augmented reality technology according to the present disclosure;
[0017] Figure 2 is a schematic structural diagram of some embodiments of a cable fault location device based on augmented reality technology according to the present disclosure;
[0018] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION
[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0020] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.
[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0024] Figure 1 The process 100 of some embodiments of the cable fault location method based on augmented reality technology according to the present disclosure is shown. The cable fault location method based on augmented reality technology includes the following steps:
[0025] Step 101, in response to receiving a user task query request for a target user, sending a cable fault inspection task information set corresponding to the user task query request to a head mounted display device corresponding to the target user for selection by the target user.
[0026] In some embodiments, the execution subject (e.g., computing device) of the cable fault location method based on augmented reality technology may respond to receiving a user task query request for a target user, and send a cable fault inspection task information set corresponding to the user task query request to a head mounted display device corresponding to the target user for selection by the target user. The target user may be an inspector who is performing a cable fault inspection task. The user task query request may be a network request for querying each cable fault inspection task to be performed by the target user. The cable fault inspection task information in the cable fault inspection task information set may be information of a cable fault inspection task to be performed. The cable fault inspection task may be a task for inspecting a cable with a fault. The cable fault inspection task information in the cable fault inspection task information set may include an abnormal cable identifier and an execution status identifier. The abnormal cable identifier may be a unique identifier of a segmented cable with an abnormality to be inspected. The segmented cable may be obtained by dividing the entire cable according to a preset rule. For example, the preset rule may be a preset fixed length starting from one end of the cable, and dividing the cable at equal distances until the other end of the cable. The segmented cable may be associated with a segmented cable position coordinate sequence. The segmented cable position coordinate sequence may be a set of pre-collected three-dimensional coordinates of each point on the segmented cable in the world coordinate system. The execution status identifier may characterize the execution progress of the cable fault inspection task. For example, the execution status identifier may be "unfinished". The head-mounted display device may be an AR (Augmented Reality) device configured with a stereo camera assembly, an IMU (Inertial Measurement Unit) sensor assembly, and a visual inertial navigation system. The visual inertial navigation system may be a system that combines a camera and an IMU sensor to estimate position and posture.
[0027] Step 102, in response to receiving task selection information for each cable fault inspection task information in the cable fault inspection task information set, obtaining a current environment image, user posture information, historical user posture information and a three-dimensional underground cable model.
[0028] In some embodiments, the execution subject may obtain the current environment image, user posture information, historical user posture information and three-dimensional underground cable model in response to receiving the task selection information for each cable fault inspection task information in the cable fault inspection task information set. Among them, the task selection information may represent the cable fault inspection task to be executed selected by the target user. The task selection information may include an abnormal cable identification. The current environment image may be a two-dimensional image of the user's front environment photographed by the stereo camera assembly on the head-mounted display device. The user posture information may be a posture matrix representing the position and posture of the target user output by the IMU assembly on the head-mounted display device. The historical user posture information may be a posture matrix when the user photographs the segmented cable in the cable trench through the stereo camera assembly on the head-mounted display device when the segmented cable is buried in the cable trench. The three-dimensional underground cable model may be a three-dimensional model corresponding to the segmented cable. According to the abnormal cable identification, the historical user posture information corresponding to the abnormal cable identification may be obtained from the database, and the three-dimensional underground cable model may be obtained from the three-dimensional underground cable model library. The three-dimensional underground cable model library can be used to store three-dimensional underground cable models corresponding to each segmented cable. Each segmented cable associated with the three-dimensional underground cable model library can correspond to the same cable.
[0029] Optionally, before obtaining the current environment image, user posture information, historical user posture information and the three-dimensional underground cable model, the execution subject may further perform the following steps:
[0030] The first step is to obtain a three-dimensional point cloud data set, wherein the three-dimensional point cloud data set may be a data set obtained by measuring the segmented cable to be inspected by a handheld laser radar device.
[0031] The second step is to construct a three-dimensional point cloud model of underground cables based on the above-mentioned three-dimensional point cloud data set. Among them, the above-mentioned three-dimensional point cloud model of underground cables can be a three-dimensional grid model corresponding to a segmented cable. The above-mentioned three-dimensional point cloud model of underground cables can include a grid information set and an edge information set. The above-mentioned grid information set can be information of each grid obtained by segmenting the surface of the three-dimensional point cloud model of underground cables according to triangles. The grid information in the above-mentioned grid information set can include an edge identification group and a vertex identification group. The edge identification in the above-mentioned edge identification group can be a unique identification of an edge that encloses a grid. The vertex identification in the above-mentioned vertex identification group can be a unique identification of the vertex of the grid. Each vertex has a corresponding three-dimensional coordinate. Each edge information in the above-mentioned edge information set can include an edge identification and an associated vertex identification group. The associated vertex identification in the above-mentioned associated vertex identification group can be the identification of the endpoint of the corresponding edge. The three-dimensional point cloud model of underground cables can be constructed based on the above-mentioned three-dimensional point cloud data set by a preset three-dimensional model construction method.
[0032] As an example, the above-mentioned three-dimensional model construction method may include but is not limited to at least one of the following: a mesh construction method based on surface reconstruction, and a mesh construction method based on a triangulation algorithm.
[0033] The third step is to perform feature extraction processing on the above three-dimensional point cloud data set to obtain a key point information set. The key point information in the above key point information set may include a key point identifier. The above key point identifier may be a unique identifier of the key point. The key point may be one of the following: an edge point, a corner point, and an inflection point. The above three-dimensional point cloud data set may be subjected to feature extraction processing by a preset feature extraction method to obtain a key point information set.
[0034] As an example, the feature extraction method may include but is not limited to at least one of the following: a corner detection method, a SIFT3D (Scale-Invariant Feature Transform for 3D) method.
[0035] Step 4: For each edge information in the above edge information set, perform the following edge folding cost generation steps:
[0036] The first sub-step is to match the associated vertex identifier group included in the edge information with the key point information set to obtain the number of matching points. The number of matching points may be the number of key point endpoints of the two endpoints of the corresponding edge. First, for each associated vertex identifier in the associated vertex identifier group, the key point information including the same key point identifier as the associated vertex identifier is selected from the key point information set as the target associated vertex identifier. Then, the number corresponding to each target associated vertex identifier is determined as the number of matching points.
[0037] The second sub-step, in response to determining that the number of matching points meets the preset quantity condition, determines the vertex error matrix and regional information entropy corresponding to each associated vertex identifier in the associated vertex identifier group included in the edge information. Among them, the preset quantity condition can be that the number of matching points is equal to 0. The vertex error matrix can characterize the sum of squares of distances from the corresponding vertex to each adjacent mesh plane. The regional information entropy can characterize the information entropy of the area where the corresponding vertex is located. First, the vertex error matrix corresponding to each associated vertex identifier in the associated vertex identifier group included in the edge information can be determined by the QEM (Quadric Error Metrics) algorithm. Then, the regional information entropy corresponding to each associated vertex identifier in the associated vertex identifier group included in the edge information is determined by the information entropy formula.
[0038] The third sub-step is to generate a first folding cost value based on the obtained vertex error matrices. The first folding cost value can represent the degree of influence of the error caused by the edge corresponding to the edge information on the three-dimensional model after being folded. First, the sum of the obtained vertex error matrices is determined as the vertex error matrix corresponding to the vertex to be added. Then, the product of the transposed vector corresponding to the preset vertex vector, the vertex error matrix and the preset vertex vector is determined as the first folding cost value. The preset vertex vector can be a pre-set column vector to be solved that represents the coordinates of the vertex to be added.
[0039] In the fourth sub-step, the sum of the obtained information entropies of each region is determined as the second folding cost value.
[0040] In a fifth sub-step, in response to determining that the second folding cost value satisfies a preset information entropy threshold condition, the ratio of the first folding cost value to the second folding cost value is determined as the edge folding cost value. The preset information entropy threshold condition may be that the second folding cost value is greater than or equal to a preset information entropy threshold. The information entropy threshold may be a lower limit value of the information entropy.
[0041] The sixth sub-step is to determine the candidate new vertex coordinates and the candidate edge folding cost values based on the edge folding cost values. The candidate new vertex coordinates may be the coordinates of the vertex to be added that makes the edge folding cost value obtain the minimum value. The candidate edge folding cost value may be the edge folding cost value corresponding to the candidate new vertex coordinates. The candidate new vertex coordinates and the candidate edge folding cost values may be determined based on the edge folding cost values by the least square method.
[0042] In a seventh sub-step, the candidate edge folding cost value, the candidate newly added vertex coordinates and the edge identifier included in the edge information are determined as target edge folding cost information.
[0043] In the fifth step, the obtained target edge folding cost information is sorted to obtain a target edge folding cost information sequence. The target edge folding cost information sequence may be a sequence in which the target edge folding cost information is arranged in ascending order according to the corresponding candidate edge folding cost value. The obtained target edge folding cost information may be sorted by a preset sorting algorithm to obtain a target edge folding cost information sequence. For example, the sorting algorithm may include but is not limited to at least one of the following: bubble sort, quick sort.
[0044] Step 6: Based on the grid information set and the target edge folding cost information sequence, perform the following grid information set update steps:
[0045] In the first sub-step, target edge folding cost information that meets a preset cost condition is selected from the target edge folding cost information sequence as the edge cost information to be folded. The preset cost condition may be the target edge folding cost information with a sequence number of 1 in the target edge folding cost information sequence.
[0046] The second sub-step is to determine the newly added vertex information and the group of vertex identifiers to be connected based on the above-mentioned edge cost information to be collapsed. The above-mentioned newly added vertex information can represent the vertex to be added. The above-mentioned group of vertex identifiers to be connected can be a set of identifiers of each vertex connected to the vertex to be added. First, a vertex identifier corresponding to the candidate newly added vertex coordinates included in the above-mentioned edge cost information to be collapsed is generated as a newly added vertex identifier through a UUID (Universally Unique Identifier) identifier generation algorithm. The above-mentioned newly added vertex identifier can be the identifier of the vertex to be added on the model surface. Secondly, the candidate newly added vertex coordinates and the above-mentioned newly added vertex identifier included in the above-mentioned edge cost information to be collapsed can be determined as the newly added vertex information. Then, for each associated vertex identifier corresponding to the edge identifier included in the above-mentioned edge cost information to be collapsed, the edge information matching the above-mentioned associated vertex identifier is selected from the above-mentioned edge information set as the target edge information to obtain the target edge information group. Matching the above-mentioned associated vertex identifier can be that there is an associated vertex identifier that is the same as the above-mentioned associated vertex identifier among the associated vertex identifiers included in the edge information. Afterwards, the associated vertex identifiers that meet the non-to-be-deleted vertex condition are selected from the associated vertex identifiers corresponding to the target edge information group as the vertex identifiers to be connected, and the vertex identifier group to be connected is obtained. The non-to-be-deleted vertex condition can be that the vertex corresponding to the associated vertex identifier is not the endpoint of the folded edge corresponding to the cost information of the folded edge.
[0047] The third sub-step is to perform a deletion operation on the edge corresponding to the edge cost information to be collapsed, and perform an edge addition operation based on the newly added vertex information and the group of vertex identifiers to be connected, to obtain a group of newly added edge information. The newly added edge information in the group of newly added edge information can be a newly added edge used to mesh the three-dimensional point cloud model of the underground cable. For each vertex identifier to be connected in the group of vertex identifiers to be connected, perform the following steps:
[0048] Sub-step one: connect the to-be-added vertices corresponding to the above-mentioned newly added vertex information and the vertices corresponding to the above-mentioned to-be-connected vertex identifiers to obtain a newly added edge, and determine each vertex identifier corresponding to the newly added edge as an associated vertex identifier to obtain an associated vertex identifier group.
[0049] Sub-step 2: Generate a new edge identifier through a UUID (Universally Unique Identifier) identifier generation algorithm.
[0050] Sub-step three: determining the newly added edge identifier and the associated vertex identifier group as newly added edge information.
[0051] It should be noted that, through the above-mentioned edge deletion operation and the above-mentioned edge addition operation, the grid information set and the edge information set of the underground cable 3D point cloud model package are also updated accordingly.
[0052] The fourth sub-step is to determine the newly added mesh information group corresponding to the newly added edge information group. Each newly added mesh information may correspond to a triangular mesh one-to-one. The triangular mesh may be a mesh obtained by segmenting the surface of the underground cable three-dimensional point cloud model according to triangles. The newly added mesh information in the newly added mesh information group may be the information of the newly added mesh on the surface of the underground cable three-dimensional point cloud model. The mesh information in the mesh information set corresponding to the underground cable three-dimensional point cloud model that matches the newly added edge information group is determined as the newly added mesh information. Matching the newly added edge information group may be that the edge identifiers included in the mesh information are the same as any newly added edge identifier corresponding to the newly added edge information group.
[0053] In a fifth sub-step, for each of the newly added mesh information in the newly added mesh information group, in response to determining that the newly added mesh information satisfies a preset regularity condition, mesh reconstruction is performed on the triangular mesh corresponding to the newly added mesh information to obtain a reconstructed mesh information group. The preset regularity condition may be that the regularity of the triangle corresponding to the newly added mesh information approaches 1. The triangular mesh corresponding to the newly added mesh information may be mesh reconstructed by a triangulation algorithm to obtain a reconstructed mesh information group.
[0054] The sixth sub-step is to update the grid information set based on the newly added grid information group and the obtained reconstructed grid information groups to obtain an updated grid information set. The updated grid information set may be a grid information set after edge folding. First, the grid information in the grid information set that matches any reconstructed grid information group is deleted to obtain a deleted grid information set. The grid information that matches any reconstructed grid information group may be that the grid information in the grid information set is different from the newly added grid information in the newly added grid information group. Then, each deleted grid information in the deleted grid information set and each reconstructed grid information in the reconstructed grid information groups are determined as updated grid information to obtain an updated grid information set.
[0055] In a seventh sub-step, in response to detecting that the updated grid information set satisfies a preset model simplification condition, the underground cable three-dimensional point cloud model including the updated grid information set is determined as a three-dimensional underground cable model. The preset model simplification condition may be: the ratio of the number of updated grid information in the updated grid information set to the number of grid information in the grid information set before the edge is collapsed is a preset ratio. The preset ratio may be a preset ratio. For example, the preset ratio may be 10%.
[0056] Optionally, the above execution entity may further perform the following steps:
[0057] In the first step, in response to detecting that the updated grid information set does not meet the preset model simplification condition, the newly added edge folding cost information corresponding to each newly added edge information in the newly added edge information group is determined. The newly added edge folding cost information can represent the information of the folding cost corresponding to the newly added edge. For each newly added edge information in the newly added edge information group, the newly added edge information is used as edge information, and the edge folding cost information generation step is performed to generate target edge folding cost information as the newly added edge folding cost information corresponding to the newly added edge information.
[0058] In the second step, based on the determined newly added edge folding cost information, the above-mentioned target edge folding cost information sequence is updated to obtain an updated target edge folding cost information sequence. Among them, the above-mentioned updated target edge folding cost information sequence can be an updated target edge folding cost information sequence. First, each newly added edge folding cost information and each target edge folding cost information in the above-mentioned target edge folding cost information sequence are determined as updated target edge folding cost information to obtain an updated target edge folding cost information set. Then, through the above-mentioned sorting algorithm, according to the candidate edge folding cost values included in the updated target edge folding cost information, the above-mentioned updated target edge folding cost information set is arranged in ascending order from small to large to obtain an updated target edge folding cost information sequence.
[0059] The third step is to use the updated grid information set as the grid information set and the updated target edge folding cost information sequence as the target edge folding cost information sequence, and perform the grid information set updating step again.
[0060] Optionally, the execution subject may also determine a preset weight as the edge collapse cost in response to determining that the number of matching points does not meet the preset quantity condition. The preset weight may be infinite.
[0061] The above-mentioned three-dimensional underground cable model generation steps and related contents, as an inventive point of an embodiment of the present disclosure, solve the technical problem three mentioned in the background technology, "more computing resources are occupied". The reasons for the large amount of computing resources are often as follows: if the underground cable is located on the ground by superimposing the real space with a three-dimensional model, when the model is more complex, a large amount of computing power is required to render the three-dimensional model in real time. If the above problems are solved, the effect of timely maintenance of faulty cables can be achieved. In order to achieve this effect, first, the three-dimensional point cloud model of the underground cable is feature extracted to obtain each key point, and the folding cost corresponding to the edge involving the key point is determined to be infinite, thereby, the edge with a large folding cost will not be folded, and the key features of the model can be retained. Then, the edge folding cost is determined by the edge folding algorithm and the regional information entropy, and the edge with a small edge folding cost is selected, thereby realizing the folding of non-key feature areas. Afterwards, the narrow and long triangle mesh with poor regularity is also meshed. Finally, by iterating the above-mentioned edge folding process, the above-mentioned underground cable three-dimensional point cloud model can be simplified into a three-dimensional underground cable model. Therefore, when the simplified three-dimensional underground cable model is rendered in real time to be superimposed on the three-dimensional environment image, the computing power resources occupied can be reduced.
[0062] Step 103: construct a three-dimensional environment image based on the current environment image.
[0063] In some embodiments, the execution subject may construct a three-dimensional environment image based on the current environment image. The three-dimensional environment image may be a three-dimensional image of the environment in front of the user. The three-dimensional environment image may be constructed based on the current environment image through a preset three-dimensional image construction interface. The three-dimensional image construction interface may encapsulate an image point cloud conversion method and a three-dimensional image construction method. The image point cloud conversion method may be a method for converting a two-dimensional image into a three-dimensional point cloud. The three-dimensional image construction method may be a method for reconstructing a three-dimensional image based on a three-dimensional point cloud.
[0064] As an example, the above-mentioned image point cloud conversion method may include but is not limited to at least one of the following: a stereoscopic vision algorithm, an algorithm based on multi-view geometry, a depth estimation and point cloud generation method based on deep learning. The above-mentioned three-dimensional image construction method may include but is not limited to at least one of the following: a three-dimensional image construction method based on voxel-based three-dimensional point cloud, a three-dimensional image construction algorithm based on surface reconstruction of three-dimensional point cloud, and a three-dimensional image construction algorithm based on deep learning of three-dimensional point cloud.
[0065] Step 104 , matching the user position information with the historical user position information to obtain task position matching information.
[0066] In some embodiments, the execution subject may match the user posture information and the historical user posture information in various ways to obtain task position matching information, wherein the task position matching information may represent the difference between the posture matrix corresponding to the user posture information and the posture matrix corresponding to the historical user posture information.
[0067] In some optional implementations of some embodiments, the execution subject may perform matching processing on the user posture information and the historical user posture information to obtain task position matching information through the following steps:
[0068] The first step is to determine the first position coordinates and the first direction angle corresponding to the user posture information. The first position coordinates can represent the current position of the target user. The first direction angle can be the yaw angle corresponding to the current posture of the target user. The first position coordinates and the first direction angle corresponding to the user posture information can be determined by a preset position and posture solution method.
[0069] As an example, the above-mentioned position and attitude solution method may include but is not limited to at least one of the following: a posture matrix method, a quaternion method, etc.
[0070] The second step is to determine the second position coordinates and the second direction angle corresponding to the above historical user posture information. The above second position coordinates can represent the position of the user corresponding to the above historical user posture information. The above second direction angle can be the angle of the yaw angle corresponding to the user posture associated with the above historical user posture information. The second position coordinates and the second direction angle corresponding to the above historical user posture information can be determined by the above position and posture solution method.
[0071] The third step is to match the first position coordinates with the second position coordinates to obtain first matching information. The first matching information may represent the difference between the first position coordinates and the second position coordinates. The absolute value of the difference between the first position coordinates and the second position coordinates may be determined as the first matching information.
[0072] In the fourth step, the first direction angle and the second direction angle are matched to obtain second matching information. The second matching information may represent the difference between the first direction angle and the second direction angle. The absolute value of the difference between the first direction angle and the second direction angle may be determined as the second matching information.
[0073] The fifth step is to generate task location matching information based on the first matching information and the second matching information. The task location matching information can indicate whether the target user has arrived at the location where the cable inspection task is to be performed. The first matching information and the second matching information can be determined as the task location matching information.
[0074] Step 105, in response to determining that the task location matching information meets the preset matching conditions, the three-dimensional environment image and the three-dimensional underground cable model are fused to obtain a cable laying location image, and the cable laying location image is sent to a head-mounted display device for the target user to dig cable trenches and repair faulty cables.
[0075] In some embodiments, the execution subject may, in response to determining that the task position matching information satisfies a preset matching condition, fuse the three-dimensional environment image and the three-dimensional underground cable model to obtain a cable laying position image, and send the cable laying position image to the head-mounted display device, so that the target user can dig a cable trench and repair a faulty cable. Among them, the preset matching condition may be: the value corresponding to the first matching information in the task position matching information is less than or equal to the preset position difference, and the value corresponding to the second matching information is less than or equal to the preset angle difference. The preset position difference may be the upper limit of the difference between the preset position coordinates. The preset angle difference may be the upper limit of the difference between the preset yaw angles. The cable laying position image may be a three-dimensional environment image superimposed with a three-dimensional underground cable model. First, the three-dimensional environment image and the three-dimensional underground cable model may be fused by augmented reality technology to obtain a cable laying position image, and the cable laying position image may be sent to the head-mounted display device. Then, the target user can view the cable laying location image through the head-mounted display device, and dig cable trenches and repair faulty cables according to the position of the above-mentioned three-dimensional underground cable model in the above-mentioned three-dimensional environmental image.
[0076] Optionally, the above execution entity may further perform the following steps:
[0077] The first step, in response to determining that the above-mentioned task position matching information does not meet the above-mentioned preset matching conditions, based on the preset position transformation matrix, the above-mentioned second position coordinates are transformed from the world coordinate system to the camera coordinate system to generate the third position coordinates. Among them, the above-mentioned camera coordinate system can be the coordinate system corresponding to the stereo camera equipped on the above-mentioned head-mounted display device. The above-mentioned preset position transformation matrix can characterize the transformation relationship of the coordinates between the world coordinate system and the camera coordinate system. The above-mentioned third position coordinates can be the coordinates of the projection point of the point corresponding to the second position coordinate in the camera coordinate system. The above-mentioned second position coordinates can be transformed from the world coordinate system to the camera coordinate system to generate the third position coordinates based on the preset position transformation matrix through the coordinate system transformation method of the transformation matrix.
[0078] The second step is to project the third position coordinates onto a plane formed by the horizontal axis and the optical axis in the camera coordinate system to generate a fourth position coordinate. The fourth position coordinates may be the coordinates of the projection of the point corresponding to the third position coordinate onto the plane formed by the horizontal axis and the optical axis in the camera coordinate system.
[0079] The third step is to determine the moving distance information and the moving direction information based on the fourth position coordinate and the origin of the preset coordinate system. The origin of the preset coordinate system may be the origin of the camera coordinate system. The moving distance information may be the distance between the fourth position coordinate and the location of the target user. The moving direction information may be the angle and direction information of the inspection location of the fault cable inspection task to be performed deviating from the facing direction of the target user. First, the distance value between the fourth position coordinate and the origin of the preset coordinate system is determined as the moving distance information. Then, in response to determining that the horizontal coordinate in the fourth position coordinate is greater than 0, the first preset direction identifier is determined as the direction information. The first preset direction identifier may indicate that the fourth position coordinate is on the right side of the optical axis. For example, the first preset direction identifier may be "right". Afterwards, in response to determining that the horizontal coordinate in the fourth position coordinate is less than 0, the second preset direction identifier is determined as the direction information. The second preset direction identifier may indicate that the fourth position coordinate is on the left side of the optical axis. For example, the second preset direction identifier may be "left". Next, the ratio of the absolute value of the horizontal coordinate of the fourth position coordinate to the distance value corresponding to the moving distance information is determined as the sine value. Next, the arc sine function value corresponding to the sine value is determined as the angle information. Finally, the direction information and the angle information are determined as the moving direction information.
[0080] Step 4: Based on the moving distance information and the moving direction information, fill in the preset prompt text template to obtain the target prompt text information. The preset prompt text template may be a pre-set text type template for prompting the target user to move to the straight-line distance of the faulty cable. For example, the preset prompt text template may be "_ turn_ degree direction, straight-line distance_".
[0081] The fifth step is to send the moving direction information and the target prompt text information to the head mounted display device, so as to display a direction arrow graphic and a prompt text box in the three-dimensional environment image. The direction arrow graphic is used to represent the direction corresponding to the moving direction information. The prompt text box is used to display the text corresponding to the target prompt text information. The head mounted display device can display the direction arrow graphic corresponding to the moving direction information in the three-dimensional environment image through augmented reality technology, and display the prompt text box above the direction arrow graphic to guide the target user to move to the target position.
[0082] The execution steps and related contents of the task location matching information that does not meet the preset matching conditions, as an inventive point of the embodiment of the present disclosure, solve the technical problem 2 mentioned in the background technology, "it is difficult to maintain the faulty cable in time". The reasons for the difficulty in timely maintenance of the faulty cable are often as follows: because the underground cable is buried underground and the longitude and latitude or drawings usually have errors, it is difficult for the inspection personnel to accurately locate the buried position and direction of the underground cable on the ground in time according to the longitude and latitude or drawings, making it difficult to dig the location of the faulty cable in time when digging the cable trench. If the above problems are solved, the effect of timely maintenance of the faulty cable can be achieved. In order to achieve this effect, when the error between the location of the target user and the buried location of the underground cable is large, first, the direct distance between the location of the target user and the buried location of the underground cable, as well as the direction and angle that the target user needs to turn from the current location to the buried location of the underground cable can be determined. Then, the target user is guided to the buried location of the underground cable by displaying the direction arrow and prompt text information representing the direction and angle in the three-dimensional environment image. Finally, the target user can quickly locate and excavate the underground cable more accurately according to the position of the three-dimensional underground cable model superimposed on the three-dimensional environment image. Therefore, it is convenient to locate the faulty cable in time, so that the faulty cable can be maintained in time.
[0083] Step 106 , in response to receiving the cable device image sent by the head mounted display device, perform fault identification processing on the cable device image based on the three-dimensional underground cable model to obtain a cable device fault information group.
[0084] In some embodiments, the execution subject may, in response to receiving the cable equipment image sent by the head mounted display device, perform fault identification processing on the cable equipment image based on the three-dimensional underground cable model in various ways to obtain a cable equipment fault information group. The cable equipment image may be an image of the cable equipment in the cable trench taken by the target user through the stereo camera assembly on the head mounted display device after the cable trench is excavated. The cable equipment fault information in the cable equipment fault information group may be information about the cable equipment with faults.
[0085] In some optional implementations of some embodiments, the execution subject may perform fault identification processing on the cable equipment image based on the three-dimensional underground cable model to obtain a cable equipment fault information group through the following steps:
[0086] The first step is to perform brightness detection processing on the cable equipment image to obtain image brightness information. The image brightness information may be information indicating whether the image is overexposed or insufficiently bright. For example, the image brightness information may be "grayscale mean 240". The cable equipment image may be subjected to brightness detection processing by a preset image brightness detection method to obtain image brightness information. For example, the image brightness detection method may be based on brightness detection of grayscale mean.
[0087] In the second step, according to the above-mentioned image brightness information, the above-mentioned cable equipment image is subjected to image enhancement processing to obtain an enhanced cable equipment image. The above-mentioned enhanced cable equipment image may be a cable equipment image after brightness adjustment. First, in response to determining that the above-mentioned image brightness information satisfies a preset overexposure condition, the above-mentioned cable equipment image is subjected to image enhancement processing by a preset brightness attenuation method to obtain an enhanced cable equipment image. The preset overexposure condition may be that the grayscale mean corresponding to the above-mentioned image brightness information is greater than a first grayscale threshold. The above-mentioned first grayscale threshold may be a preset lower limit value of the grayscale mean representing image overexposure. Then, in response to determining that the above-mentioned image brightness information satisfies a preset insufficient brightness condition, the above-mentioned cable equipment image is subjected to image enhancement processing by a preset brightness enhancement method to obtain an enhanced cable equipment image. The above-mentioned preset insufficient brightness condition may be that the grayscale mean corresponding to the above-mentioned image brightness information is less than a second grayscale threshold. The above-mentioned second grayscale threshold may be a preset upper limit value of the grayscale mean representing insufficient image brightness.
[0088] As an example, the brightness attenuation method may include but is not limited to at least one of the following: an algorithm based on histogram equalization, an algorithm based on tone mapping. The brightness enhancement method may include but is not limited to at least one of the following: a histogram equalization method, a brightness enhancement method based on a convolutional neural network.
[0089] The third step is to identify and process the enhanced cable equipment image based on the three-dimensional underground cable model to obtain a faulty equipment location information group. The faulty equipment location information in the faulty equipment location information group may be information on the location corresponding to the faulty cable equipment in the cable equipment image. The faulty equipment location information in the faulty equipment location information group may include but is not limited to a faulty equipment identifier. The faulty equipment identifier may be a unique identifier of the identified faulty cable equipment. Through a preset image recognition interface, the enhanced cable equipment image may be identified and processed based on the three-dimensional underground cable model to obtain a faulty equipment location information group. The image recognition interface may be encapsulated with an image recognition method. The image recognition method may be used to match the three-dimensional underground cable model diagram with the enhanced cable equipment image, and output a rectangular box corresponding to the abnormal cable equipment.
[0090] In the fourth step, the above-mentioned faulty equipment location information group and the above-mentioned enhanced cable equipment image are input into a pre-trained cable equipment fault diagnosis model to obtain a cable equipment fault type information group. Among them, the above-mentioned cable equipment fault diagnosis model can be a multi-classification model that takes the faulty equipment location information group and the cable equipment image after image enhancement as input, and takes the cable equipment fault type information group as output. For example, the above-mentioned cable equipment fault diagnosis model can be but is not limited to one of the following: a fault diagnosis model based on deep learning, a decision tree model. The cable equipment fault type information in the above-mentioned cable equipment fault type information group may include a faulty equipment identifier, a fault type identifier, and a fault probability. The above-mentioned fault type identifier may be a unique identifier of the fault type corresponding to the cable equipment. The fault type may include but is not limited to at least one of the following: equipment displacement, equipment damage. The above-mentioned fault probability may be the probability that the faulty equipment has a fault corresponding to the fault type identifier.
[0091] Step 5: Match the fault device location information group and the cable device fault type information group to obtain a cable device fault information group. The cable device fault information in the cable device fault information group may be information about the fault of the corresponding cable device. For each fault device location information in the fault device location information group, perform the following steps:
[0092] In the first sub-step, cable equipment fault type information having a fault device identifier identical to the fault device identifier included in the fault device location information is selected from the cable equipment fault type information group as target cable equipment fault type information.
[0093] The second sub-step is to determine the fault type identifier and the fault probability included in the fault device location information and the target cable device fault type information as the cable device fault information.
[0094] Optionally, the cable equipment fault diagnosis model can be trained by the following steps:
[0095] The first step is to obtain a historical cable equipment fault data set, a simulated cable equipment fault data set and an initial cable equipment fault diagnosis model. Among them, the historical cable equipment fault data in the above historical cable equipment fault data set can be the data of cable equipment faults obtained from a single inspection task in the historical period. The above historical period can be a time period within a preset number of years before the current date. The above preset number of years can be the number of pre-set years. For example, the above preset number of years can be 5 years. The historical cable equipment fault data in the above historical cable equipment fault data set can include inspection time, historical fault equipment location information group, historical cable equipment image and historical cable equipment fault type information group. The historical fault equipment location information in the above historical fault equipment location information group can be the information of the location of the cable equipment that failed at the corresponding inspection time. The above historical cable equipment image can be the image of the cable equipment image taken during the inspection at the corresponding inspection time after image enhancement processing. The historical cable equipment fault type information in the above historical cable equipment fault type information group can be the information of the fault type of the corresponding cable equipment. Each simulated cable equipment fault data in the above simulated cable equipment fault data set can be obtained by simulating the fault of the cable equipment according to the above three-dimensional underground cable model. Wherein, the fault simulation is realized by a preset fault simulation tool. For example, the above-mentioned fault simulation tool may be but is not limited to one of the following: Simulink simulation tool, ANSYS engineering simulation tool. Each simulated cable equipment fault data in the above-mentioned simulated cable equipment fault data set may include a simulated fault equipment location information group, a simulated cable equipment image and a simulated cable equipment fault type information group. The above-mentioned simulated fault equipment location information group may be a device location information group defined during fault simulation. The above-mentioned simulated cable equipment image may be a cable equipment image obtained by fault simulation. The above-mentioned simulated cable equipment fault type information group may be a cable equipment fault type information group obtained during fault simulation. The above-mentioned initial cable equipment fault diagnosis model may be a cable equipment fault diagnosis model that has not been trained. It should be noted that the simulated cable equipment image usually does not need to be subjected to image enhancement processing.
[0096] The second step is to generate a sample cable fault data set based on the above historical cable equipment fault data set and the above simulated cable equipment fault data set. The sample cable fault data in the above sample cable fault data set may include a sample fault device location information group, a sample cable equipment image and a sample label data. The above sample cable equipment image may be a cable equipment image after image enhancement processing. First, for each historical cable equipment fault data in the above historical cable equipment fault data set, the historical fault device location information group, the historical cable equipment image and the historical cable equipment fault type information group included in the above historical cable equipment fault data set are respectively used as a sample fault device location information group, a sample cable equipment image and a sample label data, and the sample fault device location information group, the sample cable equipment image and the sample label data are determined as the first sample cable fault data. Then, for each simulated cable equipment fault data in the above simulated cable equipment fault data set, the simulated fault device location information group, the simulated cable equipment image and the simulated cable equipment fault type information group included in the above simulated cable equipment fault data are determined as the sample fault device location information group, the sample cable equipment image and the sample label data, and the sample fault device location information group, the sample cable equipment image and the sample label data are determined as the second sample cable fault data. Finally, each first sample cable fault data and each second sample cable fault data are respectively determined as sample cable fault data to obtain a sample cable fault data set.
[0097] The third step is to train the initial cable equipment fault diagnosis model based on the sample cable fault data set, and determine the trained initial cable equipment fault diagnosis model as the cable equipment fault diagnosis model.
[0098] Step 107, based on the cable equipment fault information group, the cable equipment image is updated to obtain a cable equipment fault mark image, and the head mounted display device is controlled to display the cable equipment fault mark image so as to facilitate the target user to locate and repair the cable fault.
[0099] In some embodiments, the execution subject may update the cable device image in various ways based on the cable device fault information group to obtain a cable device fault mark image, and control the head mounted display device to display the cable device fault mark image so as to facilitate the target user to locate and repair the cable fault. The cable device fault mark image may be a cable device image that marks the location of the faulty cable device.
[0100] In some optional implementations of some embodiments, each cable device fault information in the cable device fault information group may include cable device detection frame information. The cable device detection frame information may represent the position of the rectangular frame corresponding to the faulty cable device in the cable device image. The execution subject may update the cable device image based on the cable device fault information group to obtain a cable device fault mark image through the following steps:
[0101] In the first step, for each cable device fault information in the cable device fault information group, the following steps are performed:
[0102] The first sub-step is to determine the operation and maintenance method information corresponding to the above-mentioned cable equipment fault information based on the pre-constructed cable equipment operation and maintenance knowledge graph. Among them, the above-mentioned cable equipment operation and maintenance knowledge graph can be a graph with fault type and operation and maintenance method as nodes, and the association relationship between fault type and operation and maintenance method as edges. Each fault type in the above-mentioned cable equipment operation and maintenance knowledge graph can have a corresponding fault type identifier. Each operation and maintenance method in the above-mentioned cable equipment operation and maintenance knowledge graph can have corresponding operation and maintenance method information. The above-mentioned operation and maintenance method information can be a text description information of the operation and maintenance method. For example, the above-mentioned operation and maintenance method information can be "tighten the cable equipment joints". The above-mentioned association relationship can indicate that the faulty cable equipment of the corresponding fault type needs to be maintained by the corresponding operation and maintenance method. According to the fault type identifier included in the above-mentioned cable equipment fault information, the above-mentioned cable equipment operation and maintenance knowledge graph can be queried to obtain the operation and maintenance method information.
[0103] The second sub-step is to aggregate the above-mentioned cable equipment fault information and the above-mentioned operation and maintenance method information to obtain the cable equipment fault label information. Among them, the above-mentioned cable equipment fault label information can be a text-type fault description information corresponding to the faulty cable equipment. The fault type identification, fault probability and the above-mentioned operation and maintenance method information included in the above-mentioned cable equipment fault information can be spliced in sequence through preset splicing characters to obtain the cable equipment fault label information. The above-mentioned preset splicing characters can be pre-set characters used to concatenate the previous and next two data. For example, the above-mentioned preset splicing characters can be ",".
[0104] The third sub-step is to create rectangular elements and text elements. The rectangular elements may be rectangles in three-dimensional space. The text elements may be text boxes in three-dimensional space. The rectangular elements correspond to the cable equipment detection box information. The text elements correspond to the cable equipment fault label information. Rectangular elements and text elements may be created by a preset three-dimensional drawing component library. The three-dimensional drawing component library may be a drawing component library based on WebGL (Web Graphics Library). For example, the drawing component library may be a Three.js three-dimensional drawing library.
[0105] In the second step, each created rectangular element and each text element are added to the cable equipment image to obtain a cable equipment fault mark image. Each text element is on the right side of the corresponding rectangular element. Each rectangular element can be displayed at a target position on the cable equipment image through augmented reality technology, and the text element corresponding to each rectangular element can be displayed on the right side of the corresponding target position. The target position can be the position represented by the cable equipment detection box information corresponding to the rectangular element.
[0106] Optionally, the execution entity may also add the created rectangular elements and text elements to the enhanced cable equipment image as a cable equipment fault marking image.
[0107] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the cable fault location method based on augmented reality technology of some embodiments of the present disclosure, the faulty cable equipment can be identified in time, so as to maintain the faulty cable equipment in time. Specifically, the reason why it is difficult to maintain the faulty cable equipment in time is that since there are usually a large number of cable equipment in the cable trench, it is difficult to identify the faulty cable equipment in time if the faults are removed one by one. Based on this, the cable fault location method based on augmented reality technology of some embodiments of the present disclosure, first, in response to receiving a user task query request for a target user, the cable fault inspection task information set corresponding to the user task query request is sent to the head-mounted display device corresponding to the target user for selection by the target user. Thus, each fault cable inspection task to be executed can be obtained, so that the target user can select the task to be executed. Secondly, in response to receiving the task selection information for each cable fault inspection task information in the cable fault inspection task information set, the current environment image, user posture information, historical user posture information and three-dimensional underground cable model are obtained. Thus, the relevant information of the task to be executed by the target user can be obtained, so as to locate the underground cable to be inspected. Then, based on the current environment image, a three-dimensional environment image is constructed. Thus, a three-dimensional environment image of the target user's location can be obtained. Afterwards, the user posture information and the historical user posture information are matched to obtain task position matching information. Thus, the error between the current location of the target user and the buried location of the faulty cable can be obtained. Next, in response to determining that the task position matching information meets the preset matching condition, the three-dimensional environment image and the three-dimensional underground cable model are fused to obtain a cable laying location image, and the cable laying location image is sent to the head-mounted display device for the target user to dig a cable trench and repair the faulty cable. Thus, when the target user has reached the vicinity of the buried location of the faulty cable, the target user can locate the underground cable on the ground surface according to the location of the three-dimensional underground cable model. Then, in response to receiving the cable equipment image sent by the head-mounted display device, the cable equipment image is subjected to fault identification processing based on the three-dimensional underground cable model to obtain a cable equipment fault information group. Thus, by identifying the cable equipment image taken after the cable trench is excavated, each cable equipment fault information can be quickly obtained. Finally, based on the cable equipment fault information group, the cable equipment image is updated to obtain a cable equipment fault mark image, and the head mounted display device is controlled to display the cable equipment fault mark image, so that the target user can locate and repair the cable fault. Thus, each marked cable equipment fault information can be displayed through the head mounted display device, so that the user can maintain the faulty cable equipment in time.Therefore, the cable fault location method based on augmented reality technology in some embodiments of the present disclosure can accurately locate the buried position of the underground cable by superimposing the three-dimensional underground cable model and the three-dimensional environment image, and can timely identify each faulty cable device by matching and identifying the underground cable model and the underground cable image. Therefore, the inspection personnel can timely maintain the faulty cable device.
[0108] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a cable fault location device based on augmented reality technology. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the cable fault location device 200 based on augmented reality technology can be specifically applied to various electronic devices.
[0109] like Figure 2As shown, some embodiments of the cable fault location device 200 based on augmented reality technology include: a sending unit 201, an acquisition unit 202, a construction unit 203, a matching unit 204, a fusion and sending unit 205, a fault identification unit 206 and an update and control unit 207. Among them, the sending unit 201 is configured to, in response to receiving a user task query request for a target user, send a cable fault inspection task information set corresponding to the user task query request to a head-mounted display device corresponding to the target user for selection by the target user; the acquisition unit 202 is configured to, in response to receiving task selection information for each cable fault inspection task information in the cable fault inspection task information set, acquire the current environment image, user posture information, historical user posture information and a three-dimensional underground cable model; the construction unit 203 is configured to construct a three-dimensional environment image based on the current environment image; the matching unit 204 is configured to match the user posture information and the historical user posture information to obtain task position matching information; the fusion and sending unit 205 is configured to, in response to determining the ... The position matching information satisfies the preset matching conditions, and the three-dimensional environment image and the three-dimensional underground cable model are fused to obtain a cable laying position image, and the cable laying position image is sent to the head-mounted display device for the target user to dig cable trenches and repair faulty cables; the fault identification unit 206 is configured to respond to the cable equipment image sent by the head-mounted display device, and perform fault identification processing on the cable equipment image based on the three-dimensional underground cable model to obtain a cable equipment fault information group; the update and control unit 207 is configured to update the cable equipment image based on the cable equipment fault information group to obtain a cable equipment fault mark image, and control the head-mounted display device to display the cable equipment fault mark image for the target user to locate and repair the cable fault.
[0110] It can be understood that the units described in the cable fault location device 200 based on augmented reality technology are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the cable fault location device 200 based on augmented reality technology and the units included therein, and will not be described in detail here.
[0111] Further references Figure 3 , which shows a structural schematic diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0112] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.
[0113] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0114] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.
[0115] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0116] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0117] The above-mentioned computer-readable medium may be included in the above-mentioned device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: in response to receiving a user task query request for a target user, sends the cable fault inspection task information set corresponding to the above-mentioned user task query request to the head-mounted display device corresponding to the above-mentioned target user for selection by the above-mentioned target user; in response to receiving task selection information for each cable fault inspection task information in the above-mentioned cable fault inspection task information set, obtains the current environment image, user posture information, historical user posture information and a three-dimensional underground cable model; based on the above-mentioned current environment image, constructs a three-dimensional environment image; matches the above-mentioned user posture information and the above-mentioned historical user posture information to obtain task position matching information; in response to Determine whether the task location matching information satisfies preset matching conditions, fuse the three-dimensional environment image and the three-dimensional underground cable model to obtain a cable laying location image, and send the cable laying location image to the head-mounted display device, so that the target user can dig cable trenches and repair faulty cables; in response to receiving the cable equipment image sent by the head-mounted display device, perform fault identification processing on the cable equipment image based on the three-dimensional underground cable model to obtain a cable equipment fault information group; based on the cable equipment fault information group, update the cable equipment image to obtain a cable equipment fault mark image, and control the head-mounted display device to display the cable equipment fault mark image, so that the target user can locate and repair the cable fault.
[0118] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0119] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0120] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be set in a processor, for example, it may be described as: a processor including a sending unit, an acquisition unit, a construction unit, a matching unit, a fusion and sending unit, a fault identification unit and an update and control unit. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances. For example, the sending unit may also be described as "a unit that sends the cable fault inspection task information set corresponding to the above-mentioned user task query request to the head-mounted display device corresponding to the above-mentioned target user".
[0121] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0122] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.
Claims
1. A cable fault location method based on augmented reality technology, comprising: In response to receiving a user task query request for a target user, sending a cable fault inspection task information set corresponding to the user task query request to a head mounted display device corresponding to the target user for selection by the target user; In response to receiving task selection information for each cable fault inspection task information in the cable fault inspection task information set, acquiring a current environment image, user posture information, historical user posture information, and a three-dimensional underground cable model; Based on the current environment image, construct a three-dimensional environment image; Matching the user position information with the historical user position information to obtain task position matching information; In response to determining that the task location matching information satisfies a preset matching condition, the three-dimensional environment image and the three-dimensional underground cable model are fused to obtain a cable laying location image, and the cable laying location image is sent to the head mounted display device for the target user to dig a cable trench and repair a faulty cable; In response to receiving the cable device image sent by the head mounted display device, based on the three-dimensional underground cable model, performing fault identification processing on the cable device image to obtain a cable device fault information group, wherein each cable device fault information in the cable device fault information group includes cable device detection frame information; Based on the cable equipment fault information group, the cable equipment image is updated to obtain a cable equipment fault mark image, and the head mounted display device is controlled to display the cable equipment fault mark image so as to facilitate the target user to locate and repair the cable fault; The performing of fault identification processing on the cable equipment image to obtain a cable equipment fault information group includes: Performing brightness detection processing on the cable equipment image to obtain image brightness information; Performing image enhancement processing on the cable equipment image according to the image brightness information to obtain an enhanced cable equipment image; Based on the three-dimensional underground cable model, the enhanced cable equipment image is identified and processed to obtain a fault equipment location information group; Inputting the faulty device location information group and the enhanced cable device image into a pre-trained cable device fault diagnosis model to obtain a cable device fault type information group; Matching the fault device location information group and the cable device fault type information group to obtain a cable device fault information group; The updating process of the cable equipment image based on the cable equipment fault information group to obtain the cable equipment fault mark image includes: For each cable device fault information in the cable device fault information group, the following steps are performed: Based on the pre-built cable equipment operation and maintenance knowledge graph, determine the operation and maintenance method information corresponding to the cable equipment fault information; Aggregate the cable equipment fault information and the operation and maintenance method information to obtain cable equipment fault label information; Creating a rectangular element and a text element, wherein the rectangular element corresponds to the cable device detection frame information, and the text element corresponds to the cable device fault label information; The created rectangular elements and text elements are added to the cable equipment image to obtain a cable equipment fault mark image, wherein each text element is on the right side of the corresponding rectangular element.
2. The method according to claim 1, wherein: The matching process of the user position information and the historical user position information to obtain task position matching information includes: Determine a first position coordinate and a first direction angle corresponding to the user posture information; Determine a second position coordinate and a second direction angle corresponding to the historical user posture information; Performing matching processing on the first position coordinates and the second position coordinates to obtain first matching information; Performing matching processing on the first direction angle and the second direction angle to obtain second matching information; Task location matching information is generated based on the first matching information and the second matching information.
3. The method according to claim 1, wherein: The cable equipment fault diagnosis model is trained by the following steps: Acquire a historical cable equipment fault data set, a simulated cable equipment fault data set and an initial cable equipment fault diagnosis model, wherein each simulated cable equipment fault data in the simulated cable equipment fault data set is obtained by simulating the fault of the cable equipment according to the three-dimensional underground cable model; Based on the historical cable equipment fault data set and the simulated cable equipment fault data set, a sample cable fault data set is generated, wherein the sample cable fault data in the sample cable fault data set includes a sample fault equipment location information group, a sample cable equipment image and sample label data, and the sample cable equipment image is a cable equipment image after image enhancement processing; Based on the sample cable fault data set, the initial cable equipment fault diagnosis model is trained, and the trained initial cable equipment fault diagnosis model is determined as the cable equipment fault diagnosis model.
4. A cable fault location device based on augmented reality technology, comprising: a sending unit, configured to, in response to receiving a user task query request for a target user, send a cable fault inspection task information set corresponding to the user task query request to a head mounted display device corresponding to the target user for selection by the target user; an acquisition unit, configured to acquire a current environment image, user posture information, historical user posture information and a three-dimensional underground cable model in response to receiving task selection information for each cable fault inspection task information in the cable fault inspection task information set; A construction unit, configured to construct a three-dimensional environment image based on the current environment image; A matching unit, configured to perform matching processing on the user posture information and the historical user posture information to obtain task position matching information; a fusion and sending unit, configured to, in response to determining that the task location matching information satisfies a preset matching condition, fuse the three-dimensional environment image and the three-dimensional underground cable model to obtain a cable laying location image, and send the cable laying location image to the head mounted display device, so that the target user can dig a cable trench and repair a faulty cable; a fault identification unit configured to, in response to receiving the cable device image sent by the head mounted display device, perform fault identification processing on the cable device image based on the three-dimensional underground cable model to obtain a cable device fault information group, wherein each cable device fault information in the cable device fault information group includes cable device detection frame information; An updating and control unit, configured to update the cable equipment image based on the cable equipment fault information group to obtain a cable equipment fault mark image, and control the head mounted display device to display the cable equipment fault mark image so as to facilitate the target user to locate and repair the cable fault; The performing of fault identification processing on the cable equipment image to obtain a cable equipment fault information group includes: Performing brightness detection processing on the cable equipment image to obtain image brightness information; Performing image enhancement processing on the cable equipment image according to the image brightness information to obtain an enhanced cable equipment image; Based on the three-dimensional underground cable model, the enhanced cable equipment image is identified and processed to obtain a fault equipment location information group; Inputting the faulty device location information group and the enhanced cable device image into a pre-trained cable device fault diagnosis model to obtain a cable device fault type information group; Matching the fault device location information group and the cable device fault type information group to obtain a cable device fault information group; The updating process of the cable equipment image based on the cable equipment fault information group to obtain the cable equipment fault mark image includes: For each cable device fault information in the cable device fault information group, the following steps are performed: Based on the pre-built cable equipment operation and maintenance knowledge graph, determine the operation and maintenance method information corresponding to the cable equipment fault information; Aggregate the cable equipment fault information and the operation and maintenance method information to obtain cable equipment fault label information; Creating a rectangular element and a text element, wherein the rectangular element corresponds to the cable device detection frame information, and the text element corresponds to the cable device fault label information; The created rectangular elements and text elements are added to the cable equipment image to obtain a cable equipment fault mark image, wherein each text element is on the right side of the corresponding rectangular element.
5. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 3.
